You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@logging.apache.org by leventov <gi...@git.apache.org> on 2017/06/25 05:23:43 UTC
[GitHub] logging-log4j2 pull request #87: [LOG4J2-928] Mostly wait-free MemoryMappedF...
GitHub user leventov opened a pull request:
https://github.com/apache/logging-log4j2/pull/87
[LOG4J2-928] Mostly wait-free MemoryMappedFileManager
A pull request for https://issues.apache.org/jira/browse/LOG4J2-928.
This PR is not complete - it lacks tests and benchmarks (I'm going to add them in the next few weeks), but the core implementation is reviewable already.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/leventov/logging-log4j2 wait-free-MemoryMappedFileManager
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/logging-log4j2/pull/87.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #87
----
commit ed0a67085d197618845ce27eb8a4d30447522c0f
Author: leventov <le...@ya.ru>
Date: 2017-06-25T04:35:41Z
Mostly wait-free MemoryMappedFileManager
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] logging-log4j2 issue #87: [LOG4J2-928] Mostly wait-free MemoryMappedFileMana...
Posted by remkop <gi...@git.apache.org>.
Github user remkop commented on the issue:
https://github.com/apache/logging-log4j2/pull/87
As promised, here are the throughput results on enterprise hardware.
Hardware: total 64 cores (4x 16-core) Intel(R) Xeon(R) CPU E5-2697A v4 @ 2.60GHz
Java Hotspot 64-bit server 1.8.0_112
Results look good: as soon as we introduce concurrency, the lock-free MMap appender starts outperforming the other file appenders.
*1 thread*
```
{code}
-sh-4.2$ java -jar benchmarks.jar ".*Log4j2AppenderThroughput.*" -wi 5 -f 1 -i 10
Benchmark (loggerType) (messageSizeBytes) Mode Samples Score Error Units
o.a.l.l.p.j.Log4j2AppenderThroughputBenchmark.appender File 150 thrpt 10 1344494.961 ± 221110.545 ops/s
o.a.l.l.p.j.Log4j2AppenderThroughputBenchmark.appender RAF 150 thrpt 10 1563396.748 ± 12497.136 ops/s
o.a.l.l.p.j.Log4j2AppenderThroughputBenchmark.appender MMap 150 thrpt 10 1223215.440 ± 14661.487 ops/s
o.a.l.l.p.j.Log4j2AppenderThroughputBenchmark.end2end File 150 thrpt 10 1008537.505 ± 9213.181 ops/s
o.a.l.l.p.j.Log4j2AppenderThroughputBenchmark.end2end RAF 150 thrpt 10 998564.075 ± 12207.661 ops/s
o.a.l.l.p.j.Log4j2AppenderThroughputBenchmark.end2end MMap 150 thrpt 10 785114.881 ± 8827.768 ops/s
{code}
```
*4 threads*
```
{code}
-sh-4.2$ java -jar benchmarks.jar ".*Log4j2AppenderThroughput.*" -wi 5 -f 1 -i 10 -t 4
Benchmark (loggerType) (messageSizeBytes) Mode Samples Score Error Units
o.a.l.l.p.j.Log4j2AppenderThroughputBenchmark.appender File 150 thrpt 10 1678485.793 ± 260632.925 ops/s
o.a.l.l.p.j.Log4j2AppenderThroughputBenchmark.appender RAF 150 thrpt 10 1412574.559 ± 447410.040 ops/s
o.a.l.l.p.j.Log4j2AppenderThroughputBenchmark.appender MMap 150 thrpt 10 1386414.915 ± 333117.992 ops/s
o.a.l.l.p.j.Log4j2AppenderThroughputBenchmark.end2end File 150 thrpt 10 959662.994 ± 333541.548 ops/s
o.a.l.l.p.j.Log4j2AppenderThroughputBenchmark.end2end RAF 150 thrpt 10 929405.390 ± 66673.436 ops/s
o.a.l.l.p.j.Log4j2AppenderThroughputBenchmark.end2end MMap 150 thrpt 10 1013681.438 ± 155916.949 ops/s
{code}
```
*8 threads*
```
{code}
-sh-4.2$ java -jar benchmarks.jar ".*Log4j2AppenderThroughput.*" -wi 5 -f 1 -i 10 -t 8
Benchmark (loggerType) (messageSizeBytes) Mode Samples Score Error Units
o.a.l.l.p.j.Log4j2AppenderThroughputBenchmark.appender File 150 thrpt 10 1164538.492 ± 62482.965 ops/s
o.a.l.l.p.j.Log4j2AppenderThroughputBenchmark.appender RAF 150 thrpt 10 1194865.200 ± 72251.118 ops/s
o.a.l.l.p.j.Log4j2AppenderThroughputBenchmark.appender MMap 150 thrpt 10 1338960.644 ± 71104.991 ops/s
o.a.l.l.p.j.Log4j2AppenderThroughputBenchmark.end2end File 150 thrpt 10 944670.900 ± 69190.255 ops/s
o.a.l.l.p.j.Log4j2AppenderThroughputBenchmark.end2end RAF 150 thrpt 10 930248.735 ± 28523.086 ops/s
o.a.l.l.p.j.Log4j2AppenderThroughputBenchmark.end2end MMap 150 thrpt 10 998676.286 ± 36327.805 ops/s
{code}
```
*16 threads*
```
{code}
-sh-4.2$ java -jar benchmarks.jar ".*Log4j2AppenderThroughput.*" -wi 5 -f 1 -i 10 -t 16
Benchmark (loggerType) (messageSizeBytes) Mode Samples Score Error Units
o.a.l.l.p.j.Log4j2AppenderThroughputBenchmark.appender File 150 thrpt 10 1214786.965 ± 7217.940 ops/s
o.a.l.l.p.j.Log4j2AppenderThroughputBenchmark.appender RAF 150 thrpt 10 1192036.036 ± 13696.830 ops/s
o.a.l.l.p.j.Log4j2AppenderThroughputBenchmark.appender MMap 150 thrpt 10 1548961.297 ± 27141.206 ops/s
o.a.l.l.p.j.Log4j2AppenderThroughputBenchmark.end2end File 150 thrpt 10 1003235.059 ± 10365.921 ops/s
o.a.l.l.p.j.Log4j2AppenderThroughputBenchmark.end2end RAF 150 thrpt 10 994834.287 ± 9057.470 ops/s
o.a.l.l.p.j.Log4j2AppenderThroughputBenchmark.end2end MMap 150 thrpt 10 1323043.186 ± 19960.127 ops/s
{code}
```
*32 threads*
```
{code}
-sh-4.2$ java -jar benchmarks.jar ".*Log4j2AppenderThroughput.*" -wi 5 -f 1 -i 10 -t 32
Benchmark (loggerType) (messageSizeBytes) Mode Samples Score Error Units
o.a.l.l.p.j.Log4j2AppenderThroughputBenchmark.appender File 150 thrpt 10 1372729.015 ± 40762.524 ops/s
o.a.l.l.p.j.Log4j2AppenderThroughputBenchmark.appender RAF 150 thrpt 10 1185351.707 ± 16992.542 ops/s
o.a.l.l.p.j.Log4j2AppenderThroughputBenchmark.appender MMap 150 thrpt 10 1784377.615 ± 179359.793 ops/s
o.a.l.l.p.j.Log4j2AppenderThroughputBenchmark.end2end File 150 thrpt 10 918029.971 ± 38434.750 ops/s
o.a.l.l.p.j.Log4j2AppenderThroughputBenchmark.end2end RAF 150 thrpt 10 1122310.566 ± 152840.084 ops/s
o.a.l.l.p.j.Log4j2AppenderThroughputBenchmark.end2end MMap 150 thrpt 10 1898762.678 ± 81312.274 ops/s
{code}
```
*64 threads*
```
{code}
-sh-4.2$ java -jar benchmarks.jar ".*Log4j2AppenderThroughput.*" -wi 5 -f 1 -i 10 -t 64
Benchmark (loggerType) (messageSizeBytes) Mode Samples Score Error Units
o.a.l.l.p.j.Log4j2AppenderThroughputBenchmark.appender File 150 thrpt 10 1193371.846 ± 32893.485 ops/s
o.a.l.l.p.j.Log4j2AppenderThroughputBenchmark.appender RAF 150 thrpt 10 1231136.529 ± 26895.411 ops/s
o.a.l.l.p.j.Log4j2AppenderThroughputBenchmark.appender MMap 150 thrpt 10 1409277.639 ± 91900.204 ops/s
o.a.l.l.p.j.Log4j2AppenderThroughputBenchmark.end2end File 150 thrpt 10 881261.726 ± 35023.552 ops/s
o.a.l.l.p.j.Log4j2AppenderThroughputBenchmark.end2end RAF 150 thrpt 10 929691.738 ± 22616.873 ops/s
o.a.l.l.p.j.Log4j2AppenderThroughputBenchmark.end2end MMap 150 thrpt 10 1216108.561 ± 41321.530 ops/s
{code}
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] logging-log4j2 issue #87: [LOG4J2-928] Mostly wait-free MemoryMappedFileMana...
Posted by leventov <gi...@git.apache.org>.
Github user leventov commented on the issue:
https://github.com/apache/logging-log4j2/pull/87
@mikaelstaldal it is configured for at most 1 second latency. Maybe on your machine some pause takes more.
BTW jvm crash is not a mystery anymore, it was because of reads after buffer unmapping.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] logging-log4j2 issue #87: [LOG4J2-928] Mostly wait-free MemoryMappedFileMana...
Posted by remkop <gi...@git.apache.org>.
Github user remkop commented on the issue:
https://github.com/apache/logging-log4j2/pull/87
P.S. Roman, are you okay with submitting an [ICLA](http://www.apache.org/licenses/#clas)? I am still reviewing the patch but we need the ICLA before it can be merged. If you put "Apache Logging" in the form field "notify project", the secretary will let us know when your ICLA is on file.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] logging-log4j2 issue #87: [LOG4J2-928] Mostly wait-free MemoryMappedFileMana...
Posted by garydgregory <gi...@git.apache.org>.
Github user garydgregory commented on the issue:
https://github.com/apache/logging-log4j2/pull/87
Most folks are unlikely to look to closely at the docs. IMO we should make the docs as good as we can AND come up with default values that cover 80% of the cases.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] logging-log4j2 issue #87: [LOG4J2-928] Mostly wait-free MemoryMappedFileMana...
Posted by mikaelstaldal <gi...@git.apache.org>.
Github user mikaelstaldal commented on the issue:
https://github.com/apache/logging-log4j2/pull/87
BTW, the test fails on Java 9:
```
2017-07-02 14:19:04,650 org.apache.logging.log4j.perf.jmh.Log4j2AppenderLatencyBenchmark.end2end-jmh-worker-1 ERROR Memo
ryMappedFileManager target/MemoryMappedFileAppenderTest.log Unable to remap: java.lang.reflect.InaccessibleObjectException: Unable to make public jdk.internal.ref.Cleaner java.nio.DirectByteBuffer.cleaner() accessible: module java.base does not "opens java.nio" to unnamed module @1417a6e9 java.lang.reflect.InaccessibleObjectException: Unable to make public jdk.internal.ref.Cleaner java.nio.DirectByteBuffer.cleaner() accessible: module java.base does not "opens java.nio" to unnamed module @1417a6e9
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] logging-log4j2 issue #87: [LOG4J2-928] Mostly wait-free MemoryMappedFileMana...
Posted by mikaelstaldal <gi...@git.apache.org>.
Github user mikaelstaldal commented on the issue:
https://github.com/apache/logging-log4j2/pull/87
I tried to run:
```
java -jar log4j-perf/target/benchmarks.jar ".*Log4j2AppenderLatencyBenchmark.end2end" -f 1 -wi 1 -i 30 -t 2 -p loggerType=MMap
```
but got this result:
```
# VM invoker: /opt/jvm/jdk1.8.0_131/jre/bin/java
# VM options: <none>
# Warmup: 1 iterations, 1 s each
# Measurement: 30 iterations, 1 s each
# Timeout: 10 min per iteration
# Threads: 2 threads, will synchronize iterations
# Benchmark mode: Throughput, ops/time
# Benchmark: org.apache.logging.log4j.perf.jmh.Log4j2AppenderLatencyBenchmark.end2end
# Parameters: (loggerType = MMap)
# Run progress: 0.00% complete, ETA 00:00:31
# Fork: 1 of 1
# Warmup Iteration 1: 818183.651 ops/s
Iteration 1: 677355.347 ops/s
Iteration 2: <failure>
java.lang.ArrayIndexOutOfBoundsException: value outside of histogram covered range. Caused by: java.lang.IndexOutOfBoundsException: index 21813
at org.HdrHistogram.AbstractHistogram.handleRecordException(AbstractHistogram.java:492)
at org.HdrHistogram.AbstractHistogram.recordSingleValue(AbstractHistogram.java:484)
at org.HdrHistogram.AbstractHistogram.recordValue(AbstractHistogram.java:397)
at org.apache.logging.log4j.perf.jmh.Log4j2AppenderLatencyBenchmark.end2end(Log4j2AppenderLatencyBenchmark.java:106)
at org.apache.logging.log4j.perf.jmh.generated.Log4j2AppenderLatencyBenchmark_end2end.end2end_thrpt_jmhStub(Log4j2AppenderLatencyBenchmark_end2end.java:125)
at org.apache.logging.log4j.perf.jmh.generated.Log4j2AppenderLatencyBenchmark_end2end.end2end_Throughput(Log4j2AppenderLatencyBenchmark_end2end.java:70)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.openjdk.jmh.runner.LoopBenchmarkHandler$BenchmarkTask.call(LoopBenchmarkHandler.java:198)
at org.openjdk.jmh.runner.LoopBenchmarkHandler$BenchmarkTask.call(LoopBenchmarkHandler.java:180)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
# Run complete. Total time: 00:00:04
Benchmark Mode Samples Score Error Units
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] logging-log4j2 issue #87: [LOG4J2-928] Mostly wait-free MemoryMappedFileMana...
Posted by leventov <gi...@git.apache.org>.
Github user leventov commented on the issue:
https://github.com/apache/logging-log4j2/pull/87
<h3>Throughput comparison</h3>
First number is the number of threads, "150" is the message size.
Before this PR
```
1 - o.a.l.l.p.j.Log4j2AppenderComparisonBenchmark.end2endMMap 150 thrpt 10 988106.677 ± 44561.265 ops/s
2 - o.a.l.l.p.j.Log4j2AppenderComparisonBenchmark.end2endMMap 150 thrpt 10 1378707.634 ± 45234.050 ops/s
3 - o.a.l.l.p.j.Log4j2AppenderComparisonBenchmark.end2endMMap 150 thrpt 10 1507720.952 ± 42527.798 ops/s
4 - o.a.l.l.p.j.Log4j2AppenderComparisonBenchmark.end2endMMap 150 thrpt 10 1615561.064 ± 10197.025 ops/s
1 - o.a.l.l.p.j.Log4j2AppenderComparisonBenchmark.appenderMMap 150 thrpt 10 1406710.542 ± 53654.749 ops/s
2 - o.a.l.l.p.j.Log4j2AppenderComparisonBenchmark.appenderMMap 150 thrpt 10 2032867.062 ± 33273.059 ops/s
3 - o.a.l.l.p.j.Log4j2AppenderComparisonBenchmark.appenderMMap 150 thrpt 10 2182597.197 ± 19446.011 ops/s
4 - o.a.l.l.p.j.Log4j2AppenderComparisonBenchmark.appenderMMap 150 thrpt 10 2128503.272 ± 40132.820 ops/s
```
With this PR
```
1 - o.a.l.l.p.j.Log4j2AppenderComparisonBenchmark.end2endMMap 150 thrpt 10 848596.439 ± 29081.144 ops/s
2 - o.a.l.l.p.j.Log4j2AppenderComparisonBenchmark.end2endMMap 150 thrpt 10 1302047.611 ± 21627.635 ops/s
3 - o.a.l.l.p.j.Log4j2AppenderComparisonBenchmark.end2endMMap 150 thrpt 10 1702653.587 ± 36385.194 ops/s
4 - o.a.l.l.p.j.Log4j2AppenderComparisonBenchmark.end2endMMap 150 thrpt 15 1880038.309 ± 35208.682 ops/s
1 - o.a.l.l.p.j.Log4j2AppenderComparisonBenchmark.appenderMMap 150 thrpt 10 1283001.528 ± 40095.635 ops/s
2 - o.a.l.l.p.j.Log4j2AppenderComparisonBenchmark.appenderMMap 150 thrpt 10 1803333.701 ± 27412.900 ops/s
3 - o.a.l.l.p.j.Log4j2AppenderComparisonBenchmark.appenderMMap 150 thrpt 10 2506996.207 ± 40209.437 ops/s
4 - o.a.l.l.p.j.Log4j2AppenderComparisonBenchmark.appenderMMap 150 thrpt 10 2723771.920 ± 103479.124 ops/s
```
On the same MacBook with 4 cores.
You can see that with 1 or 2 threads older version is faster, but it's the limit of it's scalability, and with 3 or 4 threads the wait-free version takes over.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] logging-log4j2 issue #87: [LOG4J2-928] Mostly wait-free MemoryMappedFileMana...
Posted by remkop <gi...@git.apache.org>.
Github user remkop commented on the issue:
https://github.com/apache/logging-log4j2/pull/87
Roman, thanks for the feedback and the benchmarks.
* Understood about keeping the new package-private classes in the same package as MemoryMappedFilemanager. No problem.
* Thanks for updating to use the new `narrow` method and for clarifying how extremely long stack traces are handled.
* I still think Log4j components should never throw an Error. I understand the rationale behind throwing Errors in java.util.concurrent: if these low level components detect an anomaly it means that the JVM cannot guarantee sane behaviour for the application. Our case is different because Log4j is an auxiliary library: logging may fail, but the application should be able to continue.
It is up to the application to decide whether they want to receive logging exceptions or not, and users can configure their choice with the `ignoreExceptions` attribute on `MemoryMappedFileAppender`. New implementations of MemoryMappedFileAppender should respect this contract.
About testing latency:
Please take a look at /log4j-core/src/test/java/org/apache/logging/log4j/core/async/perftest/ResponseTimeTest.java
ResponseTimeTest incorporates various lessons I learned from Gil Tene's [presentations](http://www.infoq.com/presentations/latency-response-time) on measuring latency as well as discussions on the Mechanical Sympathy mailing list. (The inner Pacer class is Gil's.) It distinguishes between response time and service time (see the [sidebar on the Log4j perf page](https://logging.apache.org/log4j/2.x/performance.html#benchmarks)), and it allows us to control the load. This is important because latency behaviour is often different under various workloads.
I am not a fan of using JMH for measuring latency because it only measures service time, not response time, and there is no way to control the workload, it just runs at max speed. (Gil explicitly mentioned we probably don't want to measure latency under maximum load, when the system is under pressure and latency becomes unpredictable).
Also, it looks like Log4j2AppenderLatencyBenchmark also measures how long it takes to generate a random number, which is probably not the intention.
The throughput numbers look very promising, I will try to run these tests on enterprise hardware this week when I have a chance, and report back the results.
All in all, this is great stuff! Keep 'em coming!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] logging-log4j2 issue #87: [LOG4J2-928] Mostly wait-free MemoryMappedFileMana...
Posted by jvz <gi...@git.apache.org>.
Github user jvz commented on the issue:
https://github.com/apache/logging-log4j2/pull/87
The Java 9 stuff sounds interesting. We may need to add a multi-version JAR build for log4j-core as well (we have one for log4j-api so far) if it can't reasonably done via reflection.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] logging-log4j2 issue #87: [LOG4J2-928] Mostly wait-free MemoryMappedFileMana...
Posted by leventov <gi...@git.apache.org>.
Github user leventov commented on the issue:
https://github.com/apache/logging-log4j2/pull/87
Also recent addition of "write touch" seems to improve throughput considerably, so the numbers posted above are irrelevant now.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] logging-log4j2 issue #87: [LOG4J2-928] Mostly wait-free MemoryMappedFileMana...
Posted by leventov <gi...@git.apache.org>.
Github user leventov commented on the issue:
https://github.com/apache/logging-log4j2/pull/87
@remkop I have not competed this PR yet, there are a few more things need to be done. I will try to complete PR in the next weekend. Also I've just sent ICLA to secretary@apache.org
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] logging-log4j2 issue #87: [LOG4J2-928] Mostly wait-free MemoryMappedFileMana...
Posted by remkop <gi...@git.apache.org>.
Github user remkop commented on the issue:
https://github.com/apache/logging-log4j2/pull/87
Roman, I started to review your patch. Initial feedback below (still reading):
* MemoryMappedFileManager::switchRegion should not throw Error: generally a logging problem should not bring down the application. See also the {{ignoreExceptions}} [configuration|https://logging.apache.org/log4j/2.x/manual/appenders.html#MemoryMappedFileAppender] attribute that allows applications to choose to receive logging exceptions.
* What do you think of the idea of moving some of the newly added classes to the core.async package? ThreadHints, ThreadHintsMH, UninterruptibleCountDownLatch and UnlockedAwaitableReentrantLock. Perhaps also some of MemoryMappedfileManager's inner classes (need to look more at these)?
* in MemoryMappedfileManager::getFileManager, please call AbstractManager::narrow on the result of super.getManager(). This will show users a better error message when log4j2 is misconfigured (LOG4J2-1908). This is a recent change I made, may have introduced a (small) merge conflict, sorry about that.
* Question: The javadoc for MemoryMappedFileManager::SPIN_WAIT_MINIMIZING_LOCKED_WRITE_THRESHOLD mentions a 32 KB threshold for stack traces. What happens if a user logs a stack trace exceeding this length?
... still reading, also looking forward to tests & perf results.
On the topic of perf testing: I'd be interested to see a throughput comparison of the new MemoryMappedFileAppender vs. RandomAccessFileAppender and FileAppender with a varying number of logging threads (1, 2, 4, 8, 16, 32, 64 if possible).
A throughput comparison with Async Loggers would also be interesting but be aware that JMH quickly fills up the disruptor ringbuffer. To solve this, configure with a non-existing appender (so the background thread essentially logs to /dev/null). See /log4j-perf/src/main/java/org/apache/logging/log4j/perf/jmh/AsyncLoggersBenchmark.java
Some indication of latency behaviour would also be interesting.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] logging-log4j2 issue #87: [LOG4J2-928] Mostly wait-free MemoryMappedFileMana...
Posted by remkop <gi...@git.apache.org>.
Github user remkop commented on the issue:
https://github.com/apache/logging-log4j2/pull/87
Following new developments with interest.
Looking forward to javadoc on the WriteTouch stuff. I just very briefly scanned the commit on my phone while commuting to work, but didn't see anything Java 9-ish, perhaps I misunderstood. What does this do?
Good point about StringBuilderEncoder and MemoryMappedFileManager buffer sizes. We have several options here: we can match up the buffer sizes so they are always the same, or we can document in the MemoryMappedFile appender manual that it is advisable for users to increase the StringBuilderEncoder buffer size (I believe there is a system property). Any preference anyone?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] logging-log4j2 pull request #87: [LOG4J2-928] Mostly wait-free MemoryMappedF...
Posted by jvz <gi...@git.apache.org>.
Github user jvz commented on a diff in the pull request:
https://github.com/apache/logging-log4j2/pull/87#discussion_r124592044
--- Diff: log4j-core/src/main/java/org/apache/logging/log4j/core/appender/ThreadHints.java ---
@@ -0,0 +1,63 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache license, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the license for the specific language governing permissions and
+ * limitations under the license.
+ */
+package org.apache.logging.log4j.core.appender;
+
+/**
+ * This class captures possible performance hints that may be used by some
+ * runtimes to improve code performance. It is intended to capture hinting
+ * behaviors that are implemented in or anticipated to be spec'ed under the
+ * java.lang.Thread class in some Java SE versions, but missing in prior
+ * versions.
+ *
+ * Written by Gil Tene, and released to the public domain,
+ * as explained at http://creativecommons.org/publicdomain/zero/1.0/
+ */
+final class ThreadHints {
+
+ private static final boolean okToInvokeThreadHintsMHCaller;
+
+ static {
+ boolean okToInvokeMHCaller;
+
+ try {
+ Class.forName("java.lang.invoke.MethodHandle");
--- End diff --
Two things here:
1. We have a LoaderUtil class for loading classes which tends to work a bit better than using `Class.forName`.
2. We're on Java 7 minimum, so this class is for sure available. No need for reflection here.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] logging-log4j2 issue #87: [LOG4J2-928] Mostly wait-free MemoryMappedFileMana...
Posted by remkop <gi...@git.apache.org>.
Github user remkop commented on the issue:
https://github.com/apache/logging-log4j2/pull/87
Roman, one more thing: since this is a non-trivial patch, would you mind submitting an [ICLA](http://www.apache.org/licenses/#clas)?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] logging-log4j2 issue #87: [LOG4J2-928] Mostly wait-free MemoryMappedFileMana...
Posted by mikaelstaldal <gi...@git.apache.org>.
Github user mikaelstaldal commented on the issue:
https://github.com/apache/logging-log4j2/pull/87
Then I suggest that you raise the maximum latency, since it can obviously happen.
And maybe this long latency on my machine is a flaw in the implementation?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] logging-log4j2 issue #87: [LOG4J2-928] Mostly wait-free MemoryMappedFileMana...
Posted by remkop <gi...@git.apache.org>.
Github user remkop commented on the issue:
https://github.com/apache/logging-log4j2/pull/87
I don't have access to hardware with that version of Java. I am fairly sure that 131 doesn't have performance improvements (we checked whether we should upgrade but it was too close to our go-live and 131 didn't have anything particular we needed).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] logging-log4j2 issue #87: [LOG4J2-928] Mostly wait-free MemoryMappedFileMana...
Posted by leventov <gi...@git.apache.org>.
Github user leventov commented on the issue:
https://github.com/apache/logging-log4j2/pull/87
Things still TODO / TBD in this PR:
- **Why on extreme loads the new version has significantly worse tail latency?**
- StringBuilderEncoder.DEFAULT_BYTE_BUFFER_SIZE (8k) effectively overrides MemoryMappedFileManager.DEFAULT_REGION_LENGTH (32k), is that reasonable?
- Java 9:
- Buffer unmapping, as pointed here: https://github.com/apache/logging-log4j2/pull/87#issuecomment-312488652
- WriteTouch should have an opportunistic impl based on VarHandles.
- Not doing stuff "in background" on 1-core machines; probably not doing write touch on "small" machines (< 4 cores?)
- Cleanup and comments to the recently added write touch and background stuff
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] logging-log4j2 pull request #87: [LOG4J2-928] Mostly wait-free MemoryMappedF...
Posted by leventov <gi...@git.apache.org>.
Github user leventov commented on a diff in the pull request:
https://github.com/apache/logging-log4j2/pull/87#discussion_r125168798
--- Diff: log4j-core/src/main/java/org/apache/logging/log4j/core/appender/ThreadHints.java ---
@@ -0,0 +1,63 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache license, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the license for the specific language governing permissions and
+ * limitations under the license.
+ */
+package org.apache.logging.log4j.core.appender;
+
+/**
+ * This class captures possible performance hints that may be used by some
+ * runtimes to improve code performance. It is intended to capture hinting
+ * behaviors that are implemented in or anticipated to be spec'ed under the
+ * java.lang.Thread class in some Java SE versions, but missing in prior
+ * versions.
+ *
+ * Written by Gil Tene, and released to the public domain,
+ * as explained at http://creativecommons.org/publicdomain/zero/1.0/
+ */
+final class ThreadHints {
+
+ private static final boolean okToInvokeThreadHintsMHCaller;
+
+ static {
+ boolean okToInvokeMHCaller;
+
+ try {
+ Class.forName("java.lang.invoke.MethodHandle");
--- End diff --
@jvz thanks for note, removed ThreadHintsMH
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] logging-log4j2 issue #87: [LOG4J2-928] Mostly wait-free MemoryMappedFileMana...
Posted by remkop <gi...@git.apache.org>.
Github user remkop commented on the issue:
https://github.com/apache/logging-log4j2/pull/87
@leventov We may have a Log4j 2.9 release starting as soon as this weekend. Let me know if you want to make the remaining changes to complete this PR for merging into master before that release.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] logging-log4j2 issue #87: [LOG4J2-928] Mostly wait-free MemoryMappedFileMana...
Posted by leventov <gi...@git.apache.org>.
Github user leventov commented on the issue:
https://github.com/apache/logging-log4j2/pull/87
@remkop note that this PR is primarily about _latency_, not throughput at all. I will do some testing, but if the throughput will degrade (that I expect in a single-thread case), and latency with improve, I consider this a success.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] logging-log4j2 issue #87: [LOG4J2-928] Mostly wait-free MemoryMappedFileMana...
Posted by jvz <gi...@git.apache.org>.
Github user jvz commented on the issue:
https://github.com/apache/logging-log4j2/pull/87
I'll try to run those benchmarks on an AWS instance next week, though that's not an ideal place to benchmark due to noisy neighbours and that sort of thing.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] logging-log4j2 issue #87: [LOG4J2-928] Mostly wait-free MemoryMappedFileMana...
Posted by leventov <gi...@git.apache.org>.
Github user leventov commented on the issue:
https://github.com/apache/logging-log4j2/pull/87
@remkop
> MemoryMappedFileManager::switchRegion and ::closeAndSwitchRegionWhileLocked should not throw Error: generally a logging problem should not bring down the application. See also the {{ignoreExceptions}} [configuration|https://logging.apache.org/log4j/2.x/manual/appenders.html#MemoryMappedFileAppender] attribute that allows applications to choose to receive logging exceptions.
I follow the approach of java.util.concurrent internals here, where `throw new Error(...)` is used in the face of unrecoverable and totally unexpected errors. the alternative is `AssertionError`, or `assert` statements.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] logging-log4j2 issue #87: [LOG4J2-928] Mostly wait-free MemoryMappedFileMana...
Posted by leventov <gi...@git.apache.org>.
Github user leventov commented on the issue:
https://github.com/apache/logging-log4j2/pull/87
New latency benchmarking results "done right" (with Pacer), On 4-core MBP with SSD, benchmarking with 3 threads:
- [200 messages/sec/thread](https://cdn.rawgit.com/leventov/eee84006a53dfb8d2acaebfe126bb499/raw/805c548d1f41e49e67b6815dde84d92be0568012/load200.html)
- [2k messages/sec/thread](https://cdn.rawgit.com/leventov/eee84006a53dfb8d2acaebfe126bb499/raw/805c548d1f41e49e67b6815dde84d92be0568012/load2k.html)
- [20k messages/sec/thread](https://cdn.rawgit.com/leventov/eee84006a53dfb8d2acaebfe126bb499/raw/805c548d1f41e49e67b6815dde84d92be0568012/load20k.html)
- [100k messages/sec/thread](https://cdn.rawgit.com/leventov/eee84006a53dfb8d2acaebfe126bb499/raw/805c548d1f41e49e67b6815dde84d92be0568012/load100k.html)
I still use JMH to piggy-back threading, warmup, setup/teardown, parameter setting, etc.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] logging-log4j2 issue #87: [LOG4J2-928] Mostly wait-free MemoryMappedFileMana...
Posted by garydgregory <gi...@git.apache.org>.
Github user garydgregory commented on the issue:
https://github.com/apache/logging-log4j2/pull/87
Thank you for running the benchmarks. Is this worth doing over on the
_current_ Java 8 runtime (_131)?
Gary
On Wed, Jul 5, 2017 at 6:59 AM, Remko Popma <no...@github.com>
wrote:
> As promised, here are the throughput results on enterprise hardware.
>
> Hardware: total 64 cores (4x 16-core) Intel(R) Xeon(R) CPU E5-2697A v4 @
> 2.60GHz
> Java Hotspot 64-bit server 1.8.0_112
>
> Results look good: as soon as we introduce concurrency, the lock-free MMap
> appender starts outperforming the other file appenders.
>
> *1 thread*
>
> {code}
> -sh-4.2$ java -jar benchmarks.jar ".*Log4j2AppenderThroughput.*" -wi 5 -f 1 -i 10
>
> Benchmark (loggerType) (messageSizeBytes) Mode Samples Score Error Units
> o.a.l.l.p.j.Log4j2AppenderThroughputBenchmark.appender File 150 thrpt 10 1344494.961 ± 221110.545 ops/s
> o.a.l.l.p.j.Log4j2AppenderThroughputBenchmark.appender RAF 150 thrpt 10 1563396.748 ± 12497.136 ops/s
> o.a.l.l.p.j.Log4j2AppenderThroughputBenchmark.appender MMap 150 thrpt 10 1223215.440 ± 14661.487 ops/s
> o.a.l.l.p.j.Log4j2AppenderThroughputBenchmark.end2end File 150 thrpt 10 1008537.505 ± 9213.181 ops/s
> o.a.l.l.p.j.Log4j2AppenderThroughputBenchmark.end2end RAF 150 thrpt 10 998564.075 ± 12207.661 ops/s
> o.a.l.l.p.j.Log4j2AppenderThroughputBenchmark.end2end MMap 150 thrpt 10 785114.881 ± 8827.768 ops/s
> {code}
>
> *4 threads*
>
> {code}
> -sh-4.2$ java -jar benchmarks.jar ".*Log4j2AppenderThroughput.*" -wi 5 -f 1 -i 10 -t 4
>
> Benchmark (loggerType) (messageSizeBytes) Mode Samples Score Error Units
> o.a.l.l.p.j.Log4j2AppenderThroughputBenchmark.appender File 150 thrpt 10 1678485.793 ± 260632.925 ops/s
> o.a.l.l.p.j.Log4j2AppenderThroughputBenchmark.appender RAF 150 thrpt 10 1412574.559 ± 447410.040 ops/s
> o.a.l.l.p.j.Log4j2AppenderThroughputBenchmark.appender MMap 150 thrpt 10 1386414.915 ± 333117.992 ops/s
> o.a.l.l.p.j.Log4j2AppenderThroughputBenchmark.end2end File 150 thrpt 10 959662.994 ± 333541.548 ops/s
> o.a.l.l.p.j.Log4j2AppenderThroughputBenchmark.end2end RAF 150 thrpt 10 929405.390 ± 66673.436 ops/s
> o.a.l.l.p.j.Log4j2AppenderThroughputBenchmark.end2end MMap 150 thrpt 10 1013681.438 ± 155916.949 ops/s
> {code}
>
> *8 threads*
>
> {code}
> -sh-4.2$ java -jar benchmarks.jar ".*Log4j2AppenderThroughput.*" -wi 5 -f 1 -i 10 -t 8
>
> Benchmark (loggerType) (messageSizeBytes) Mode Samples Score Error Units
> o.a.l.l.p.j.Log4j2AppenderThroughputBenchmark.appender File 150 thrpt 10 1164538.492 ± 62482.965 ops/s
> o.a.l.l.p.j.Log4j2AppenderThroughputBenchmark.appender RAF 150 thrpt 10 1194865.200 ± 72251.118 ops/s
> o.a.l.l.p.j.Log4j2AppenderThroughputBenchmark.appender MMap 150 thrpt 10 1338960.644 ± 71104.991 ops/s
> o.a.l.l.p.j.Log4j2AppenderThroughputBenchmark.end2end File 150 thrpt 10 944670.900 ± 69190.255 ops/s
> o.a.l.l.p.j.Log4j2AppenderThroughputBenchmark.end2end RAF 150 thrpt 10 930248.735 ± 28523.086 ops/s
> o.a.l.l.p.j.Log4j2AppenderThroughputBenchmark.end2end MMap 150 thrpt 10 998676.286 ± 36327.805 ops/s
> {code}
>
> *16 threads*
>
> {code}
> -sh-4.2$ java -jar benchmarks.jar ".*Log4j2AppenderThroughput.*" -wi 5 -f 1 -i 10 -t 16
>
> Benchmark (loggerType) (messageSizeBytes) Mode Samples Score Error Units
> o.a.l.l.p.j.Log4j2AppenderThroughputBenchmark.appender File 150 thrpt 10 1214786.965 ± 7217.940 ops/s
> o.a.l.l.p.j.Log4j2AppenderThroughputBenchmark.appender RAF 150 thrpt 10 1192036.036 ± 13696.830 ops/s
> o.a.l.l.p.j.Log4j2AppenderThroughputBenchmark.appender MMap 150 thrpt 10 1548961.297 ± 27141.206 ops/s
> o.a.l.l.p.j.Log4j2AppenderThroughputBenchmark.end2end File 150 thrpt 10 1003235.059 ± 10365.921 ops/s
> o.a.l.l.p.j.Log4j2AppenderThroughputBenchmark.end2end RAF 150 thrpt 10 994834.287 ± 9057.470 ops/s
> o.a.l.l.p.j.Log4j2AppenderThroughputBenchmark.end2end MMap 150 thrpt 10 1323043.186 ± 19960.127 ops/s
> {code}
>
> *32 threads*
>
> {code}
> -sh-4.2$ java -jar benchmarks.jar ".*Log4j2AppenderThroughput.*" -wi 5 -f 1 -i 10 -t 32
>
> Benchmark (loggerType) (messageSizeBytes) Mode Samples Score Error Units
> o.a.l.l.p.j.Log4j2AppenderThroughputBenchmark.appender File 150 thrpt 10 1372729.015 ± 40762.524 ops/s
> o.a.l.l.p.j.Log4j2AppenderThroughputBenchmark.appender RAF 150 thrpt 10 1185351.707 ± 16992.542 ops/s
> o.a.l.l.p.j.Log4j2AppenderThroughputBenchmark.appender MMap 150 thrpt 10 1784377.615 ± 179359.793 ops/s
> o.a.l.l.p.j.Log4j2AppenderThroughputBenchmark.end2end File 150 thrpt 10 918029.971 ± 38434.750 ops/s
> o.a.l.l.p.j.Log4j2AppenderThroughputBenchmark.end2end RAF 150 thrpt 10 1122310.566 ± 152840.084 ops/s
> o.a.l.l.p.j.Log4j2AppenderThroughputBenchmark.end2end MMap 150 thrpt 10 1898762.678 ± 81312.274 ops/s
> {code}
>
> *64 threads*
>
> {code}
> -sh-4.2$ java -jar benchmarks.jar ".*Log4j2AppenderThroughput.*" -wi 5 -f 1 -i 10 -t 64
>
> Benchmark (loggerType) (messageSizeBytes) Mode Samples Score Error Units
> o.a.l.l.p.j.Log4j2AppenderThroughputBenchmark.appender File 150 thrpt 10 1193371.846 ± 32893.485 ops/s
> o.a.l.l.p.j.Log4j2AppenderThroughputBenchmark.appender RAF 150 thrpt 10 1231136.529 ± 26895.411 ops/s
> o.a.l.l.p.j.Log4j2AppenderThroughputBenchmark.appender MMap 150 thrpt 10 1409277.639 ± 91900.204 ops/s
> o.a.l.l.p.j.Log4j2AppenderThroughputBenchmark.end2end File 150 thrpt 10 881261.726 ± 35023.552 ops/s
> o.a.l.l.p.j.Log4j2AppenderThroughputBenchmark.end2end RAF 150 thrpt 10 929691.738 ± 22616.873 ops/s
> o.a.l.l.p.j.Log4j2AppenderThroughputBenchmark.end2end MMap 150 thrpt 10 1216108.561 ± 41321.530 ops/s
> {code}
>
> —
> You are receiving this because you are subscribed to this thread.
> Reply to this email directly, view it on GitHub
> <https://github.com/apache/logging-log4j2/pull/87#issuecomment-313111330>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/ABIfN9oh2p20v8fnX4yBVzi-pzelEgr1ks5sK5bPgaJpZM4OEhNj>
> .
>
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] logging-log4j2 issue #87: [LOG4J2-928] Mostly wait-free MemoryMappedFileMana...
Posted by remkop <gi...@git.apache.org>.
Github user remkop commented on the issue:
https://github.com/apache/logging-log4j2/pull/87
@leventov The log4j community is discussing releasing a 2.9 version soon (in July). I would like to merge your contribution but have you had a chance to look at the [ICLA](http://www.apache.org/licenses/#clas)?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] logging-log4j2 issue #87: [LOG4J2-928] Mostly wait-free MemoryMappedFileMana...
Posted by leventov <gi...@git.apache.org>.
Github user leventov commented on the issue:
https://github.com/apache/logging-log4j2/pull/87
<h3>Latency comparison</h3>
```
java -jar log4j-perf/target/benchmarks.jar ".*Log4j2AppenderLatencyBenchmark.end2end" -f 1 -wi 1 -i 30 -t 2 -p loggerType=MMap
```
On my MBP with 4 cores and SSD.
https://cdn.rawgit.com/leventov/288f6ff4ea960998e6479001976b0755/raw/aeb2f65560241deaa9394ff1ef8044bb961ea37e/plotFiles.html (red line is before this PR, blue line is this PR)
Tail latency is much better than it used to be, but worse than I expected. Calling `MappedByteBuffer.load()` significantly improves latency around 99.99-99.999 percentiles, but dramatically drops the extreme tail latency, from 4.5 milliseconds to ~300 milliseconds. I don't know the reason for that.
Also quite "fun" than hand-rolled version of MappedByteBuffer.load(): https://github.com/leventov/logging-log4j2/tree/wait-free-MemoryMappedFileManager-crash crashes the JVM when I try to run the same command
```
java -jar log4j-perf/target/benchmarks.jar ".*Log4j2AppenderLatencyBenchmark.end2end" -f 1 -wi 1 -i 30 -t 2 -p loggerType=MMap
```
Could somebody else please try it on other machines?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] logging-log4j2 issue #87: [LOG4J2-928] Mostly wait-free MemoryMappedFileMana...
Posted by jvz <gi...@git.apache.org>.
Github user jvz commented on the issue:
https://github.com/apache/logging-log4j2/pull/87
Thanks, we got notification of the ICLA.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---