You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@bookkeeper.apache.org by GitBox <gi...@apache.org> on 2021/11/05 06:24:08 UTC
[GitHub] [bookkeeper] wuzhanpeng opened a new issue #2878: Any suggestions for optimizing rocksdb parameters in huge throughput scenarios?
wuzhanpeng opened a new issue #2878:
URL: https://github.com/apache/bookkeeper/issues/2878
**QUESTION**
In our usage scenario, in the case of high-throughput writes, we found that the cpu utilization of rokskdb:low:XX thread can easily reach 100%. Are there any good suggestions for optimizing the rocksdb parameters in this situation?
As you can see from the screenshot:
We have tried turning off the compression, but nothing seems to have changed.
```
dbStorage_rocksDB_lz4CompressionEnabled=false
```
Any suggestions would be greatly appreciated~
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@bookkeeper.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bookkeeper] wuzhanpeng commented on issue #2878: Any suggestions for optimizing rocksdb parameters in huge throughput scenarios?
Posted by GitBox <gi...@apache.org>.
wuzhanpeng commented on issue #2878:
URL: https://github.com/apache/bookkeeper/issues/2878#issuecomment-961656004
@merlimat @eolivelli @dlg99 Would you mind to take a look?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@bookkeeper.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bookkeeper] dlg99 commented on issue #2878: Any suggestions for optimizing rocksdb parameters in huge throughput scenarios?
Posted by GitBox <gi...@apache.org>.
dlg99 commented on issue #2878:
URL: https://github.com/apache/bookkeeper/issues/2878#issuecomment-1018109001
@wuzhanpeng I suggest starting with profiling.
JFR (java flight recorder) is packaged with the JDK. It is an easy to use profiler (cpu/memory/GC).
In case you haven't used it:
```jcmd <PID_HERE> JFR.start duration=60s settings=profile filename=flight.jfr```
This will collect the profile into the file. If the systems is too busy the profile can get quite large (but it compresses well), you can try to decrease duration.
In some cases longer durations help.
this should work on java 11
pre-java11 requires
`jcmd <PID_HERE> VM.unlock_commercial_features`
before running jfr
to find the pid: run `jcmd`
the jfr file can be opened with jmc (UI app) https://github.com/openjdk/jmc#downloading-builds or https://www.oracle.com/java/technologies/javase/products-jmc8-downloads.html (used to be included in the jdk)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@bookkeeper.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bookkeeper] wuzhanpeng commented on issue #2878: Any suggestions for optimizing rocksdb parameters in huge throughput scenarios?
Posted by GitBox <gi...@apache.org>.
wuzhanpeng commented on issue #2878:
URL: https://github.com/apache/bookkeeper/issues/2878#issuecomment-961656004
@merlimat @eolivelli @dlg99 Would you mind to take a look?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@bookkeeper.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bookkeeper] wuzhanpeng commented on issue #2878: Any suggestions for optimizing rocksdb parameters in huge throughput scenarios?
Posted by GitBox <gi...@apache.org>.
wuzhanpeng commented on issue #2878:
URL: https://github.com/apache/bookkeeper/issues/2878#issuecomment-961656004
@merlimat @eolivelli @dlg99 Would you mind to take a look?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@bookkeeper.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org