You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kvrocks.apache.org by GitBox <gi...@apache.org> on 2022/10/19 15:10:10 UTC
[GitHub] [incubator-kvrocks] aleksraiden created a discussion: Change RocksDB to 7.х?
GitHub user aleksraiden created a discussion: Change RocksDB to 7.х?
So, after we are switched to C++17, we are ready to use an 7.х family of RocksDB. Maybe we can try to use 7.0.4 - as a latest version without any changes in public API sections, and then step-by-step migrate to newest versions?
I will submit PR, of course
GitHub link: https://github.com/apache/incubator-kvrocks/discussions/1013
----
This is an automatically sent email for dev@kvrocks.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@kvrocks.apache.org
[GitHub] [incubator-kvrocks] git-hulk added a comment to the discussion: Change RocksDB to 7.х?
Posted by GitBox <gi...@apache.org>.
GitHub user git-hulk added a comment to the discussion: Change RocksDB to 7.х?
Thanks for @mapleFU input.
GitHub link: https://github.com/apache/incubator-kvrocks/discussions/1013#discussioncomment-4024516
----
This is an automatically sent email for issues@kvrocks.apache.org.
To unsubscribe, please send an email to: issues-unsubscribe@kvrocks.apache.org
[GitHub] [incubator-kvrocks] xiaobiaozhao added a comment to the discussion: Change RocksDB to 7.х?
Posted by GitBox <gi...@apache.org>.
GitHub user xiaobiaozhao added a comment to the discussion: Change RocksDB to 7.х?
## kvrocks with rocksdb_v6 vs rocksdv_v7 report
### Build
build current kvrocks and use rocksdb-v7 kvrocks pr is https://github.com/xiaobiaozhao/incubator-kvrocks/pull/4
### Test steps
Test deploy
stress script
```
docker run -d --rm redislabs/memtier_benchmark:latest -t 2 -c 20 -s 172.27.255.230 -p 6666 --distinct-client-seed --key-minimum=100000000 --key-maximum=10000000000 --random-data --data-size=10000 -n 10000000000 --command "set __key__ __data__"
```
kvrocks hardware settings
```
Cores:4C
RAM:32GB
Disk: nvme SSD 890GB
```
### monitor
> 172.27.255.230 -> kvrocks-v7
>
> 172.27.255.229 -> kvrocks-v6
kvrocks-v7 vs kvrocks-v6 full compact monitor
```
127.0.0.1:6666> COMPACT
OK
```
172.27.255.229 disk monitor
172.27.255.230 disk monitor
Other monitoring measures were consistent
### When kvrocks compacting
| | kvrocks(v6) | kvrocks(v7) |
| ------- | ----------------------------------------------- | ----------------- |
| cpu | to lower | normal |
| mem | to lower then to higher | normal |
| disk | reads doubled and limited 500MB（kvrocks.conf） | a little big high |
| ops | to almost zero | normal |
| latency | to higher | normal |
### Conclusion
1. I think the v7 version has more consistent performance than v6 in compact
2. I think kvrocks worth upgrading to rocksdb v7
GitHub link: https://github.com/apache/incubator-kvrocks/discussions/1013#discussioncomment-4005680
----
This is an automatically sent email for issues@kvrocks.apache.org.
To unsubscribe, please send an email to: issues-unsubscribe@kvrocks.apache.org
[GitHub] [incubator-kvrocks] git-hulk edited a comment on the discussion: Change RocksDB to 7.х?
Posted by GitBox <gi...@apache.org>.
GitHub user git-hulk edited a comment on the discussion: Change RocksDB to 7.х?
Thanks for @xiaobiaozhao great benchmark report, it's truly worth upgrading in bulk writing scenarios.
It looks can solve your issues. cc @juancrescente
GitHub link: https://github.com/apache/incubator-kvrocks/discussions/1013#discussioncomment-4006072
----
This is an automatically sent email for issues@kvrocks.apache.org.
To unsubscribe, please send an email to: issues-unsubscribe@kvrocks.apache.org
Re: [D] Change RocksDB to 7.х? [kvrocks]
Posted by "xiaobiaozhao (via GitHub)" <gi...@apache.org>.
GitHub user xiaobiaozhao added a comment to the discussion: Change RocksDB to 7.х?
which pic?
GitHub link: https://github.com/apache/kvrocks/discussions/1013#discussioncomment-9058069
----
This is an automatically sent email for issues@kvrocks.apache.org.
To unsubscribe, please send an email to: issues-unsubscribe@kvrocks.apache.org
[GitHub] [incubator-kvrocks] git-hulk added a comment to the discussion: Change RocksDB to 7.х?
Posted by GitBox <gi...@apache.org>.
GitHub user git-hulk added a comment to the discussion: Change RocksDB to 7.х?
https://groups.google.com/g/rocksdb/c/umRPboha0as/m/5VP6zvwiCgAJ?utm_medium=email&utm_source=footer
RocksDB community mentioned that 7.5.3 could significantly improve compaction time, so it may become one of the reasons to upgrade to 7.x, record it here first.
GitHub link: https://github.com/apache/incubator-kvrocks/discussions/1013#discussioncomment-3982517
----
This is an automatically sent email for issues@kvrocks.apache.org.
To unsubscribe, please send an email to: issues-unsubscribe@kvrocks.apache.org
[GitHub] [incubator-kvrocks] git-hulk added a comment to the discussion: Change RocksDB to 7.х?
Posted by GitBox <gi...@apache.org>.
GitHub user git-hulk added a comment to the discussion: Change RocksDB to 7.х?
Thanks for @xiaobiaozhao benchmark, it looks can solve your issues. cc @juancrescente
GitHub link: https://github.com/apache/incubator-kvrocks/discussions/1013#discussioncomment-4006072
----
This is an automatically sent email for issues@kvrocks.apache.org.
To unsubscribe, please send an email to: issues-unsubscribe@kvrocks.apache.org
[GitHub] [incubator-kvrocks] git-hulk added a comment to the discussion: Change RocksDB to 7.х?
Posted by GitBox <gi...@apache.org>.
GitHub user git-hulk added a comment to the discussion: Change RocksDB to 7.х?
Thanks for @aleksraiden proposal, I think it'd be better to wait for more services using 7.x in production, or it also makes sense if we need requests which appears only in 7.x.
GitHub link: https://github.com/apache/incubator-kvrocks/discussions/1013#discussioncomment-3916976
----
This is an automatically sent email for dev@kvrocks.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@kvrocks.apache.org
[GitHub] [incubator-kvrocks] xiaobiaozhao added a comment to the discussion: Change RocksDB to 7.х?
Posted by GitBox <gi...@apache.org>.
GitHub user xiaobiaozhao added a comment to the discussion: Change RocksDB to 7.х?
I think if we plan to upgrade to 7.x, as well just upgrade to the latest version
Here's a demo pr
https://github.com/xiaobiaozhao/incubator-kvrocks/pull/4
I plan to do some test for performance improvements about rocksdb 7 vs rocksdb 6 at the weekend.
GitHub link: https://github.com/apache/incubator-kvrocks/discussions/1013#discussioncomment-3983839
----
This is an automatically sent email for issues@kvrocks.apache.org.
To unsubscribe, please send an email to: issues-unsubscribe@kvrocks.apache.org
[GitHub] [incubator-kvrocks] git-hulk edited a comment on the discussion: Change RocksDB to 7.х?
Posted by GitBox <gi...@apache.org>.
GitHub user git-hulk edited a comment on the discussion: Change RocksDB to 7.х?
Thanks for @xiaobiaozhao great benchmark report, it's truly worth upgrading in bulk writing scenario.
It looks can solve your issues. cc @juancrescente
GitHub link: https://github.com/apache/incubator-kvrocks/discussions/1013#discussioncomment-4006072
----
This is an automatically sent email for issues@kvrocks.apache.org.
To unsubscribe, please send an email to: issues-unsubscribe@kvrocks.apache.org
[GitHub] [incubator-kvrocks] mapleFU added a comment to the discussion: Change RocksDB to 7.х?
Posted by GitBox <gi...@apache.org>.
GitHub user mapleFU added a comment to the discussion: Change RocksDB to 7.х?
Here is another patch in RocksDB 7.6:
https://github.com/facebook/rocksdb/commit/1e9bf25f61fa5b08fe2b3fe5c1fcf33e4c1c515c
Which makes tasks faster
GitHub link: https://github.com/apache/incubator-kvrocks/discussions/1013#discussioncomment-4024449
----
This is an automatically sent email for issues@kvrocks.apache.org.
To unsubscribe, please send an email to: issues-unsubscribe@kvrocks.apache.org
[GitHub] [incubator-kvrocks] git-hulk edited a comment on the discussion: Change RocksDB to 7.х?
Posted by GitBox <gi...@apache.org>.
GitHub user git-hulk edited a comment on the discussion: Change RocksDB to 7.х?
It will help a lot if you can provide more information about the 7.x performance improvement.
GitHub link: https://github.com/apache/incubator-kvrocks/discussions/1013#discussioncomment-3983977
----
This is an automatically sent email for issues@kvrocks.apache.org.
To unsubscribe, please send an email to: issues-unsubscribe@kvrocks.apache.org
[GitHub] [incubator-kvrocks] git-hulk added a comment to the discussion: Change RocksDB to 7.х?
Posted by GitBox <gi...@apache.org>.
GitHub user git-hulk added a comment to the discussion: Change RocksDB to 7.х?
It will help a lot if you can provide more information about the 7.x improvement.
GitHub link: https://github.com/apache/incubator-kvrocks/discussions/1013#discussioncomment-3983977
----
This is an automatically sent email for issues@kvrocks.apache.org.
To unsubscribe, please send an email to: issues-unsubscribe@kvrocks.apache.org
[GitHub] [incubator-kvrocks] git-hulk added a comment to the discussion: Change RocksDB to 7.х?
Posted by GitBox <gi...@apache.org>.
GitHub user git-hulk added a comment to the discussion: Change RocksDB to 7.х?
@aleksraiden Do you test the performance of the latest RocksDB? From RocksDB's release logs and group discussion, they said 7.x will significantly improve bulk writing and compaction.
GitHub link: https://github.com/apache/incubator-kvrocks/discussions/1013#discussioncomment-3987745
----
This is an automatically sent email for issues@kvrocks.apache.org.
To unsubscribe, please send an email to: issues-unsubscribe@kvrocks.apache.org
[GitHub] [incubator-kvrocks] PragmaTwice edited a comment on the discussion: Change RocksDB to 7.х?
Posted by GitBox <gi...@apache.org>.
GitHub user PragmaTwice edited a comment on the discussion: Change RocksDB to 7.х?
Hi @aleksraiden ,
I notice there are lots of public API and behavior changes in rocksdb 7 (https://github.com/facebook/rocksdb/blob/main/HISTORY.md), will these affect kvrocks?
I am also curious about the performance of kvrocks between rocksdb 6 and 7, are there some benchmark about it?
For update to rocksdb 7 (if we decide to do that), I prefer update to the latest version directly.
GitHub link: https://github.com/apache/incubator-kvrocks/discussions/1013#discussioncomment-3917328
----
This is an automatically sent email for dev@kvrocks.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@kvrocks.apache.org
[GitHub] [incubator-kvrocks] PragmaTwice added a comment to the discussion: Change RocksDB to 7.х?
Posted by GitBox <gi...@apache.org>.
GitHub user PragmaTwice added a comment to the discussion: Change RocksDB to 7.х?
I notice there are lots of public API and behavior changes in rocksdb 7 (https://github.com/facebook/rocksdb/blob/main/HISTORY.md), will these affect kvrocks?
I am also curious about the performance of kvrocks between rocksdb 6 and 7, are there some benchmark about it?
For update to rocksdb 7 (if we decide to do that), I prefer update to the latest version directly.
GitHub link: https://github.com/apache/incubator-kvrocks/discussions/1013#discussioncomment-3917328
----
This is an automatically sent email for dev@kvrocks.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@kvrocks.apache.org
Re: [D] Change RocksDB to 7.х? [kvrocks]
Posted by "bushijesus (via GitHub)" <gi...@apache.org>.
GitHub user bushijesus added a comment to the discussion: Change RocksDB to 7.х?
How to generate the picture like this、please tell me，thanks
GitHub link: https://github.com/apache/kvrocks/discussions/1013#discussioncomment-8936343
----
This is an automatically sent email for issues@kvrocks.apache.org.
To unsubscribe, please send an email to: issues-unsubscribe@kvrocks.apache.org