You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@kvrocks.apache.org by GitBox <gi...@apache.org> on 2022/10/16 12:43:04 UTC
[GitHub] [incubator-kvrocks] ColinChamber opened a new issue, #1001: Make kvrocks dump rdb file which redis can load.
ColinChamber opened a new issue, #1001:
URL: https://github.com/apache/incubator-kvrocks/issues/1001
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/incubator-kvrocks/issues) and found no similar issues.
### Motivation
Making Kvrocks write RDB is more straightforward than making it parse RDB.
Because Redis is backward compatible, which can load the original RDB file.
We can easily utilize the primitive RDB object types, which follow the simple
encoding pattern: <type> <len> <key> <len> <value>
or <type> <len> <key> <size> <data> (<data> consists of <len><key(value)>)
RDB_TYPE_STRING, RDB_TYPE_LIST, RDB_TYPE_SET, RDB_TYPE_ZSET, RDB_TYPE_HASH,
these five types follow this pattern. We ignore the RDB_TYPE_HASH_ZIPMAP,
RDB_TYPE_LIST_ZIPLIST, RDB_TYPE_SET_INTSET, and other complicated types
only implement the five simple type.
There is a mapping relationship between Redis object type and RDB object type.
But you don't have to worry about Redis using ineffective object types after loading RDB.
Redis will convert it to an efficient type automatically.
E.g
kvrocks> hmset foo foo1 bar1 foo2 bar2
kvrocks> save (use RDB_TYPE_HASH rather than RDB_TYPE_HASH_ZIPLIST)
-------- Redis loads the RDB that kvrocks dumped --------
redis> OBJECT ENCODING foo
"ziplist"
There are more detailed references to RDB.
https://github.com/sripathikrishnan/redis-rdb-tools/wiki/Redis-RDB-Dump-File-Format
https://github.com/sripathikrishnan/redis-rdb-tools/blob/master/docs/RDB_Version_History.textile
https://rdb.fnordig.de/file_format.html
### Solution
#958
### Are you willing to submit a PR?
- [X] I'm willing to submit a PR!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@kvrocks.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kvrocks] ColinChamber closed issue #1001: Make kvrocks dump rdb file which redis can load.
Posted by GitBox <gi...@apache.org>.
ColinChamber closed issue #1001: Make kvrocks dump rdb file which redis can load.
URL: https://github.com/apache/incubator-kvrocks/issues/1001
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@kvrocks.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kvrocks] git-hulk commented on issue #1001: Make kvrocks dump rdb file which redis can load.
Posted by GitBox <gi...@apache.org>.
git-hulk commented on issue #1001:
URL: https://github.com/apache/incubator-kvrocks/issues/1001#issuecomment-1280686187
cool, thanks for @ColinChamber great work.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@kvrocks.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kvrocks] git-hulk commented on issue #1001: Make kvrocks dump rdb file which redis can load.
Posted by GitBox <gi...@apache.org>.
git-hulk commented on issue #1001:
URL: https://github.com/apache/incubator-kvrocks/issues/1001#issuecomment-1305078511
> Hi @ColinChamber @git-hulk i don't think supporting dumping RDB in Kvrocks is a good idea. Kvrocks builds its data based on SSD, and its data volume usually is huge, so the RDB usually is big, i don't think it is a general requirement of loading so big RDB file into redis. What's more, Kvrocks is Kvrocks rather than a fork of redis, we support redis protocol and some commands, don't need to support everything, so i think it is not suitable to support dumping RDB in Kvrocks core code, and now we already has `kvrocks2redis` which could generate AOF, i think it looks enough, even we want to support dumping rdb, we should implement this feature in `kvrocks2redis`.
Yes, maybe we need a basic guide or consensus to determine what features can be merged or not.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@kvrocks.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kvrocks] ShooterIT commented on issue #1001: Make kvrocks dump rdb file which redis can load.
Posted by GitBox <gi...@apache.org>.
ShooterIT commented on issue #1001:
URL: https://github.com/apache/incubator-kvrocks/issues/1001#issuecomment-1304982649
Hi @ColinChamber @git-hulk i don't think supporting dumping RDB in Kvrocks is a good idea.
Kvrocks builds its data based on SSD, and its data volume usually is huge, so the RDB usually is big, i don't think it is a general requirement of loading so big RDB file into redis. What's more, Kvrocks is Kvrocks rather than a fork of redis, we support redis protocol and some commands, don't need to support everything, so i think it is not suitable to support dumping RDB in Kvrocks core code, and now we already has `kvrocks2redis` which could generate AOF, i think it looks enough, even we want to support dumping rdb, we should implement this feature in `kvrocks2redis`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@kvrocks.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org