You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@kvrocks.apache.org by "git-hulk (via GitHub)" <gi...@apache.org> on 2023/08/15 13:09:08 UTC
[GitHub] [kvrocks] git-hulk created a discussion: How to maintain the backward compatibility between different versions
GitHub user git-hulk created a discussion: How to maintain the backward compatibility between different versions
For example, we introduced the STREAM column family in 2.2.0 and this breaks the backward compatibility due to the missing column family in the old version, so it's impossible for users to rollback if there's something wrong with the new version. Another example is the RocksDB upgrade RocksDB from 6.29.5 to 7.x in 2.3.0 which makes kXXH3 checksum the new default value for the block, and this cause 2.3.0 can't rollback to 2.0.x(its RocksDB version is 6.19.3 and kXXH3 was introduced in 6.27.0).
For those two specific cases:
- We can create a patch for 2.1 and 2.0, and ask users to upgrade to the corresponding version before upgrading to the newer version
- Create a patch for 2.0 and upgrade RocksDB to 6.29.5
But for future compatibility, I think we can add more guide documents or test cases.
GitHub link: https://github.com/apache/kvrocks/discussions/1678
----
This is an automatically sent email for issues@kvrocks.apache.org.
To unsubscribe, please send an email to: issues-unsubscribe@kvrocks.apache.org
[GitHub] [kvrocks] git-hulk edited a comment on the discussion: How to maintain the forward/backward compatibility between different versions
Posted by "git-hulk (via GitHub)" <gi...@apache.org>.
GitHub user git-hulk edited a comment on the discussion: How to maintain the forward/backward compatibility between different versions
Yes, I think it's easier for testing backward compatibility and we can do it first.
GitHub link: https://github.com/apache/kvrocks/discussions/1678#discussioncomment-6730844
----
This is an automatically sent email for issues@kvrocks.apache.org.
To unsubscribe, please send an email to: issues-unsubscribe@kvrocks.apache.org
[GitHub] [kvrocks] git-hulk added a comment to the discussion: How to maintain the forward/backward compatibility between different versions
Posted by "git-hulk (via GitHub)" <gi...@apache.org>.
GitHub user git-hulk added a comment to the discussion: How to maintain the forward/backward compatibility between different versions
> Newer datatype doesn't matter. When user uses newer datatype, he/she should ensure the responsbility of cannot downgrade.
Yes, that right
> Bugfix and RocksDB upgrade might need to be backport?
Maybe we should backport if we know the newer RocksDB has the compatible issue.
GitHub link: https://github.com/apache/kvrocks/discussions/1678#discussioncomment-6730964
----
This is an automatically sent email for issues@kvrocks.apache.org.
To unsubscribe, please send an email to: issues-unsubscribe@kvrocks.apache.org
[GitHub] [kvrocks] git-hulk edited a discussion: How to maintain the forward/backward compatibility between different versions
Posted by "git-hulk (via GitHub)" <gi...@apache.org>.
GitHub user git-hulk edited a discussion: How to maintain the forward/backward compatibility between different versions
For example, we introduced the STREAM column family in 2.2.0 and this breaks the forward compatibility due to the missing column family in the old version, so it's impossible for users to rollback if there's something wrong with the new version. Another example is the RocksDB upgrade RocksDB from 6.29.5 to 7.x in 2.3.0 which makes kXXH3 checksum the new default value for the block, and this cause 2.3.0 can't rollback to 2.0.x(its RocksDB version is 6.19.3 and kXXH3 was introduced in 6.27.0).
For those two specific cases:
- We can create a patch for 2.1 and 2.0, and ask users to upgrade to the corresponding version before upgrading to the newer version
- Create a patch for 2.0 and upgrade RocksDB to 6.29.5
But for future compatibility, I think we can add more documents to guide developers to keep the compatibility and test cases to ensure that.
GitHub link: https://github.com/apache/kvrocks/discussions/1678
----
This is an automatically sent email for issues@kvrocks.apache.org.
To unsubscribe, please send an email to: issues-unsubscribe@kvrocks.apache.org
[GitHub] [kvrocks] PragmaTwice added a comment to the discussion: How to maintain the backward compatibility between different versions
Posted by "PragmaTwice (via GitHub)" <gi...@apache.org>.
GitHub user PragmaTwice added a comment to the discussion: How to maintain the backward compatibility between different versions
Firstly, I think the thing you mentioned is not backward compatibility, but forward compatibility, i.e. new data can still be read by old binaries.
I think the most important thing is backward compatibility, i.e. old data can still be read by new binaries. And then we can care about forward compatibility, which seems less important to me than code maintainability and feature implementation.
To me, if it does not hurt code maintainability, we can keep forward compatibility, and if we cannot, we can at least alleviate it by highlighting in release note and releasing patch versions for the old version.
BTW, I think it is rare to rollback from 2.3.0 to 2.0.6. Users can firstly try to rollback to more close versions like 2.2.0 or 2.1.0.
GitHub link: https://github.com/apache/kvrocks/discussions/1678#discussioncomment-6730660
----
This is an automatically sent email for issues@kvrocks.apache.org.
To unsubscribe, please send an email to: issues-unsubscribe@kvrocks.apache.org
[GitHub] [kvrocks] git-hulk added a comment to the discussion: How to maintain the backward compatibility between different versions
Posted by "git-hulk (via GitHub)" <gi...@apache.org>.
GitHub user git-hulk added a comment to the discussion: How to maintain the backward compatibility between different versions
An intuitive way to test the compatibility is by testing if the new dataset can be loaded and read by previous binaries. But I don't know how to do that yet. To see if you guys have any thoughts on this topic. @apache/kvrocks-committers
GitHub link: https://github.com/apache/kvrocks/discussions/1678#discussioncomment-6730548
----
This is an automatically sent email for issues@kvrocks.apache.org.
To unsubscribe, please send an email to: issues-unsubscribe@kvrocks.apache.org
[GitHub] [kvrocks] git-hulk added a comment to the discussion: How to maintain the forward/backward compatibility between different versions
Posted by "git-hulk (via GitHub)" <gi...@apache.org>.
GitHub user git-hulk added a comment to the discussion: How to maintain the forward/backward compatibility between different versions
> Firstly, I think the thing you mentioned is not backward compatibility, but forward compatibility, i.e. new data can still be read by old binaries.
Yes, I'm wrongly using the forward and backward. What I want to discuss is forward compatibility indeed. I have fixed the title.
> BTW, I think it is rare to rollback from 2.3.0 to 2.0.6. Users can firstly try to rollback to more close versions like 2.2.0 or 2.1.0.
Yes, but upgrade 2.0.6 to 2.1.0 still have the forward compatibility issue due to the missing column family.
GitHub link: https://github.com/apache/kvrocks/discussions/1678#discussioncomment-6730828
----
This is an automatically sent email for issues@kvrocks.apache.org.
To unsubscribe, please send an email to: issues-unsubscribe@kvrocks.apache.org
[GitHub] [kvrocks] git-hulk added a comment to the discussion: How to maintain the forward/backward compatibility between different versions
Posted by "git-hulk (via GitHub)" <gi...@apache.org>.
GitHub user git-hulk added a comment to the discussion: How to maintain the forward/backward compatibility between different versions
The previously mentioned compatibility cases weren't aware before users report them, so it will be better to let test cases find that.
GitHub link: https://github.com/apache/kvrocks/discussions/1678#discussioncomment-6735877
----
This is an automatically sent email for issues@kvrocks.apache.org.
To unsubscribe, please send an email to: issues-unsubscribe@kvrocks.apache.org
[GitHub] [kvrocks] enjoy-binbin added a comment to the discussion: How to maintain the forward/backward compatibility between different versions
Posted by "enjoy-binbin (via GitHub)" <gi...@apache.org>.
GitHub user enjoy-binbin added a comment to the discussion: How to maintain the forward/backward compatibility between different versions
i suppose it is this one #1203? (bump rocksdb to 7.8.3)
We now seem to be a bit casual about the upgrade of libs.
Usually, the libs can be updated less frequently (of course, there seems to be nothing wrong with frequent updates).
Frequent updates may introduce changes we are not sure about since we are not so family with those libs.
Libs with minor version can be updated, but we need to be cared with the major versions.
Libs with major version can be released together with our major version, but we need to be aware of the changes, otherwise we don't upgrade.
GitHub link: https://github.com/apache/kvrocks/discussions/1678#discussioncomment-6736133
----
This is an automatically sent email for issues@kvrocks.apache.org.
To unsubscribe, please send an email to: issues-unsubscribe@kvrocks.apache.org
[GitHub] [kvrocks] git-hulk edited a comment on the discussion: How to maintain the forward/backward compatibility between different versions
Posted by "git-hulk (via GitHub)" <gi...@apache.org>.
GitHub user git-hulk edited a comment on the discussion: How to maintain the forward/backward compatibility between different versions
For #1203, it's a minor version update(7.7.3->7.8.3) and it makes sense for RocksDB to do this change since they don't promise to be forward-compatible with 6.x versions. I think the key problem is we don't have enough test cases for confirming the compatibility.
GitHub link: https://github.com/apache/kvrocks/discussions/1678#discussioncomment-6736195
----
This is an automatically sent email for issues@kvrocks.apache.org.
To unsubscribe, please send an email to: issues-unsubscribe@kvrocks.apache.org
[GitHub] [kvrocks] xiaobiaozhao added a comment to the discussion: How to maintain the forward/backward compatibility between different versions
Posted by "xiaobiaozhao (via GitHub)" <gi...@apache.org>.
GitHub user xiaobiaozhao added a comment to the discussion: How to maintain the forward/backward compatibility between different versions
I agree with this suggestion
GitHub link: https://github.com/apache/kvrocks/discussions/1678#discussioncomment-6736366
----
This is an automatically sent email for issues@kvrocks.apache.org.
To unsubscribe, please send an email to: issues-unsubscribe@kvrocks.apache.org
[GitHub] [kvrocks] mapleFU added a comment to the discussion: How to maintain the forward/backward compatibility between different versions
Posted by "mapleFU (via GitHub)" <gi...@apache.org>.
GitHub user mapleFU added a comment to the discussion: How to maintain the forward/backward compatibility between different versions
I guess:
1. Newer datatype doesn't matter. When user uses newer datatype, he/she should ensure the responsbility of cannot downgrade.
2. Bugfix and RocksDB upgrade might need to be backport?
3. Change/Evolution on existing data type should has some control flags
GitHub link: https://github.com/apache/kvrocks/discussions/1678#discussioncomment-6730879
----
This is an automatically sent email for issues@kvrocks.apache.org.
To unsubscribe, please send an email to: issues-unsubscribe@kvrocks.apache.org
[GitHub] [kvrocks] git-hulk edited a comment on the discussion: How to maintain the forward/backward compatibility between different versions
Posted by "git-hulk (via GitHub)" <gi...@apache.org>.
GitHub user git-hulk edited a comment on the discussion: How to maintain the forward/backward compatibility between different versions
> Cannot read streams data doesn't hurt, because old versions don't know those data and the server API doesn't answer on those read/write calls. As long as the server can gracefully ignore unknown data (instead of corrupted; that is, we can check the integrity, but with a version or type metadata indicates the server doesn't support), it's not a big issue.
For adding the new column family, we can backport the column family to the previous versions and it won't cause any problem since users have no way to read the data.
> Change the default value of kXXH3 checksum can be a serious compatibility issues as data "corrupted" for old versions. But how can we merge that PR (can you share the link?)? I suggest it should be always backed by an option to choose the old behavior to support in-place upgrading.
It's NOT changed by Kvrocks, rocksdb did that in [v7.8.3](https://github.com/facebook/rocksdb/releases/tag/v7.8.3)
GitHub link: https://github.com/apache/kvrocks/discussions/1678#discussioncomment-6730933
----
This is an automatically sent email for issues@kvrocks.apache.org.
To unsubscribe, please send an email to: issues-unsubscribe@kvrocks.apache.org
[GitHub] [kvrocks] git-hulk added a comment to the discussion: How to maintain the forward/backward compatibility between different versions
Posted by "git-hulk (via GitHub)" <gi...@apache.org>.
GitHub user git-hulk added a comment to the discussion: How to maintain the forward/backward compatibility between different versions
For #1203, it's a minor version update(7.7.3->7.8.3).
GitHub link: https://github.com/apache/kvrocks/discussions/1678#discussioncomment-6736195
----
This is an automatically sent email for issues@kvrocks.apache.org.
To unsubscribe, please send an email to: issues-unsubscribe@kvrocks.apache.org
[GitHub] [kvrocks] git-hulk added a comment to the discussion: How to maintain the forward/backward compatibility between different versions
Posted by "git-hulk (via GitHub)" <gi...@apache.org>.
GitHub user git-hulk added a comment to the discussion: How to maintain the forward/backward compatibility between different versions
> Cannot read streams data doesn't hurt, because old versions don't know those data and the server API doesn't answer on those read/write calls. As long as the server can gracefully ignore unknown data (instead of corrupted; that is, we can check the integrity, but with a version or type metadata indicates the server doesn't support), it's not a big issue.
For adding the new column family, we can backport the column family to the previous versions and it won't cause any problem since users have no way to read the data.
> Change the default value of kXXH3 checksum can be a serious compatibility issues as data "corrupted" for old versions. But how can we merge that PR (can you share the link?)? I suggest it should be always backed by an option to choose the old behavior to support in-place upgrading.
It's changed by Kvrocks, rocksdb did that in [v7.8.3](https://github.com/facebook/rocksdb/releases/tag/v7.8.3)
GitHub link: https://github.com/apache/kvrocks/discussions/1678#discussioncomment-6730933
----
This is an automatically sent email for issues@kvrocks.apache.org.
To unsubscribe, please send an email to: issues-unsubscribe@kvrocks.apache.org
[GitHub] [kvrocks] git-hulk added a comment to the discussion: How to maintain the forward/backward compatibility between different versions
Posted by "git-hulk (via GitHub)" <gi...@apache.org>.
GitHub user git-hulk added a comment to the discussion: How to maintain the forward/backward compatibility between different versions
Yes, should be forward NOT backward.
GitHub link: https://github.com/apache/kvrocks/discussions/1678#discussioncomment-6730854
----
This is an automatically sent email for issues@kvrocks.apache.org.
To unsubscribe, please send an email to: issues-unsubscribe@kvrocks.apache.org
[GitHub] [kvrocks] tisonkun added a comment to the discussion: How to maintain the forward/backward compatibility between different versions
Posted by "tisonkun (via GitHub)" <gi...@apache.org>.
GitHub user tisonkun added a comment to the discussion: How to maintain the forward/backward compatibility between different versions
> It's changed by Kvrocks, rocksdb did that in [v7.8.3](https://github.com/facebook/rocksdb/releases/tag/v7.8.3)
Can you refer to the PR in Kvrocks?
GitHub link: https://github.com/apache/kvrocks/discussions/1678#discussioncomment-6735635
----
This is an automatically sent email for issues@kvrocks.apache.org.
To unsubscribe, please send an email to: issues-unsubscribe@kvrocks.apache.org
[GitHub] [kvrocks] git-hulk edited a comment on the discussion: How to maintain the forward/backward compatibility between different versions
Posted by "git-hulk (via GitHub)" <gi...@apache.org>.
GitHub user git-hulk edited a comment on the discussion: How to maintain the forward/backward compatibility between different versions
Should be this PR: https://github.com/facebook/rocksdb/pull/10778
GitHub link: https://github.com/apache/kvrocks/discussions/1678#discussioncomment-6735847
----
This is an automatically sent email for issues@kvrocks.apache.org.
To unsubscribe, please send an email to: issues-unsubscribe@kvrocks.apache.org
[GitHub] [kvrocks] tisonkun added a comment to the discussion: How to maintain the backward compatibility between different versions
Posted by "tisonkun (via GitHub)" <gi...@apache.org>.
GitHub user tisonkun added a comment to the discussion: How to maintain the backward compatibility between different versions
Concretely,
* Cannot read streams data doesn't hurt, because old versions don't know those data and the server API doesn't answer on read/write calls. As long as the server can gracefully ignore unknown data (instead of corrupted; that is, we can check the integrity, but with a version or type metadata indicates the server doesn't support), it's not a big issue.
* Change the default value of kXXH3 checksum can be a serious compatibility issues as data "corrupted" for old versions. But how can we merge that PR (can you share the link?)? I suggest it should be always backed by an option to choose the old behavior to support in-place upgrading.
GitHub link: https://github.com/apache/kvrocks/discussions/1678#discussioncomment-6730747
----
This is an automatically sent email for issues@kvrocks.apache.org.
To unsubscribe, please send an email to: issues-unsubscribe@kvrocks.apache.org
[GitHub] [kvrocks] tisonkun added a comment to the discussion: How to maintain the backward compatibility between different versions
Posted by "tisonkun (via GitHub)" <gi...@apache.org>.
GitHub user tisonkun added a comment to the discussion: How to maintain the backward compatibility between different versions
> An intuitive way to test the compatibility is by testing if the new dataset can be loaded and read by previous binaries. But I don't know how to do that yet.
Roughtly if we do not delete the data (folder), we can start a server in new version, shutting down, and start a server in old version with the same data dir and perform basic operations. I'm unsure if it's viable according to our bootstrap and teardown logics.
GitHub link: https://github.com/apache/kvrocks/discussions/1678#discussioncomment-6730654
----
This is an automatically sent email for issues@kvrocks.apache.org.
To unsubscribe, please send an email to: issues-unsubscribe@kvrocks.apache.org
[GitHub] [kvrocks] mapleFU added a comment to the discussion: How to maintain the backward compatibility between different versions
Posted by "mapleFU (via GitHub)" <gi...@apache.org>.
GitHub user mapleFU added a comment to the discussion: How to maintain the backward compatibility between different versions
I guess the tile is "forward-compatible"? Since it means a old-version kvrocks read latest data.
RocksDB has some flags about compatible: https://github.com/facebook/rocksdb/wiki/RocksDB-Compatibility-Between-Different-Releases#format-compatibility . It gurantee the forward compatible on a major version.
As for minor version, if schema change happens, we'd better add them in document, and maybe provide some flags to control them?
GitHub link: https://github.com/apache/kvrocks/discussions/1678#discussioncomment-6730665
----
This is an automatically sent email for issues@kvrocks.apache.org.
To unsubscribe, please send an email to: issues-unsubscribe@kvrocks.apache.org
[GitHub] [kvrocks] tisonkun added a comment to the discussion: How to maintain the backward compatibility between different versions
Posted by "tisonkun (via GitHub)" <gi...@apache.org>.
GitHub user tisonkun added a comment to the discussion: How to maintain the backward compatibility between different versions
As we are trying to follow the semver flavor, when we make breaking changes, it's necessary to bump major versions.
This should happen during the review phase and we can pay extra attention to known risks as listed above from now on.
GitHub link: https://github.com/apache/kvrocks/discussions/1678#discussioncomment-6730642
----
This is an automatically sent email for issues@kvrocks.apache.org.
To unsubscribe, please send an email to: issues-unsubscribe@kvrocks.apache.org
[GitHub] [kvrocks] git-hulk edited a discussion: How to maintain the forward/backward compatibility between different versions
Posted by "git-hulk (via GitHub)" <gi...@apache.org>.
GitHub user git-hulk edited a discussion: How to maintain the forward/backward compatibility between different versions
For example, we introduced the STREAM column family in 2.2.0 and this breaks the backward compatibility due to the missing column family in the old version, so it's impossible for users to rollback if there's something wrong with the new version. Another example is the RocksDB upgrade RocksDB from 6.29.5 to 7.x in 2.3.0 which makes kXXH3 checksum the new default value for the block, and this cause 2.3.0 can't rollback to 2.0.x(its RocksDB version is 6.19.3 and kXXH3 was introduced in 6.27.0).
For those two specific cases:
- We can create a patch for 2.1 and 2.0, and ask users to upgrade to the corresponding version before upgrading to the newer version
- Create a patch for 2.0 and upgrade RocksDB to 6.29.5
But for future compatibility, I think we can add more documents to guide developers to keep the compatibility and test cases to ensure that.
GitHub link: https://github.com/apache/kvrocks/discussions/1678
----
This is an automatically sent email for issues@kvrocks.apache.org.
To unsubscribe, please send an email to: issues-unsubscribe@kvrocks.apache.org
[GitHub] [kvrocks] caipengbo added a comment to the discussion: How to maintain the forward/backward compatibility between different versions
Posted by "caipengbo (via GitHub)" <gi...@apache.org>.
GitHub user caipengbo added a comment to the discussion: How to maintain the forward/backward compatibility between different versions
I also have a small suggestion, I think we can add a tag mark on CR, when we CR, we take the compatibility issue as a point of consideration.
GitHub link: https://github.com/apache/kvrocks/discussions/1678#discussioncomment-6735682
----
This is an automatically sent email for issues@kvrocks.apache.org.
To unsubscribe, please send an email to: issues-unsubscribe@kvrocks.apache.org
[GitHub] [kvrocks] git-hulk edited a comment on the discussion: How to maintain the forward/backward compatibility between different versions
Posted by "git-hulk (via GitHub)" <gi...@apache.org>.
GitHub user git-hulk edited a comment on the discussion: How to maintain the forward/backward compatibility between different versions
Yes, should be forward NOT backward. My intention is to discuss how to find or detect the compatible issue, for the previous two cases, we didn't notice that before users complains.
GitHub link: https://github.com/apache/kvrocks/discussions/1678#discussioncomment-6730854
----
This is an automatically sent email for issues@kvrocks.apache.org.
To unsubscribe, please send an email to: issues-unsubscribe@kvrocks.apache.org
[GitHub] [kvrocks] tisonkun added a comment to the discussion: How to maintain the backward compatibility between different versions
Posted by "tisonkun (via GitHub)" <gi...@apache.org>.
GitHub user tisonkun added a comment to the discussion: How to maintain the backward compatibility between different versions
BTW, we can prepare a breaking changes page like what I did for Curator - https://curator.apache.org/docs/breaking-changes
GitHub link: https://github.com/apache/kvrocks/discussions/1678#discussioncomment-6730664
----
This is an automatically sent email for issues@kvrocks.apache.org.
To unsubscribe, please send an email to: issues-unsubscribe@kvrocks.apache.org
[GitHub] [kvrocks] enjoy-binbin added a comment to the discussion: How to maintain the forward/backward compatibility between different versions
Posted by "enjoy-binbin (via GitHub)" <gi...@apache.org>.
GitHub user enjoy-binbin added a comment to the discussion: How to maintain the forward/backward compatibility between different versions
The proposer of the PR (someone who is familiar with it, not the normal user) or the maintainer needs to be able to review breaking changes.
For breaking changes, we can label them and call people to review and reach consensus (but do not merge, breaking change is a major decision, ideally with PMC approval), delayed merging until a major version update (or merge into the unstable branch but do not release).
Unless it is a breaking change caused by a very serious bug, it can be merged into a minor version.
And then when releasing, release notes need to be specially added a breaking changes section.
My saying are a bit off topic, but we also need to pay attention to normal breaking changes
GitHub link: https://github.com/apache/kvrocks/discussions/1678#discussioncomment-6736135
----
This is an automatically sent email for issues@kvrocks.apache.org.
To unsubscribe, please send an email to: issues-unsubscribe@kvrocks.apache.org
[GitHub] [kvrocks] git-hulk added a comment to the discussion: How to maintain the forward/backward compatibility between different versions
Posted by "git-hulk (via GitHub)" <gi...@apache.org>.
GitHub user git-hulk added a comment to the discussion: How to maintain the forward/backward compatibility between different versions
Should be this PR: https://github.com/facebook/rocksdb/pull/10778/files
GitHub link: https://github.com/apache/kvrocks/discussions/1678#discussioncomment-6735847
----
This is an automatically sent email for issues@kvrocks.apache.org.
To unsubscribe, please send an email to: issues-unsubscribe@kvrocks.apache.org
[GitHub] [kvrocks] tisonkun added a comment to the discussion: How to maintain the forward/backward compatibility between different versions
Posted by "tisonkun (via GitHub)" <gi...@apache.org>.
GitHub user tisonkun added a comment to the discussion: How to maintain the forward/backward compatibility between different versions
I think here is where we can introduce a PR template to materialize some check items the community agrees on.
I'll try to prepare a patch for review.
GitHub link: https://github.com/apache/kvrocks/discussions/1678#discussioncomment-6738856
----
This is an automatically sent email for issues@kvrocks.apache.org.
To unsubscribe, please send an email to: issues-unsubscribe@kvrocks.apache.org
[GitHub] [kvrocks] tisonkun edited a comment on the discussion: How to maintain the backward compatibility between different versions
Posted by "tisonkun (via GitHub)" <gi...@apache.org>.
GitHub user tisonkun edited a comment on the discussion: How to maintain the backward compatibility between different versions
Concretely,
* Cannot read streams data doesn't hurt, because old versions don't know those data and the server API doesn't answer on those read/write calls. As long as the server can gracefully ignore unknown data (instead of corrupted; that is, we can check the integrity, but with a version or type metadata indicates the server doesn't support), it's not a big issue.
* Change the default value of kXXH3 checksum can be a serious compatibility issues as data "corrupted" for old versions. But how can we merge that PR (can you share the link?)? I suggest it should be always backed by an option to choose the old behavior to support in-place upgrading.
GitHub link: https://github.com/apache/kvrocks/discussions/1678#discussioncomment-6730747
----
This is an automatically sent email for issues@kvrocks.apache.org.
To unsubscribe, please send an email to: issues-unsubscribe@kvrocks.apache.org
[GitHub] [kvrocks] tisonkun added a comment to the discussion: How to maintain the backward compatibility between different versions
Posted by "tisonkun (via GitHub)" <gi...@apache.org>.
GitHub user tisonkun added a comment to the discussion: How to maintain the backward compatibility between different versions
Yeah. Supporting arbitrary backward and forward rolling update is challenging even in the business world. Basically, operators should roll up between the adjacent versions.
However, it's a major concern to users that: After I upgrade to new versions and encounter bugs, I cannot roll back from where I come from. This requires, at least, an "IMPORTANT NOTICE" as we should also seldom change the on-disk data format.
GitHub link: https://github.com/apache/kvrocks/discussions/1678#discussioncomment-6730712
----
This is an automatically sent email for issues@kvrocks.apache.org.
To unsubscribe, please send an email to: issues-unsubscribe@kvrocks.apache.org
[GitHub] [kvrocks] git-hulk edited a discussion: How to maintain the backward compatibility between different versions
Posted by "git-hulk (via GitHub)" <gi...@apache.org>.
GitHub user git-hulk edited a discussion: How to maintain the backward compatibility between different versions
For example, we introduced the STREAM column family in 2.2.0 and this breaks the backward compatibility due to the missing column family in the old version, so it's impossible for users to rollback if there's something wrong with the new version. Another example is the RocksDB upgrade RocksDB from 6.29.5 to 7.x in 2.3.0 which makes kXXH3 checksum the new default value for the block, and this cause 2.3.0 can't rollback to 2.0.x(its RocksDB version is 6.19.3 and kXXH3 was introduced in 6.27.0).
For those two specific cases:
- We can create a patch for 2.1 and 2.0, and ask users to upgrade to the corresponding version before upgrading to the newer version
- Create a patch for 2.0 and upgrade RocksDB to 6.29.5
But for future compatibility, I think we can add more documents to guide developers to keep the compatibility and test cases to ensure that.
GitHub link: https://github.com/apache/kvrocks/discussions/1678
----
This is an automatically sent email for issues@kvrocks.apache.org.
To unsubscribe, please send an email to: issues-unsubscribe@kvrocks.apache.org
[GitHub] [kvrocks] git-hulk added a comment to the discussion: How to maintain the forward/backward compatibility between different versions
Posted by "git-hulk (via GitHub)" <gi...@apache.org>.
GitHub user git-hulk added a comment to the discussion: How to maintain the forward/backward compatibility between different versions
Yes, I think it's easier for testing forward compatibility and we can do it first.
GitHub link: https://github.com/apache/kvrocks/discussions/1678#discussioncomment-6730844
----
This is an automatically sent email for issues@kvrocks.apache.org.
To unsubscribe, please send an email to: issues-unsubscribe@kvrocks.apache.org
[GitHub] [kvrocks] jihuayu added a comment to the discussion: How to maintain the forward/backward compatibility between different versions
Posted by "jihuayu (via GitHub)" <gi...@apache.org>.
GitHub user jihuayu added a comment to the discussion: How to maintain the forward/backward compatibility between different versions
I have an idea for handling modifications across multiple versions. We can provide an ["Upgrade paths" similar to GitLab](https://docs.gitlab.com/ee/update/#upgrade-paths), where we only need to ensure compatibility between 2 nearly key versions, rather than between any two versions.
GitHub link: https://github.com/apache/kvrocks/discussions/1678#discussioncomment-6731034
----
This is an automatically sent email for issues@kvrocks.apache.org.
To unsubscribe, please send an email to: issues-unsubscribe@kvrocks.apache.org