[Cassandra Wiki] Update of "CompatibilityGuarantees" by SylvainLebresne

[Apache Wiki <wi...@apache.org>]

Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Cassandra Wiki" for change notification.

The "CompatibilityGuarantees" page has been changed by SylvainLebresne:
https://wiki.apache.org/cassandra/CompatibilityGuarantees

New page:
= Compatibility guarantees =

The following document describe the compatibility guarantees offered during upgrade of Apache Cassandra. When a version is mentioned, this document assumes the “tick-tock” versioning and so in X.Y, X is the major version and Y the minor one.

== General Definition ==

When we say that upgrading from version X to version Y is supported, we always at least mean that there is a path (documented in the NEWS file if any specifics are required) for upgrading all the nodes of a cluster from X to Y in a rolling fashion and so without incurring the unavailability of the database as a whole (that is, without loss of service).

Note however that during major upgrades (3.x to 4.y) ALTER, repair, bootstrap, and decommission might be temporary unavailable until the upgrade complete.  Starting with 4.y, we plan to remove this limitation.

It is also always strongly discouraged to upgrade to any version without testing the upgrade in a staging environment and without having at least some snapshot of the sstables around. This is particularly ill advised for major upgrades.

== Stable vs Experimental ==

Anything is either considered experimental or stable. No guarantee of any sort is provided on something experimental, outside of a gentleman's agreement of not completely changing/removing features in a minor release without serious reasons.

== Minor upgrades ==

Upgrading a node to the minor versions of an equivalent major should be virtually indistinguishable from simply restarting the node (without upgrading it) from a user point of view. This means in particular:
 * No removal/modifications of any configuration option, startup option, exposed metrics or general behavior of the Cassandra process.
 * No removal nor syntactical/semantical change to either CQL, authentication, any existing version of the binary protocol or thrift. 

Those guarantees should be enforced as strongly as possible. In the real world however, despite our efforts to avoid it, unfortunate backward incompatible changes might end up in a release due to:
 * an error: if such change was to pass our vigilance and testing and make it in a release, we'll fix that break as soon as possible (in a “patch” release).
 * in rare occasions, fixing a bug might take the form of a breaking change. In the hopefully very rare case where preserving the bug is considered a lot worst than preserving compatibility, we might do such a change in a minor release.
In both case, we will communicate any such breaking change to the mailing list as soon as it is found.

While no features will be removed in a minor upgrade, some feature could be deprecated in a minor from time to time. See the section on deprecation for more details.

New features may and will be added however, though will be limited to feature releases (even-numbered ones). Those new features may not and should not be used until the full cluster has been upgraded to support them. 

The corollary of this is that, provided you accept to be limited to the features supported by the smaller version in the cluster, clusters with mixed versions _within_ a major are supported.

== Major upgrades ==

Major upgrades are only supported to the very next major version. That is, upgrading from any 3.x release to 4.x will be supported, but upgrading from 2.x to 4.x is not guaranteed. While efforts will be made to allow upgrading from any minor to the next major, there may be restrictions: for instance, upgrading to 4.0 may only be supported from 3.4. If such restrictions exists, it will be clearly documented in the NEWS file of the new major release.

== Deprecation ==

We may deprecate some features/options over time. Typically, this could be because an option doesn’t do anything useful following some internal changes, or it has been superseded by another “better” option. Deprecation means that the use of the feature/option is discouraged and that it is likely to be removed in the next major release. We will remove a deprecated feature in the next major assuming it has been deprecated since at least 6 months. Given the monthly cadence of tick-tock, this means that a feature deprecated in 3.3 will (likely) be removed in 4.0. A feature deprecated in 3.10 however, will only be removed in 5.0 however.

The deprecation of an option will always be indicated in the NEWS file for the release on which it is first deprecated, and a warning will be issued in the log file if a deprecated feature is used.

Note that we may sometimes remove an option in a major release without having deprecated it before when prior deprecation wasn’t justified. For instance, options can be removed from the YAML file that way when that option is simply made invalid by the new major version.

== Downgrading ==

Downgrading is only guaranteed within minor releases and only to the previous bug-fix release (odd-numbered ones). So for instance, if you upgrade from 3.3 to 3.4 and have a problem, you can downgrade to 3.3, but not necessarily to 3.1. If you upgrade from 3.4 to 3.5, then you’ll be able to downgrade to both 3.4 or 3.3, but not necessarily any previous release.

We don’t provide downgrading guarantees between major releases: this is why is it crucial to test those updates carefully, and to keep a pre-upgrade snapshot of every node until your upgrade is complete and you haven’t noticed any major problem. 


== Developer guide ==

This section is at the attention of developers/contribute. It identifies which area of the code base is versioned and what and when change can be made to those area to respect the guarantees above:
 * CQL: this is the version of the language (not the implementation). The current “major” version is 3 and for now, there is no discussion of breaking compatibility in a major way so all versions are of the form 3.Y.Z. Historically, we’ve bumped the Y number on major releases of C* as they usually contained bigger additions to the language, and the Z number for small addition made in minor C* versions. ''When can we upgrade'': Overall, additive changes can be added in any C* version (minor or major). In very rare case, we might deprecate some syntax that has been replaced by something better, but this will be clearly documented (and ample time to update client code is given).
 * Native protocol: there is no backward compatibility between native protocol versions per-se, but a given C* version will always support at least the 2 last versions of the binary protocol to allow easy upgrade of the driver. Further, we will only remove support for a binary protocol versions in major release and coordinate with the maintainers of the main drivers to ensure we don’t deprecate a version until versions of the drivers supporting the newer native protocol versions are stable. ''When can we upgrade'': Given the guarantee on supporting previous protocol versions and the fact we don’t remove versions in minor releases, we can add new native protocol versions in any feature release (even if no driver support it yet, no harm done in adding one). We do however try to avoid adding new versions too often as it always require work from drivers maintainers, so we buffer multiple improvements before releasing a new version, instead of releasing one every time we want to change something.
 * Inter-node protocol (aka MessagingService version): Encompass everything that is sent on the wire between nodes (outside of streaming, see below), so every message serialization including the data serialization (the latter being currently shared with sstables). Backward compatibility is provided by keeping the ability to read the previous version version when a new version is created (node exchange their highest protocol version when connecting and agree on talking the smallest of these highest supported version). ''When can we upgrade'': In theory, since new node always talk at least the previous version of the protocol, we could allow bumping the version in minor releases. In practice, too much is at stake so we only allow such bump in major releases. Because we only support upgrading to the next major, this mean we can drop support for messaging protocol versions before the second to last version. Note that in practice this means we cannot make change to the serialization format (at least the messaging one) within minor releases. Note that this versioning is very coarse grained and a big limitation on what can be done in minor release. We will be working on getting much more finer grained versioning of the involved components (messages themselves, serialization format, schema, hints, ….) to make changes more manageable and allow some of them in minor upgrade. A single version for everything messaging is the current state however.
 * Streaming protocol: versions anything related to streaming. We currently do not provide backward compatibility on this, upgrading to a new version of streaming imply not being able to use streaming related functionality until the full cluster is upgraded. This is reason why repair, bootstrap, moving and decommission are temporarily unavailable during major upgrade. ''When can we upgrade'': due to the limitations above, we only ever bump the streaming version in major upgrade. As indicated earlier, we will be working on lifting this limitation for 4.x.
 * SSTables: we have minor and major version bumps for sstables. A minor version is limited to strictly backward compatible change, which mostly only mean adding new fields to the metadata (as old nodes will simply ignore them). Any change is allowed for a major version bump, but when a new major version is created, we must maintain code to read the previous major sstable version (we only preserve code for reading the old format however at the moment since we always write the most recent sstable version). ''When can we upgrade:'' We can do a minor upgrade anytime, but a major sstable version bump can only be done in major Cassandra version.