Re: [DISCUSS] Backport CASSANDRA-19800 to Cassandra-4.0, 4.1 and 5.0

[Sam Tunnicliffe <sa...@beobal.com>]

Sorry to derail the discussion but just on a point of order, there is actually precedent for requiring a minimum patch level before a major upgrade. For instance, from NEWS.txt:  

	Upgrade to 3.0 is supported from Cassandra 2.1 versions greater or equal to 2.1.9, or Cassandra 2.2 versions greater or equal to 2.2.2.

This approach has also been mentioned [1][2] as a means to introduce a property or setting to disable schema changes and the like before a major upgrade, though in this case it would be optional not required.

[1] https://issues.apache.org/jira/browse/CASSANDRA-19556?focusedCommentId=17848544&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17848544
[2] https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-21%3A+Transactional+Cluster+Metadata#CEP21:TransactionalClusterMetadata-MigrationPlan

> On 31 Jul 2024, at 22:13, C. Scott Andreas <sc...@paradoxica.net> wrote:
> 
> There are a few things unclear to me in this thread and the ticket, and details in the Jira are slim.
> 
> Yifan / others supportive of backporting this feature, could you help me answer these questions?
> 
> – What's this for? I'd appreciate a detailed explanation of what "Enhance CQLSSTableWriter to notify clients on sstable production" does and how it's meant to be used. Why is it needed for rolling upgrades? The phrasing of the ticket right now reads as nice-to-have rather than must-have. An earlier email described the value as "code quality and brings other benefits," but I'd expect the standard for feature backports to be higher.
> 
> – What's not possible if this isn't backported? What experience suffers today for lack of it / what problem does it solve? And what is the alternative/fallback if others are not supportive of backport?
> 
> – If this is deemed required for upgrade, that means users of previous releases would have to first upgrade to the latest release on their current train before upgrading to the latest major version. This is not a pattern that has been required in the past, and we should not introduce such a requirement. Do you intend this to be the required path for 4.1.x upgrades to 5.x+?
> 
> My general thoughts:
> 
> – The patch is small and the feature is small so I don't have much concern with a backport; zero-ish vote.
> – It definitely shouldn't block release of the current 4.1.x vote thread that's in progress.
> – We shouldn't introduce upgrade paths that require users to upgrade to at-minimum-patchlevel versions of current releases before upgrading to future majors; I'm -1 on that.
> 
> Thanks,
> 
> – Scott
> 
>> On Jul 31, 2024, at 2:04 PM, Jon Haddad <jo...@jonhaddad.com> wrote:
>> 
>> 
>> I'm kind of neutral on this, maybe -0.  It's a small enough patch, but it's of limited value, given that Cassandra Analytics doesn't work with vnodes. That's the overwhelming majority of deployments.  So I'm not really sure what we gain here.
>> 
>> On Wed, Jul 31, 2024 at 1:58 PM Yifan Cai <yc25code@gmail.com <ma...@gmail.com>> wrote:
>>> Hi PMC team, 
>>> 
>>> There are so far two +1 and one -1. Please vote if you want to. It is open for another 12 hours. 
>>> 
>>> 4.1 is to be released. I would like to include the patch, if possible, according to the vote result. 
>>> 
>>> I recognize that patches to stable releases can be risky. When talking about the trade off, we need to evaluate the benefit holistically, considering all the projects under the Cassandra umbrella. 
>>> We surely do not want to backport the user-facing Cassandra features and potentially demotivating upgrade. 
>>> IMO, the internal changes should be treated differently, as long as the public interface does not change. In this particular case, Cassandra Analytics provides the same bulk write feature regardless of backporting the patch or not. But having the patch backported improves the code quality and brings other benefits. Hope it answers Mick's message. 
>>> 
>>> Maybe it is time we start to think about categorizing the interfaces in Cassandra into 1) public API, 2) internal API, and 3) evolving API, etc. It is not a suitable topic for this thread and requires a separate discussion. 
>>> 
>>> - Yifan
>>> 
>>> On Tue, Jul 30, 2024 at 11:47 AM Mick Semb Wever <mck@apache.org <ma...@apache.org>> wrote:
>>>> reply below.
>>>> 
>>>>>  We're moving into a world where we will likely more frequently modify the mainline branch with new functionality to integrate with ecosystem changes (sidecar, analytics, drivers?). It's probably at least worth a conversation as to whether our current policy (improvements and new features main branch only) is optimal across everything equally or if there should be nuance for ecosystem integrations.
>>>> 
>>>> 
>>>> 
>>>> This also incentivises intentionally not introducing support for that api in older mainlines.  We KISS, if the user wants that ecosystem benefit they need to upgrade to at least mainline X.  
>>>> 
>>>> Once older mainlines have it then we have this problem.  An alternative to the risk of having to always update all the mainlines, is to let the ecosystem branch to provide support for the different mainlines as/when needed.  Both are painful.
> 
> 
> 
> 

Re: [DISCUSS] Backport CASSANDRA-19800 to Cassandra-4.0, 4.1 and 5.0

[Yifan Cai <yc...@gmail.com>]

I've realized that, although a vote is called, I forgot to create the
voting thread. I apologize for the oversight.

It looks like there's been no further discussion on this topic, so I'll go
ahead and set up a dedicated voting thread for the backport proposal
shortly.

Please vote once it is up.

- Yifan

On Thu, Aug 1, 2024 at 5:14 AM Sam Tunnicliffe <sa...@beobal.com> wrote:

> Sorry to derail the discussion but just on a point of order, there is
> actually precedent for requiring a minimum patch level before a major
> upgrade. For instance, from NEWS.txt:
>
> Upgrade to 3.0 is supported from Cassandra 2.1 versions greater or equal
> to 2.1.9, or Cassandra 2.2 versions greater or equal to 2.2.2.
>
> This approach has also been mentioned [1][2] as a means to introduce a
> property or setting to disable schema changes and the like before a major
> upgrade, though in this case it would be optional not required.
>
> [1]
> https://issues.apache.org/jira/browse/CASSANDRA-19556?focusedCommentId=17848544&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17848544
> [2]
> https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-21%3A+Transactional+Cluster+Metadata#CEP21:TransactionalClusterMetadata-MigrationPlan
>
> On 31 Jul 2024, at 22:13, C. Scott Andreas <sc...@paradoxica.net> wrote:
>
> There are a few things unclear to me in this thread and the ticket, and
> details in the Jira are slim.
>
> Yifan / others supportive of backporting this feature, could you help me
> answer these questions?
>
> – What's this for? I'd appreciate a detailed explanation of what "Enhance
> CQLSSTableWriter to notify clients on sstable production" does and how it's
> meant to be used. Why is it needed for rolling upgrades? The phrasing of
> the ticket right now reads as nice-to-have rather than must-have. An
> earlier email described the value as "code quality and brings other
> benefits," but I'd expect the standard for feature backports to be higher.
>
> – What's not possible if this isn't backported? What experience suffers
> today for lack of it / what problem does it solve? And what is the
> alternative/fallback if others are not supportive of backport?
>
> – If this is deemed required for upgrade, that means users of previous
> releases would have to first upgrade to the latest release on their current
> train before upgrading to the latest major version. This is not a pattern
> that has been required in the past, and we should not introduce such a
> requirement. Do you intend this to be the required path for 4.1.x upgrades
> to 5.x+?
>
> My general thoughts:
>
> – The patch is small and the feature is small so I don't have much concern
> with a backport; zero-ish vote.
> – It definitely shouldn't block release of the current 4.1.x vote thread
> that's in progress.
> – We shouldn't introduce upgrade paths that require users to upgrade to
> at-minimum-patchlevel versions of current releases before upgrading to
> future majors; I'm -1 on that.
>
> Thanks,
>
> – Scott
>
> On Jul 31, 2024, at 2:04 PM, Jon Haddad <jo...@jonhaddad.com> wrote:
>
>
> I'm kind of neutral on this, maybe -0.  It's a small enough patch, but
> it's of limited value, given that Cassandra Analytics doesn't work with
> vnodes. That's the overwhelming majority of deployments.  So I'm not really
> sure what we gain here.
>
> On Wed, Jul 31, 2024 at 1:58 PM Yifan Cai <yc...@gmail.com> wrote:
>
>> Hi PMC team,
>>
>> There are so far two +1 and one -1. Please vote if you want to. It is
>> open for another 12 hours.
>>
>> 4.1 is to be released. I would like to include the patch, if possible,
>> according to the vote result.
>>
>> I recognize that patches to stable releases can be risky. When talking
>> about the trade off, we need to evaluate the benefit holistically,
>> considering all the projects under the Cassandra umbrella.
>> We surely do not want to backport the user-facing Cassandra features and
>> potentially demotivating upgrade.
>> IMO, the internal changes should be treated differently, as long as the
>> public interface does not change. In this particular case, Cassandra
>> Analytics provides the same bulk write feature regardless of backporting
>> the patch or not. But having the patch backported improves the code quality
>> and brings other benefits. Hope it answers Mick's message.
>>
>> Maybe it is time we start to think about categorizing the interfaces in
>> Cassandra into 1) public API, 2) internal API, and 3) evolving API, etc. It
>> is not a suitable topic for this thread and requires a separate discussion.
>>
>> - Yifan
>>
>> On Tue, Jul 30, 2024 at 11:47 AM Mick Semb Wever <mc...@apache.org> wrote:
>>
>>> reply below.
>>>
>>>  We're moving into a world where we will likely more frequently modify
>>>> the mainline branch with new functionality to integrate with ecosystem
>>>> changes (sidecar, analytics, drivers?). It's probably at least worth a
>>>> conversation as to whether our current policy (improvements and new
>>>> features main branch only) is optimal across everything equally or if there
>>>> should be nuance for ecosystem integrations.
>>>>
>>>
>>>
>>> This also incentivises intentionally not introducing support for that
>>> api in older mainlines.  We KISS, if the user wants that ecosystem benefit
>>> they need to upgrade to at least mainline X.
>>>
>>> Once older mainlines have it then we have this problem.  An alternative
>>> to the risk of having to always update all the mainlines, is to let the
>>> ecosystem branch to provide support for the different mainlines as/when
>>> needed.  Both are painful.
>>>
>>
>
>
>
>
>