You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cassandra.apache.org by Ekaterina Dimitrova <ek...@datastax.com> on 2020/05/26 15:20:28 UTC
[DISCUSS] CASSANDRA-13994
Dear all,
Following the ticket review sent on 12th May I wanted to bring up
https://issues.apache.org/jira/browse/CASSANDRA-13994: Remove COMPACT
STORAGE internals before 4.0 release.
It is already under review by Dinesh Joshi and Alex Petrov. Not a
blocker but already under review.
Below are my responses to the questions brought up.
1) Would you block the release over this
ticket? - probably not
2) Would you prioritize this ticket over testing? - already
implemented but if there are some big changes needed after the review,
I doubt it we will want to prioritize over the testing
3) Does fixing
this ticket make 4.0 a more stable release? - I will just cite Alex
Petrov who reported this Jira and I think the rest of us would agree
with him here.
"I would say it's quite important to clean up compact storage
internals in 4.0 before the release. It should have no visible
side-effects, but it'd be very good to have as it simplifies multiple
code paths."
Ekaterina Dimitrova
e. ekaterina.dimitrova@datastax.com
w. www.datastax.com
Re: [DISCUSS] CASSANDRA-13994
Posted by Sylvain Lebresne <le...@gmail.com>.
There appears to be a rather important misunderstanding here.
Compact storage *is removed* from 4.0, it has been since at least
CASSANDRA-10857 which prevented 4.0 to start if any compact tables existed.
This was done 2.5+ years ago and is explicitly indicated in the NEWS file
since
then.
The ticket we're discussing, CASSANDRA-13994, truly is a minor and
unconsequential patch. It does not remove "COMPACT STORAGE", since it is
already removed, it removes some leftover "COMPACT STORAGE *internals*". In
other words, it removes *dead* code, code that cannot ever be run, and
that is just polluting the codebase. None of those changes are invasive. I
don't
know where the idea this patch was invasive came from, but it's just not.
It is not meant to have user-visible consequences and it won't impact any
(past
or future) testing in any meaningful way.
And yes, the patch does cover a non trivial amount of _lines_, but it's all
either the removal of clearly unreachable code, or near automatic changes.
So, tbc, my PMC and reviewer opinion is that this is fine to commit in
either
4.0-alpha or 4.0-beta. It cleans the internal, feels as safe as safe can be
and does not impact testing.
--
Sylvain
On Wed, Jun 24, 2020 at 9:03 PM Dinesh Joshi <dj...@apache.org> wrote:
> I might be missing some context here but I thought we did not intend to
> remove compact storage for 4.0 as it was deemed to be too invasive at this
> point. Did that decision change?
>
> Dinesh
>
> > On Jun 24, 2020, at 10:48 AM, Ekaterina Dimitrova <
> ekaterina.dimitrova@datastax.com> wrote:
> >
> > Dear all,
> > As we talked yesterday during our meeting, I am bringing back to the
> table
> > CASSANDRA-13994 <https://issues.apache.org/jira/browse/CASSANDRA-13994>.
> > Description
> >
> > 4.0 comes without thrift (after CASSANDRA-11115
> > <https://issues.apache.org/jira/browse/CASSANDRA-11115>) and COMPACT
> > STORAGE (after CASSANDRA-10857
> > <https://issues.apache.org/jira/browse/CASSANDRA-10857>), and since
> Compact
> > Storage flags are now disabled, all of the related functionality is
> useless.
> >
> > There are still some things to consider:
> > 1. One of the system tables (built indexes) was compact. For now, we just
> > added value column to it to make sure it's backwards-compatible, but we
> > might want to make sure it's just a "normal" table and doesn't have
> > redundant columns.
> > 2. Compact Tables were building indexes in KEYS mode. Removing it is
> > trivial, but this would mean that all built indexes will be defunct. We
> > could log a warning for now and ask users to migrate off those for now
> and
> > completely remove it from future releases. It's just a couple of classes
> > though.
> >
> >
> > Thank you Sylvain for the first pass of review.
> >
> > An option, as also Sylvain suggested, is to split the ticket in two
> tickets
> > - clean the Compact storage dead code in this one and open a new ticket
> to
> > handle the indexes.
> >
> > I am pasting here the latest comment, hopefully this makes it easier for
> > the audience:
> >
> > I have made a first pass of review and offered a few remarks above.
> >
> > But I think this ticket is hanging up on us deciding whether removing the
> > KEYS 2ndary index code is ok or not. And this yields, to me, the question
> > of what is the upgrade path to 4.0 for users that still have KEYS index
> > (which, reminder, could only be created with Thrift, but could *used*
> with
> > CQL and thus still be around).
> >
> > Because, while I haven't tested this myself, I suspect we have a hole
> here.
> >
> > Namely, KEYS index were compact tables, and 4.0 does not start if there
> is
> > still compact tables. And while for user tables, user are asked to use
> DROP
> > COMPACT STORAGE before upgrading, this cannot be done on KEYS index
> (there
> > is just no syntax to do it), so unless there is code I'm not aware of
> (and
> > please, someone correct me if I'm wrong), I don't think user can upgrade
> to
> > 4.0 at all if they still have KEYS index. They'd have to drop those index
> > first.
> >
> > So If I'm right here, this technically mean removing the KEYS index code
> in
> > 4.0 is fine, since you cannot upgrade in the first place if you have KEYS
> > index. But the more important question for 4.0 imo is what is the upgrade
> > path for users if they have a KEYS index in 3.X?
> >
> > Currently (without code changes), the only available option I can think
> of
> > is that before upgrade to 4.0, users would have to 1) drop their KEYS
> index
> > and then 2) re-create a "normal" (non-KEYS) equivalent index.
> >
> > Are we comfortable with that being the upgrade path for KEYS index?
> >
> > Personally, I'm not sure I am because this is not a seamless upgrade, as
> > between the 1) and 2) above, there is a window where there is no
> accessible
> > index, so if the user application rely on it, it means a period of
> downtime
> > for the application to perform the upgrade. However, if we want a more
> > seamless upgrade, we need to figure something out, and that probably
> > involve non trivial amounts of code and testing. And, playing devil's
> > advocate, KEYS index being so old, maybe nobody that plans to upgrade to
> > 4.0 have them anymore, and maybe it's not worth bothering?
> >
> > So I could use others opinions here.
> >
> > Tl;dr, this ticket raises the point that "Oops, I'm not sure we have
> though
> > through the question of upgrade to 4.0 for KEYS indexes". And tbc, it's
> not
> > directly related to this ticket, only indirectly, but it is still
> something
> > we need to figure out. And I'd say, before 4.0-alpha. But I'm happy to
> > create a separate ticket specific to that question if that helps.
> >
> >
> > Ekaterina
> >
> >>
> >>
> >> It was moved to alpha following the lifecycle document and the latest
> >> discussion on the ML. Sylvain made a review yesterday. To me the biggest
> >> question is about the indexing. (Posted questions on the ticket itself,
> >> waiting for opinion and agreement) For the rest of the ticket I support
> >> Sylvain that it is primarily removal of dead code.
> >>
> >> My understanding is either it lands in alpha before the big testing in
> >> beta or it is delayed for v.5
> >>
> >> Ekaterina
> >>
> >> On Wed, 10 Jun 2020 at 12:59, Joshua McKenzie <jm...@apache.org>
> >> wrote:
> >>
> >>> Where did we land on inclusion of this ticket in 4.0? Since it's now
> >>> flagged as blocking beta rel, I am now very interested in this
> question.
> >>>
> >>> :)
> >>>
> >>>
> >>> On Wed, May 27, 2020 at 7:52 PM Jordan West <jo...@gmail.com>
> wrote:
> >>>
> >>>> On Wed, May 27, 2020 at 1:23 PM Joshua McKenzie <jmckenzie@apache.org
> >
> >>>> wrote:
> >>>>
> >>>>> Maybe. Do we just time box, say we're going to cut an RC and give it
> 4
> >>>>> weeks, if nothing awful surfaces we GA?
> >>>>>
> >>>>
> >>>> I've seen that work well in the past on other projects. I agree with
> the
> >>>> notion that RCs are real candidates for release if no one finds issues
> >>> with
> >>>> them. Ideally we would have as little RCs as possible and have more
> >>>> alphas/betas.
> >>>>
> >>>>>
> >>>>> On Wed, May 27, 2020 at 4:12 PM Brandon Williams <dr...@gmail.com>
> >>>> wrote:
> >>>>>
> >>>>>> Absolutely my understanding.
> >>>>>>
> >>>>>> On Wed, May 27, 2020, 2:49 PM Jeremiah D Jordan <
> >>>>> jeremiah.jordan@gmail.com
> >>>>>>>
> >>>>>> wrote:
> >>>>>>
> >>>>>>>> A clear point to cut RC's doesn't surface from the above for me.
> >>>>>>> Releasing
> >>>>>>>> an RC before broad verification seems wrong, and cutting an RC
> >>>> after
> >>>>>> the
> >>>>>>> 4
> >>>>>>>> points above may as well be GA because it's all known scope.
> >>>>>>>
> >>>>>>> Isn’t the whole point of an RC is that it could be the GA? It is
> >>> a
> >>>>>>> “release candidate”, meaning if no one finds any issues with it,
> >>> that
> >>>>> can
> >>>>>>> them become the release? So that seems like exactly the right
> >>> time
> >>>> to
> >>>>>> make
> >>>>>>> RC releases?
> >>>>>>>
> >>>>>>>> On May 27, 2020, at 2:45 PM, Joshua McKenzie <
> >>> jmckenzie@apache.org
> >>>>>
> >>>>>>> wrote:
> >>>>>>>>
> >>>>>>>> I think we're all on the same page here; I was focusing more on
> >>> the
> >>>>>>> release
> >>>>>>>> lifecycles and sequencing than the entire version cycle. Good to
> >>>>>> broaden
> >>>>>>>> scope I think.
> >>>>>>>>
> >>>>>>>> One thing we're not considering is the separation of API changes
> >>>> from
> >>>>>>> major
> >>>>>>>> changes and how that intersects with release milestones.
> >>>>>>>>
> >>>>>>>> Meaning:
> >>>>>>>> 1. alpha phase
> >>>>>>>> 2. Milestone: API freeze (all API changes pushed to next major)
> >>>>>>>> 3. beta phase
> >>>>>>>> 4. Verification phase (all major disruptive pushed to next
> >>> major)
> >>>>>>>>
> >>>>>>>> A clear point to cut RC's doesn't surface from the above for me.
> >>>>>>> Releasing
> >>>>>>>> an RC before broad verification seems wrong, and cutting an RC
> >>>> after
> >>>>>> the
> >>>>>>> 4
> >>>>>>>> points above may as well be GA because it's all known scope.
> >>>>>>>>
> >>>>>>>> Thoughts?
> >>>>>>>>
> >>>>>>>> On Wed, May 27, 2020 at 3:28 PM Scott Andreas <
> >>>> scott@paradoxica.net>
> >>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> That makes sense to me, yep.
> >>>>>>>>>
> >>>>>>>>> My hope and expectation is that the time required for
> >>>> "verification
> >>>>>>> work"
> >>>>>>>>> will shrink dramatically in the not too distant future -
> >>> ideally
> >>>> to
> >>>>> a
> >>>>>>>>> period of less than a month. In this world, the cost of missing
> >>>> one
> >>>>>>> train
> >>>>>>>>> is reduced to catching the next one.
> >>>>>>>>>
> >>>>>>>>> One of the main goals in shifting focus from "testing" and
> >>> "test
> >>>>>> plans"
> >>>>>>> to
> >>>>>>>>> "test engineering" is automating as many aspects of release
> >>>>>>> qualification
> >>>>>>>>> as possible, with an asymptotic ideal as a function of compute
> >>>>>> capacity
> >>>>>>> and
> >>>>>>>>> time. While such automation will never be complete (it's likely
> >>>> that
> >>>>>>>>> development of new features will/must include qualification
> >>> infra
> >>>>>>> changes
> >>>>>>>>> to exercise them), if we're able to apply the same rigor to
> >>> major
> >>>>>>> releases
> >>>>>>>>> as we are to patchlevel builds with little incremental effort,
> >>> I'd
> >>>>> be
> >>>>>>>>> thrilled.
> >>>>>>>>>
> >>>>>>>>> This is mostly a way of saying:
> >>>>>>>>> – I like the cadence/sequencing Benedict proposes below.
> >>>>>>>>> – I think improvements in test engineering can reduce/eliminate
> >>>>>>>>> invalidation and may increase the scope of what can be a
> >>> candidate
> >>>>> for
> >>>>>>>>> merge on a given branch
> >>>>>>>>> – And if not, the cost of missing the train is lower because
> >>> we'll
> >>>>> be
> >>>>>>> able
> >>>>>>>>> to deliver major releases more often.
> >>>>>>>>>
> >>>>>>>>> Scott
> >>>>>>>>>
> >>>>>>>>> ________________________________________
> >>>>>>>>> From: Jeremiah D Jordan <je...@gmail.com>
> >>>>>>>>> Sent: Wednesday, May 27, 2020 11:54 AM
> >>>>>>>>> To: Cassandra DEV
> >>>>>>>>> Subject: Re: [DISCUSS] CASSANDRA-13994
> >>>>>>>>>
> >>>>>>>>> +1 strongly agree. If we aren’t going to let something go into
> >>>>> 4.0.0
> >>>>>>>>> because it would "invalidate testing” then we can not let such
> >>> a
> >>>>> thing
> >>>>>>> go
> >>>>>>>>> into 4.0.1 unless we plan to re-do said testing for the patch
> >>>>> release.
> >>>>>>>>>
> >>>>>>>>>> On May 27, 2020, at 1:31 PM, Benedict Elliott Smith <
> >>>>>>> benedict@apache.org>
> >>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>> I'm being told this still isn't clear, so let me try in a
> >>>>>> bullet-point
> >>>>>>>>> timeline:
> >>>>>>>>>>
> >>>>>>>>>> * 4.0 Beta
> >>>>>>>>>> * 4.0 Verification Work
> >>>>>>>>>> * [Merge Window]
> >>>>>>>>>> * 4.0 GA
> >>>>>>>>>> * 4.0 Minor Releases
> >>>>>>>>>> * ...
> >>>>>>>>>> * 5.0 Dev
> >>>>>>>>>> * ...
> >>>>>>>>>> * 5.0 Verification Work
> >>>>>>>>>> * GA 5.0
> >>>>>>>>>>
> >>>>>>>>>> I think that anything that is prohibited from "[Merge Window]"
> >>>>>> because
> >>>>>>>>> it invalidates "4.0 Verification Work" must also be prohibited
> >>>> until
> >>>>>>> "5.0
> >>>>>>>>> Dev" because the next equivalent work that can now validate it
> >>>>> occurs
> >>>>>>> only
> >>>>>>>>> at "5.0 Verification Work"
> >>>>>>>>>>
> >>>>>>>>>> On 27/05/2020, 19:05, "Benedict Elliott Smith" <
> >>>>> benedict@apache.org
> >>>>>>>
> >>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>> I'm not sure if I communicated my point very well. I mean
> >>> to
> >>>> say
> >>>>>>>>> that if the reason we are prohibiting a patch to land
> >>> post-beta is
> >>>>>>> because
> >>>>>>>>> it invalidates work we only perform pre-ga, then it probably
> >>>> should
> >>>>>> not
> >>>>>>> be
> >>>>>>>>> permitted to land post-ga either, since it must also invalidate
> >>>> the
> >>>>>> same
> >>>>>>>>> work?
> >>>>>>>>>>
> >>>>>>>>>> That is to say, if we're comfortable with work landing
> >>> post-ga
> >>>>>>>>> because we believe it to be safe to release without our
> >>>>>>> pre-major-release
> >>>>>>>>> verification, we should be comfortable with it landing at any
> >>> time
> >>>>>>> pre-ga
> >>>>>>>>> too. Anything else seems inconsistent to me, and we should
> >>>> examine
> >>>>>> what
> >>>>>>>>> assumptions we're making that permit this inconsistency to
> >>> arise.
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> On 27/05/2020, 18:49, "Joshua McKenzie" <
> >>> jmckenzie@apache.org
> >>>>>
> >>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> because it invalidates our pre-release verification, then it
> >>>>> should
> >>>>>>> not
> >>>>>>>>>>> land
> >>>>>>>>>>
> >>>>>>>>>> until we next perform pre-release verification
> >>>>>>>>>>
> >>>>>>>>>> At least for me there's a little softness around our
> >>>>> collective
> >>>>>>>>> alignment
> >>>>>>>>>> on when pre-release verification takes place. If it's
> >>>> between
> >>>>>>>>> alpha-1 and
> >>>>>>>>>> ga we don't want changes that would invalidate those
> >>>> changes
> >>>>> to
> >>>>>>>>> land during
> >>>>>>>>>> that time frame. Different for beta-1 to ga. We also
> >>> risk
> >>>>>>>>> invalidating
> >>>>>>>>>> testing if we do any of that testing before wherever
> >>> that
> >>>>>> cutoff
> >>>>>>>>> is, and a
> >>>>>>>>>> lack of clarity on that cutoff further muddies those
> >>>> waters.
> >>>>>>>>>>
> >>>>>>>>>> My very loosely held perspective is that beta-1 to ga is
> >>>> the
> >>>>>>>>> window in
> >>>>>>>>>> which we apply the "don't do things that will invalidate
> >>>>>>>>> verification", and
> >>>>>>>>>> we plan to do that verification during the beta phase. I
> >>>>>> *think*
> >>>>>>>>> this is
> >>>>>>>>>> consistent w/the current framing of the lifecycle doc.
> >>> That
> >>>>>> being
> >>>>>>>>> said, I
> >>>>>>>>>> don't have strong religion on this so if we collectively
> >>>> want
> >>>>>> to
> >>>>>>>>> call it
> >>>>>>>>>> "don't majorly disrupt from alpha-1 to ga", we can
> >>>> formalize
> >>>>>> that
> >>>>>>>>> in the
> >>>>>>>>>> docs and go ahead and triage current open scope for 4.0
> >>> and
> >>>>>> move
> >>>>>>>>> things out.
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> On Wed, May 27, 2020 at 12:59 PM Ekaterina Dimitrova <
> >>>>>>>>>> ekaterina.dimitrova@datastax.com> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Thank you all for your input.
> >>>>>>>>>>> I think an important topic is again to revise the lifecycle
> >>> and
> >>>>>> ensure
> >>>>>>>>> we
> >>>>>>>>>>> really have the vision on what is left until beta. I will
> >>> start
> >>>> a
> >>>>>>>>> separate
> >>>>>>>>>>> thread on the flaky tests situation soon.
> >>>>>>>>>>>
> >>>>>>>>>>> For this particular ticket I see a couple of things:
> >>>>>>>>>>> - There are a lot of deletions of already not used code
> >>>>>>>>>>> - I implemented it still in alpha as per our agreement that
> >>> this
> >>>>>> will
> >>>>>>>>> give
> >>>>>>>>>>> us enough time for testing. Probably Dinesh as a reviewer can
> >>>> give
> >>>>>>> some
> >>>>>>>>>>> valuable feedback/opinion on the patch.
> >>>>>>>>>>> - It definitely touches around important places but the
> >>>> important
> >>>>>>> thing
> >>>>>>>>> is
> >>>>>>>>>>> to see how exactly it touches, I think
> >>>>>>>>>>> - Considering it for alpha before the major testing in beta
> >>>> sounds
> >>>>>>>>>>> reasonable to me but I guess it also depends on people
> >>>>> availability
> >>>>>> to
> >>>>>>>>>>> review it in detail and the exact test plans afterwards
> >>>>>>>>>>>
> >>>>>>>>>>> On Wed, 27 May 2020 at 7:14, Benedict Elliott Smith <
> >>>>>>>>> benedict@apache.org>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> I think our "pre-beta" criteria should also be our "not in a
> >>>>> major"
> >>>>>>>>>>>> criteria.
> >>>>>>>>>>>>
> >>>>>>>>>>>> If work is prohibited because it invalidates our pre-release
> >>>>>>>>>>> verification,
> >>>>>>>>>>>> then it should not land until we next perform pre-release
> >>>>>>> verification,
> >>>>>>>>>>>> which only currently happens once per major.
> >>>>>>>>>>>>
> >>>>>>>>>>>> This could mean either landing less in a major, or
> >>> permitting
> >>>>> more
> >>>>>> in
> >>>>>>>>>>> beta
> >>>>>>>>>>>> etc.
> >>>>>>>>>>>>
> >>>>>>>>>>>> On 26/05/2020, 19:24, "Joshua McKenzie" <
> >>> jmckenzie@apache.org
> >>>>>
> >>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>> I think an interesting question that informs when to stop
> >>>>>> accepting
> >>>>>>>>>>>> specific changes in a release is when we expect any
> >>> extensive
> >>>>>>>>>>>> pre-release
> >>>>>>>>>>>> testing to take place.
> >>>>>>>>>>>>
> >>>>>>>>>>>> If we go by our release lifecycle, gutting deprecated code
> >>>>> seems
> >>>>>>>>>>>> compatible
> >>>>>>>>>>>> w/Alpha but I wouldn't endorse merging it into Beta:
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>
> >>>>>>
> >>>>
> https://cwiki.apache.org/confluence/display/CASSANDRA/Release+Lifecycle
> >>>>>>> .
> >>>>>>>>>>>> Since almost all of the 40_quality_testing epic stuff is
> >>> also
> >>>>>> beta
> >>>>>>>>>>>> phase
> >>>>>>>>>>>> and hasn't really taken off yet, it also seems like there
> >>>> will
> >>>>> be
> >>>>>>>>>>>> extensive
> >>>>>>>>>>>> testing after this phase transition.
> >>>>>>>>>>>>
> >>>>>>>>>>>> All that being said, I'd advocate for marking FixVer 4.x
> >>> to
> >>>>>>> indicate
> >>>>>>>>>>>> optionality and disallow merge of tickets like this after
> >>>> we're
> >>>>>>> done
> >>>>>>>>>>>> w/alpha phase in keeping w/our lifecycle doc in general.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Does that make sense? Should we consider revisiting and
> >>>>> revising
> >>>>>>> the
> >>>>>>>>>>>> lifecycle doc re: larger deprecation / changes and cycle
> >>>>> stages?
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Tue, May 26, 2020 at 12:53 PM Oleksandr Petrov <
> >>>>>>>>>>>> oleksandr.petrov@gmail.com> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>>> 1) Would you block the release over this ticket?
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> I would definitely not block the release on this ticket.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> 2) Would you prioritize this ticket over testing?
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Same here, I would prioritise testing.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> 3) Does fixing this ticket make 4.0 a more stable release?
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> I wanted to give some context: I wrote that in August 2018.
> >>>>> While
> >>>>>> I
> >>>>>>>>>>>> still
> >>>>>>>>>>>>> believe it is important to get rid of this code, I'm
> >>>> disinclined
> >>>>>> to
> >>>>>>>>>>>> merge
> >>>>>>>>>>>>> it into 4.0.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Given that the patch is rather big (421 additions and 1,480
> >>>>>>>>>>>> deletions) and
> >>>>>>>>>>>>> touches many important places, including parser, I would be
> >>>>>>>>>>> extremely
> >>>>>>>>>>>>> cautious to merge it that late in release cycle. It would
> >>> be
> >>>>> great
> >>>>>>>>>>>> to also
> >>>>>>>>>>>>> hear arguments that would justify the risk.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Thank you for starting this discussion,
> >>>>>>>>>>>>> -- Alex
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Tue, May 26, 2020 at 5:20 PM Ekaterina Dimitrova <
> >>>>>>>>>>>>> ekaterina.dimitrova@datastax.com> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> Dear all,
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Following the ticket review sent on 12th May I wanted to
> >>>> bring
> >>>>> up
> >>>>>>>>>>>>>> https://issues.apache.org/jira/browse/CASSANDRA-13994:
> >>>> Remove
> >>>>>>>>>>>> COMPACT
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> STORAGE internals before 4.0 release.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> It is already under review by Dinesh Joshi and Alex
> >>> Petrov.
> >>>>> Not a
> >>>>>>>>>>>>>> blocker but already under review.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Below are my responses to the questions brought up.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> 1) Would you block the release over this
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> ticket? - probably not
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> 2) Would you prioritize this ticket over testing? -
> >>> already
> >>>>>>>>>>>>>> implemented but if there are some big changes needed after
> >>>> the
> >>>>>>>>>>>> review,
> >>>>>>>>>>>>>> I doubt it we will want to prioritize over the testing
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> 3) Does fixing
> >>>>>>>>>>>>>> this ticket make 4.0 a more stable release? - I will just
> >>>> cite
> >>>>>>>>>>> Alex
> >>>>>>>>>>>>>> Petrov who reported this Jira and I think the rest of us
> >>>> would
> >>>>>>>>>>>> agree
> >>>>>>>>>>>>>> with him here.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> "I would say it's quite important to clean up compact
> >>> storage
> >>>>>>>>>>>>>> internals in 4.0 before the release. It should have no
> >>>> visible
> >>>>>>>>>>>>>> side-effects, but it'd be very good to have as it
> >>> simplifies
> >>>>>>>>>>>> multiple
> >>>>>>>>>>>>>> code paths."
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Ekaterina Dimitrova
> >>>>>>>>>>>>>> e. ekaterina.dimitrova@datastax.com
> >>>>>>>>>>>>>> w. www.datastax.com
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> --
> >>>>>>>>>>>>> alex p
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>
> >>> ---------------------------------------------------------------------
> >>>>>>>>>>>> To unsubscribe, e-mail:
> >>> dev-unsubscribe@cassandra.apache.org
> >>>>>>>>>>>> For additional commands, e-mail:
> >>> dev-help@cassandra.apache.org
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>
> >>> ---------------------------------------------------------------------
> >>>>>>>>>> To unsubscribe, e-mail:
> >>> dev-unsubscribe@cassandra.apache.org
> >>>>>>>>>> For additional commands, e-mail:
> >>> dev-help@cassandra.apache.org
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>> ---------------------------------------------------------------------
> >>>>>>>>>> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> >>>>>>>>>> For additional commands, e-mail:
> >>> dev-help@cassandra.apache.org
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>> ---------------------------------------------------------------------
> >>>>>>>>> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> >>>>>>>>> For additional commands, e-mail: dev-help@cassandra.apache.org
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>> ---------------------------------------------------------------------
> >>>>>>>>> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> >>>>>>>>> For additional commands, e-mail: dev-help@cassandra.apache.org
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>> ---------------------------------------------------------------------
> >>>>>>> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> >>>>>>> For additional commands, e-mail: dev-help@cassandra.apache.org
> >>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>>
> >>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> For additional commands, e-mail: dev-help@cassandra.apache.org
>
>
Re: [DISCUSS] CASSANDRA-13994
Posted by Joshua McKenzie <jm...@apache.org>.
The new release lifecycle guarantees changes these dynamics. Given our
commitment to API stability between beta 1 and GA, this has introduced a
time where changes being deferred to the next major due to API modification
will be unable to be merged unless we branch upon beta.
This warrants another discuss thread I think rather than nesting it here.
I'll take that on.
On Fri, Jun 26, 2020 at 4:38 AM Mick Semb Wever <mc...@apache.org> wrote:
> >
> > Also, what is the plan for cutting beta branch? I am still learning the
> > Apache/C* way so excuse me if I missed something. Looking at the
> Lifecycle
> > document, I see a point only about GA major version branch.
> > My understanding is that we are already in freeze for big changes. When
> > would be the right time for creating a beta branch? Something more than
> > closing the last beta blockers tickets?
>
>
>
> My understanding from the doc is that creating the release branch
> `cassandra-4.0` happens with GA. That means we're working on trunk until
> GA. Previously C* hasn't had any branches but these release branches, i.e.
> no beta or rc branches… (afaik)
>
> I would have thought it better to branch on RC? if at that point much of
> the testing efforts are subsiding, and folk are ready for other things.
>
Re: [DISCUSS] CASSANDRA-13994
Posted by Mick Semb Wever <mc...@apache.org>.
>
> Also, what is the plan for cutting beta branch? I am still learning the
> Apache/C* way so excuse me if I missed something. Looking at the Lifecycle
> document, I see a point only about GA major version branch.
> My understanding is that we are already in freeze for big changes. When
> would be the right time for creating a beta branch? Something more than
> closing the last beta blockers tickets?
My understanding from the doc is that creating the release branch
`cassandra-4.0` happens with GA. That means we're working on trunk until
GA. Previously C* hasn't had any branches but these release branches, i.e.
no beta or rc branches… (afaik)
I would have thought it better to branch on RC? if at that point much of
the testing efforts are subsiding, and folk are ready for other things.
Re: [DISCUSS] CASSANDRA-13994
Posted by Ekaterina Dimitrova <e....@gmail.com>.
Considering the latest comments on the ticket from today, I don’t see this
ticket anymore as a beta blocker.
Also, what is the plan for cutting beta branch? I am still learning the
Apache/C* way so excuse me if I missed something. Looking at the Lifecycle
document, I see a point only about GA major version branch.
My understanding is that we are already in freeze for big changes. When
would be the right time for creating a beta branch? Something more than
closing the last beta blockers tickets?
Thanks
Ekaterina
On Thu, 25 Jun 2020 at 14:25, Joshua McKenzie <jm...@apache.org> wrote:
> I don't think we really reached a consensus on this one. Some folks
> certainly vocalized a preference for not removing in 4.0, but I read it as
> debate still ongoing then losing momentum. Specifically thinking of
> Sylvain's comment here that kind of dead-ended:
>
> https://issues.apache.org/jira/browse/CASSANDRA-13994?focusedCommentId=17138278&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17138278
>
> fwiw I'm personally fine w/us deferring this until after 4.0 assuming we
> cut a branch for beta so changes like this don't bitrot.
>
> On Wed, Jun 24, 2020 at 3:03 PM Dinesh Joshi <dj...@apache.org> wrote:
>
> > I might be missing some context here but I thought we did not intend to
> > remove compact storage for 4.0 as it was deemed to be too invasive at
> this
> > point. Did that decision change?
> >
> > Dinesh
> >
> > > On Jun 24, 2020, at 10:48 AM, Ekaterina Dimitrova <
> > ekaterina.dimitrova@datastax.com> wrote:
> > >
> > > Dear all,
> > > As we talked yesterday during our meeting, I am bringing back to the
> > table
> > > CASSANDRA-13994 <https://issues.apache.org/jira/browse/CASSANDRA-13994
> >.
> > > Description
> > >
> > > 4.0 comes without thrift (after CASSANDRA-11115
> > > <https://issues.apache.org/jira/browse/CASSANDRA-11115>) and COMPACT
> > > STORAGE (after CASSANDRA-10857
> > > <https://issues.apache.org/jira/browse/CASSANDRA-10857>), and since
> > Compact
> > > Storage flags are now disabled, all of the related functionality is
> > useless.
> > >
> > > There are still some things to consider:
> > > 1. One of the system tables (built indexes) was compact. For now, we
> just
> > > added value column to it to make sure it's backwards-compatible, but we
> > > might want to make sure it's just a "normal" table and doesn't have
> > > redundant columns.
> > > 2. Compact Tables were building indexes in KEYS mode. Removing it is
> > > trivial, but this would mean that all built indexes will be defunct. We
> > > could log a warning for now and ask users to migrate off those for now
> > and
> > > completely remove it from future releases. It's just a couple of
> classes
> > > though.
> > >
> > >
> > > Thank you Sylvain for the first pass of review.
> > >
> > > An option, as also Sylvain suggested, is to split the ticket in two
> > tickets
> > > - clean the Compact storage dead code in this one and open a new ticket
> > to
> > > handle the indexes.
> > >
> > > I am pasting here the latest comment, hopefully this makes it easier
> for
> > > the audience:
> > >
> > > I have made a first pass of review and offered a few remarks above.
> > >
> > > But I think this ticket is hanging up on us deciding whether removing
> the
> > > KEYS 2ndary index code is ok or not. And this yields, to me, the
> question
> > > of what is the upgrade path to 4.0 for users that still have KEYS index
> > > (which, reminder, could only be created with Thrift, but could *used*
> > with
> > > CQL and thus still be around).
> > >
> > > Because, while I haven't tested this myself, I suspect we have a hole
> > here.
> > >
> > > Namely, KEYS index were compact tables, and 4.0 does not start if there
> > is
> > > still compact tables. And while for user tables, user are asked to use
> > DROP
> > > COMPACT STORAGE before upgrading, this cannot be done on KEYS index
> > (there
> > > is just no syntax to do it), so unless there is code I'm not aware of
> > (and
> > > please, someone correct me if I'm wrong), I don't think user can
> upgrade
> > to
> > > 4.0 at all if they still have KEYS index. They'd have to drop those
> index
> > > first.
> > >
> > > So If I'm right here, this technically mean removing the KEYS index
> code
> > in
> > > 4.0 is fine, since you cannot upgrade in the first place if you have
> KEYS
> > > index. But the more important question for 4.0 imo is what is the
> upgrade
> > > path for users if they have a KEYS index in 3.X?
> > >
> > > Currently (without code changes), the only available option I can think
> > of
> > > is that before upgrade to 4.0, users would have to 1) drop their KEYS
> > index
> > > and then 2) re-create a "normal" (non-KEYS) equivalent index.
> > >
> > > Are we comfortable with that being the upgrade path for KEYS index?
> > >
> > > Personally, I'm not sure I am because this is not a seamless upgrade,
> as
> > > between the 1) and 2) above, there is a window where there is no
> > accessible
> > > index, so if the user application rely on it, it means a period of
> > downtime
> > > for the application to perform the upgrade. However, if we want a more
> > > seamless upgrade, we need to figure something out, and that probably
> > > involve non trivial amounts of code and testing. And, playing devil's
> > > advocate, KEYS index being so old, maybe nobody that plans to upgrade
> to
> > > 4.0 have them anymore, and maybe it's not worth bothering?
> > >
> > > So I could use others opinions here.
> > >
> > > Tl;dr, this ticket raises the point that "Oops, I'm not sure we have
> > though
> > > through the question of upgrade to 4.0 for KEYS indexes". And tbc, it's
> > not
> > > directly related to this ticket, only indirectly, but it is still
> > something
> > > we need to figure out. And I'd say, before 4.0-alpha. But I'm happy to
> > > create a separate ticket specific to that question if that helps.
> > >
> > >
> > > Ekaterina
> > >
> > >>
> > >>
> > >> It was moved to alpha following the lifecycle document and the latest
> > >> discussion on the ML. Sylvain made a review yesterday. To me the
> biggest
> > >> question is about the indexing. (Posted questions on the ticket
> itself,
> > >> waiting for opinion and agreement) For the rest of the ticket I
> support
> > >> Sylvain that it is primarily removal of dead code.
> > >>
> > >> My understanding is either it lands in alpha before the big testing in
> > >> beta or it is delayed for v.5
> > >>
> > >> Ekaterina
> > >>
> > >> On Wed, 10 Jun 2020 at 12:59, Joshua McKenzie <jm...@apache.org>
> > >> wrote:
> > >>
> > >>> Where did we land on inclusion of this ticket in 4.0? Since it's now
> > >>> flagged as blocking beta rel, I am now very interested in this
> > question.
> > >>>
> > >>> :)
> > >>>
> > >>>
> > >>> On Wed, May 27, 2020 at 7:52 PM Jordan West <jo...@gmail.com>
> > wrote:
> > >>>
> > >>>> On Wed, May 27, 2020 at 1:23 PM Joshua McKenzie <
> jmckenzie@apache.org
> > >
> > >>>> wrote:
> > >>>>
> > >>>>> Maybe. Do we just time box, say we're going to cut an RC and give
> it
> > 4
> > >>>>> weeks, if nothing awful surfaces we GA?
> > >>>>>
> > >>>>
> > >>>> I've seen that work well in the past on other projects. I agree with
> > the
> > >>>> notion that RCs are real candidates for release if no one finds
> issues
> > >>> with
> > >>>> them. Ideally we would have as little RCs as possible and have more
> > >>>> alphas/betas.
> > >>>>
> > >>>>>
> > >>>>> On Wed, May 27, 2020 at 4:12 PM Brandon Williams <driftx@gmail.com
> >
> > >>>> wrote:
> > >>>>>
> > >>>>>> Absolutely my understanding.
> > >>>>>>
> > >>>>>> On Wed, May 27, 2020, 2:49 PM Jeremiah D Jordan <
> > >>>>> jeremiah.jordan@gmail.com
> > >>>>>>>
> > >>>>>> wrote:
> > >>>>>>
> > >>>>>>>> A clear point to cut RC's doesn't surface from the above for me.
> > >>>>>>> Releasing
> > >>>>>>>> an RC before broad verification seems wrong, and cutting an RC
> > >>>> after
> > >>>>>> the
> > >>>>>>> 4
> > >>>>>>>> points above may as well be GA because it's all known scope.
> > >>>>>>>
> > >>>>>>> Isn’t the whole point of an RC is that it could be the GA? It is
> > >>> a
> > >>>>>>> “release candidate”, meaning if no one finds any issues with it,
> > >>> that
> > >>>>> can
> > >>>>>>> them become the release? So that seems like exactly the right
> > >>> time
> > >>>> to
> > >>>>>> make
> > >>>>>>> RC releases?
> > >>>>>>>
> > >>>>>>>> On May 27, 2020, at 2:45 PM, Joshua McKenzie <
> > >>> jmckenzie@apache.org
> > >>>>>
> > >>>>>>> wrote:
> > >>>>>>>>
> > >>>>>>>> I think we're all on the same page here; I was focusing more on
> > >>> the
> > >>>>>>> release
> > >>>>>>>> lifecycles and sequencing than the entire version cycle. Good to
> > >>>>>> broaden
> > >>>>>>>> scope I think.
> > >>>>>>>>
> > >>>>>>>> One thing we're not considering is the separation of API changes
> > >>>> from
> > >>>>>>> major
> > >>>>>>>> changes and how that intersects with release milestones.
> > >>>>>>>>
> > >>>>>>>> Meaning:
> > >>>>>>>> 1. alpha phase
> > >>>>>>>> 2. Milestone: API freeze (all API changes pushed to next major)
> > >>>>>>>> 3. beta phase
> > >>>>>>>> 4. Verification phase (all major disruptive pushed to next
> > >>> major)
> > >>>>>>>>
> > >>>>>>>> A clear point to cut RC's doesn't surface from the above for me.
> > >>>>>>> Releasing
> > >>>>>>>> an RC before broad verification seems wrong, and cutting an RC
> > >>>> after
> > >>>>>> the
> > >>>>>>> 4
> > >>>>>>>> points above may as well be GA because it's all known scope.
> > >>>>>>>>
> > >>>>>>>> Thoughts?
> > >>>>>>>>
> > >>>>>>>> On Wed, May 27, 2020 at 3:28 PM Scott Andreas <
> > >>>> scott@paradoxica.net>
> > >>>>>>> wrote:
> > >>>>>>>>
> > >>>>>>>>> That makes sense to me, yep.
> > >>>>>>>>>
> > >>>>>>>>> My hope and expectation is that the time required for
> > >>>> "verification
> > >>>>>>> work"
> > >>>>>>>>> will shrink dramatically in the not too distant future -
> > >>> ideally
> > >>>> to
> > >>>>> a
> > >>>>>>>>> period of less than a month. In this world, the cost of missing
> > >>>> one
> > >>>>>>> train
> > >>>>>>>>> is reduced to catching the next one.
> > >>>>>>>>>
> > >>>>>>>>> One of the main goals in shifting focus from "testing" and
> > >>> "test
> > >>>>>> plans"
> > >>>>>>> to
> > >>>>>>>>> "test engineering" is automating as many aspects of release
> > >>>>>>> qualification
> > >>>>>>>>> as possible, with an asymptotic ideal as a function of compute
> > >>>>>> capacity
> > >>>>>>> and
> > >>>>>>>>> time. While such automation will never be complete (it's likely
> > >>>> that
> > >>>>>>>>> development of new features will/must include qualification
> > >>> infra
> > >>>>>>> changes
> > >>>>>>>>> to exercise them), if we're able to apply the same rigor to
> > >>> major
> > >>>>>>> releases
> > >>>>>>>>> as we are to patchlevel builds with little incremental effort,
> > >>> I'd
> > >>>>> be
> > >>>>>>>>> thrilled.
> > >>>>>>>>>
> > >>>>>>>>> This is mostly a way of saying:
> > >>>>>>>>> – I like the cadence/sequencing Benedict proposes below.
> > >>>>>>>>> – I think improvements in test engineering can reduce/eliminate
> > >>>>>>>>> invalidation and may increase the scope of what can be a
> > >>> candidate
> > >>>>> for
> > >>>>>>>>> merge on a given branch
> > >>>>>>>>> – And if not, the cost of missing the train is lower because
> > >>> we'll
> > >>>>> be
> > >>>>>>> able
> > >>>>>>>>> to deliver major releases more often.
> > >>>>>>>>>
> > >>>>>>>>> Scott
> > >>>>>>>>>
> > >>>>>>>>> ________________________________________
> > >>>>>>>>> From: Jeremiah D Jordan <je...@gmail.com>
> > >>>>>>>>> Sent: Wednesday, May 27, 2020 11:54 AM
> > >>>>>>>>> To: Cassandra DEV
> > >>>>>>>>> Subject: Re: [DISCUSS] CASSANDRA-13994
> > >>>>>>>>>
> > >>>>>>>>> +1 strongly agree. If we aren’t going to let something go into
> > >>>>> 4.0.0
> > >>>>>>>>> because it would "invalidate testing” then we can not let such
> > >>> a
> > >>>>> thing
> > >>>>>>> go
> > >>>>>>>>> into 4.0.1 unless we plan to re-do said testing for the patch
> > >>>>> release.
> > >>>>>>>>>
> > >>>>>>>>>> On May 27, 2020, at 1:31 PM, Benedict Elliott Smith <
> > >>>>>>> benedict@apache.org>
> > >>>>>>>>> wrote:
> > >>>>>>>>>>
> > >>>>>>>>>> I'm being told this still isn't clear, so let me try in a
> > >>>>>> bullet-point
> > >>>>>>>>> timeline:
> > >>>>>>>>>>
> > >>>>>>>>>> * 4.0 Beta
> > >>>>>>>>>> * 4.0 Verification Work
> > >>>>>>>>>> * [Merge Window]
> > >>>>>>>>>> * 4.0 GA
> > >>>>>>>>>> * 4.0 Minor Releases
> > >>>>>>>>>> * ...
> > >>>>>>>>>> * 5.0 Dev
> > >>>>>>>>>> * ...
> > >>>>>>>>>> * 5.0 Verification Work
> > >>>>>>>>>> * GA 5.0
> > >>>>>>>>>>
> > >>>>>>>>>> I think that anything that is prohibited from "[Merge Window]"
> > >>>>>> because
> > >>>>>>>>> it invalidates "4.0 Verification Work" must also be prohibited
> > >>>> until
> > >>>>>>> "5.0
> > >>>>>>>>> Dev" because the next equivalent work that can now validate it
> > >>>>> occurs
> > >>>>>>> only
> > >>>>>>>>> at "5.0 Verification Work"
> > >>>>>>>>>>
> > >>>>>>>>>> On 27/05/2020, 19:05, "Benedict Elliott Smith" <
> > >>>>> benedict@apache.org
> > >>>>>>>
> > >>>>>>>>> wrote:
> > >>>>>>>>>>
> > >>>>>>>>>> I'm not sure if I communicated my point very well. I mean
> > >>> to
> > >>>> say
> > >>>>>>>>> that if the reason we are prohibiting a patch to land
> > >>> post-beta is
> > >>>>>>> because
> > >>>>>>>>> it invalidates work we only perform pre-ga, then it probably
> > >>>> should
> > >>>>>> not
> > >>>>>>> be
> > >>>>>>>>> permitted to land post-ga either, since it must also invalidate
> > >>>> the
> > >>>>>> same
> > >>>>>>>>> work?
> > >>>>>>>>>>
> > >>>>>>>>>> That is to say, if we're comfortable with work landing
> > >>> post-ga
> > >>>>>>>>> because we believe it to be safe to release without our
> > >>>>>>> pre-major-release
> > >>>>>>>>> verification, we should be comfortable with it landing at any
> > >>> time
> > >>>>>>> pre-ga
> > >>>>>>>>> too. Anything else seems inconsistent to me, and we should
> > >>>> examine
> > >>>>>> what
> > >>>>>>>>> assumptions we're making that permit this inconsistency to
> > >>> arise.
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>> On 27/05/2020, 18:49, "Joshua McKenzie" <
> > >>> jmckenzie@apache.org
> > >>>>>
> > >>>>>>>>> wrote:
> > >>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>> because it invalidates our pre-release verification, then it
> > >>>>> should
> > >>>>>>> not
> > >>>>>>>>>>> land
> > >>>>>>>>>>
> > >>>>>>>>>> until we next perform pre-release verification
> > >>>>>>>>>>
> > >>>>>>>>>> At least for me there's a little softness around our
> > >>>>> collective
> > >>>>>>>>> alignment
> > >>>>>>>>>> on when pre-release verification takes place. If it's
> > >>>> between
> > >>>>>>>>> alpha-1 and
> > >>>>>>>>>> ga we don't want changes that would invalidate those
> > >>>> changes
> > >>>>> to
> > >>>>>>>>> land during
> > >>>>>>>>>> that time frame. Different for beta-1 to ga. We also
> > >>> risk
> > >>>>>>>>> invalidating
> > >>>>>>>>>> testing if we do any of that testing before wherever
> > >>> that
> > >>>>>> cutoff
> > >>>>>>>>> is, and a
> > >>>>>>>>>> lack of clarity on that cutoff further muddies those
> > >>>> waters.
> > >>>>>>>>>>
> > >>>>>>>>>> My very loosely held perspective is that beta-1 to ga is
> > >>>> the
> > >>>>>>>>> window in
> > >>>>>>>>>> which we apply the "don't do things that will invalidate
> > >>>>>>>>> verification", and
> > >>>>>>>>>> we plan to do that verification during the beta phase. I
> > >>>>>> *think*
> > >>>>>>>>> this is
> > >>>>>>>>>> consistent w/the current framing of the lifecycle doc.
> > >>> That
> > >>>>>> being
> > >>>>>>>>> said, I
> > >>>>>>>>>> don't have strong religion on this so if we collectively
> > >>>> want
> > >>>>>> to
> > >>>>>>>>> call it
> > >>>>>>>>>> "don't majorly disrupt from alpha-1 to ga", we can
> > >>>> formalize
> > >>>>>> that
> > >>>>>>>>> in the
> > >>>>>>>>>> docs and go ahead and triage current open scope for 4.0
> > >>> and
> > >>>>>> move
> > >>>>>>>>> things out.
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>> On Wed, May 27, 2020 at 12:59 PM Ekaterina Dimitrova <
> > >>>>>>>>>> ekaterina.dimitrova@datastax.com> wrote:
> > >>>>>>>>>>
> > >>>>>>>>>>> Thank you all for your input.
> > >>>>>>>>>>> I think an important topic is again to revise the lifecycle
> > >>> and
> > >>>>>> ensure
> > >>>>>>>>> we
> > >>>>>>>>>>> really have the vision on what is left until beta. I will
> > >>> start
> > >>>> a
> > >>>>>>>>> separate
> > >>>>>>>>>>> thread on the flaky tests situation soon.
> > >>>>>>>>>>>
> > >>>>>>>>>>> For this particular ticket I see a couple of things:
> > >>>>>>>>>>> - There are a lot of deletions of already not used code
> > >>>>>>>>>>> - I implemented it still in alpha as per our agreement that
> > >>> this
> > >>>>>> will
> > >>>>>>>>> give
> > >>>>>>>>>>> us enough time for testing. Probably Dinesh as a reviewer can
> > >>>> give
> > >>>>>>> some
> > >>>>>>>>>>> valuable feedback/opinion on the patch.
> > >>>>>>>>>>> - It definitely touches around important places but the
> > >>>> important
> > >>>>>>> thing
> > >>>>>>>>> is
> > >>>>>>>>>>> to see how exactly it touches, I think
> > >>>>>>>>>>> - Considering it for alpha before the major testing in beta
> > >>>> sounds
> > >>>>>>>>>>> reasonable to me but I guess it also depends on people
> > >>>>> availability
> > >>>>>> to
> > >>>>>>>>>>> review it in detail and the exact test plans afterwards
> > >>>>>>>>>>>
> > >>>>>>>>>>> On Wed, 27 May 2020 at 7:14, Benedict Elliott Smith <
> > >>>>>>>>> benedict@apache.org>
> > >>>>>>>>>>> wrote:
> > >>>>>>>>>>>
> > >>>>>>>>>>>> I think our "pre-beta" criteria should also be our "not in a
> > >>>>> major"
> > >>>>>>>>>>>> criteria.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> If work is prohibited because it invalidates our pre-release
> > >>>>>>>>>>> verification,
> > >>>>>>>>>>>> then it should not land until we next perform pre-release
> > >>>>>>> verification,
> > >>>>>>>>>>>> which only currently happens once per major.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> This could mean either landing less in a major, or
> > >>> permitting
> > >>>>> more
> > >>>>>> in
> > >>>>>>>>>>> beta
> > >>>>>>>>>>>> etc.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> On 26/05/2020, 19:24, "Joshua McKenzie" <
> > >>> jmckenzie@apache.org
> > >>>>>
> > >>>>>>> wrote:
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> I think an interesting question that informs when to stop
> > >>>>>> accepting
> > >>>>>>>>>>>> specific changes in a release is when we expect any
> > >>> extensive
> > >>>>>>>>>>>> pre-release
> > >>>>>>>>>>>> testing to take place.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> If we go by our release lifecycle, gutting deprecated code
> > >>>>> seems
> > >>>>>>>>>>>> compatible
> > >>>>>>>>>>>> w/Alpha but I wouldn't endorse merging it into Beta:
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>
> > >>>>>>
> > >>>>
> > https://cwiki.apache.org/confluence/display/CASSANDRA/Release+Lifecycle
> > >>>>>>> .
> > >>>>>>>>>>>> Since almost all of the 40_quality_testing epic stuff is
> > >>> also
> > >>>>>> beta
> > >>>>>>>>>>>> phase
> > >>>>>>>>>>>> and hasn't really taken off yet, it also seems like there
> > >>>> will
> > >>>>> be
> > >>>>>>>>>>>> extensive
> > >>>>>>>>>>>> testing after this phase transition.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> All that being said, I'd advocate for marking FixVer 4.x
> > >>> to
> > >>>>>>> indicate
> > >>>>>>>>>>>> optionality and disallow merge of tickets like this after
> > >>>> we're
> > >>>>>>> done
> > >>>>>>>>>>>> w/alpha phase in keeping w/our lifecycle doc in general.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> Does that make sense? Should we consider revisiting and
> > >>>>> revising
> > >>>>>>> the
> > >>>>>>>>>>>> lifecycle doc re: larger deprecation / changes and cycle
> > >>>>> stages?
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> On Tue, May 26, 2020 at 12:53 PM Oleksandr Petrov <
> > >>>>>>>>>>>> oleksandr.petrov@gmail.com> wrote:
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>>> 1) Would you block the release over this ticket?
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> I would definitely not block the release on this ticket.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>> 2) Would you prioritize this ticket over testing?
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> Same here, I would prioritise testing.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>> 3) Does fixing this ticket make 4.0 a more stable release?
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> I wanted to give some context: I wrote that in August 2018.
> > >>>>> While
> > >>>>>> I
> > >>>>>>>>>>>> still
> > >>>>>>>>>>>>> believe it is important to get rid of this code, I'm
> > >>>> disinclined
> > >>>>>> to
> > >>>>>>>>>>>> merge
> > >>>>>>>>>>>>> it into 4.0.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> Given that the patch is rather big (421 additions and 1,480
> > >>>>>>>>>>>> deletions) and
> > >>>>>>>>>>>>> touches many important places, including parser, I would be
> > >>>>>>>>>>> extremely
> > >>>>>>>>>>>>> cautious to merge it that late in release cycle. It would
> > >>> be
> > >>>>> great
> > >>>>>>>>>>>> to also
> > >>>>>>>>>>>>> hear arguments that would justify the risk.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> Thank you for starting this discussion,
> > >>>>>>>>>>>>> -- Alex
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> On Tue, May 26, 2020 at 5:20 PM Ekaterina Dimitrova <
> > >>>>>>>>>>>>> ekaterina.dimitrova@datastax.com> wrote:
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Dear all,
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Following the ticket review sent on 12th May I wanted to
> > >>>> bring
> > >>>>> up
> > >>>>>>>>>>>>>> https://issues.apache.org/jira/browse/CASSANDRA-13994:
> > >>>> Remove
> > >>>>>>>>>>>> COMPACT
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> STORAGE internals before 4.0 release.
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> It is already under review by Dinesh Joshi and Alex
> > >>> Petrov.
> > >>>>> Not a
> > >>>>>>>>>>>>>> blocker but already under review.
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Below are my responses to the questions brought up.
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> 1) Would you block the release over this
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> ticket? - probably not
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> 2) Would you prioritize this ticket over testing? -
> > >>> already
> > >>>>>>>>>>>>>> implemented but if there are some big changes needed after
> > >>>> the
> > >>>>>>>>>>>> review,
> > >>>>>>>>>>>>>> I doubt it we will want to prioritize over the testing
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> 3) Does fixing
> > >>>>>>>>>>>>>> this ticket make 4.0 a more stable release? - I will just
> > >>>> cite
> > >>>>>>>>>>> Alex
> > >>>>>>>>>>>>>> Petrov who reported this Jira and I think the rest of us
> > >>>> would
> > >>>>>>>>>>>> agree
> > >>>>>>>>>>>>>> with him here.
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> "I would say it's quite important to clean up compact
> > >>> storage
> > >>>>>>>>>>>>>> internals in 4.0 before the release. It should have no
> > >>>> visible
> > >>>>>>>>>>>>>> side-effects, but it'd be very good to have as it
> > >>> simplifies
> > >>>>>>>>>>>> multiple
> > >>>>>>>>>>>>>> code paths."
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Ekaterina Dimitrova
> > >>>>>>>>>>>>>> e. ekaterina.dimitrova@datastax.com
> > >>>>>>>>>>>>>> w. www.datastax.com
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> --
> > >>>>>>>>>>>>> alex p
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>
> > >>> ---------------------------------------------------------------------
> > >>>>>>>>>>>> To unsubscribe, e-mail:
> > >>> dev-unsubscribe@cassandra.apache.org
> > >>>>>>>>>>>> For additional commands, e-mail:
> > >>> dev-help@cassandra.apache.org
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>
> > >>> ---------------------------------------------------------------------
> > >>>>>>>>>> To unsubscribe, e-mail:
> > >>> dev-unsubscribe@cassandra.apache.org
> > >>>>>>>>>> For additional commands, e-mail:
> > >>> dev-help@cassandra.apache.org
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>
> ---------------------------------------------------------------------
> > >>>>>>>>>> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> > >>>>>>>>>> For additional commands, e-mail:
> > >>> dev-help@cassandra.apache.org
> > >>>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>
> ---------------------------------------------------------------------
> > >>>>>>>>> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> > >>>>>>>>> For additional commands, e-mail: dev-help@cassandra.apache.org
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>
> ---------------------------------------------------------------------
> > >>>>>>>>> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> > >>>>>>>>> For additional commands, e-mail: dev-help@cassandra.apache.org
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>> ---------------------------------------------------------------------
> > >>>>>>> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> > >>>>>>> For additional commands, e-mail: dev-help@cassandra.apache.org
> > >>>>>>>
> > >>>>>>>
> > >>>>>>
> > >>>>>
> > >>>>
> > >>>
> > >>
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> > For additional commands, e-mail: dev-help@cassandra.apache.org
> >
> >
>
Re: [DISCUSS] CASSANDRA-13994
Posted by Joshua McKenzie <jm...@apache.org>.
I don't think we really reached a consensus on this one. Some folks
certainly vocalized a preference for not removing in 4.0, but I read it as
debate still ongoing then losing momentum. Specifically thinking of
Sylvain's comment here that kind of dead-ended:
https://issues.apache.org/jira/browse/CASSANDRA-13994?focusedCommentId=17138278&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17138278
fwiw I'm personally fine w/us deferring this until after 4.0 assuming we
cut a branch for beta so changes like this don't bitrot.
On Wed, Jun 24, 2020 at 3:03 PM Dinesh Joshi <dj...@apache.org> wrote:
> I might be missing some context here but I thought we did not intend to
> remove compact storage for 4.0 as it was deemed to be too invasive at this
> point. Did that decision change?
>
> Dinesh
>
> > On Jun 24, 2020, at 10:48 AM, Ekaterina Dimitrova <
> ekaterina.dimitrova@datastax.com> wrote:
> >
> > Dear all,
> > As we talked yesterday during our meeting, I am bringing back to the
> table
> > CASSANDRA-13994 <https://issues.apache.org/jira/browse/CASSANDRA-13994>.
> > Description
> >
> > 4.0 comes without thrift (after CASSANDRA-11115
> > <https://issues.apache.org/jira/browse/CASSANDRA-11115>) and COMPACT
> > STORAGE (after CASSANDRA-10857
> > <https://issues.apache.org/jira/browse/CASSANDRA-10857>), and since
> Compact
> > Storage flags are now disabled, all of the related functionality is
> useless.
> >
> > There are still some things to consider:
> > 1. One of the system tables (built indexes) was compact. For now, we just
> > added value column to it to make sure it's backwards-compatible, but we
> > might want to make sure it's just a "normal" table and doesn't have
> > redundant columns.
> > 2. Compact Tables were building indexes in KEYS mode. Removing it is
> > trivial, but this would mean that all built indexes will be defunct. We
> > could log a warning for now and ask users to migrate off those for now
> and
> > completely remove it from future releases. It's just a couple of classes
> > though.
> >
> >
> > Thank you Sylvain for the first pass of review.
> >
> > An option, as also Sylvain suggested, is to split the ticket in two
> tickets
> > - clean the Compact storage dead code in this one and open a new ticket
> to
> > handle the indexes.
> >
> > I am pasting here the latest comment, hopefully this makes it easier for
> > the audience:
> >
> > I have made a first pass of review and offered a few remarks above.
> >
> > But I think this ticket is hanging up on us deciding whether removing the
> > KEYS 2ndary index code is ok or not. And this yields, to me, the question
> > of what is the upgrade path to 4.0 for users that still have KEYS index
> > (which, reminder, could only be created with Thrift, but could *used*
> with
> > CQL and thus still be around).
> >
> > Because, while I haven't tested this myself, I suspect we have a hole
> here.
> >
> > Namely, KEYS index were compact tables, and 4.0 does not start if there
> is
> > still compact tables. And while for user tables, user are asked to use
> DROP
> > COMPACT STORAGE before upgrading, this cannot be done on KEYS index
> (there
> > is just no syntax to do it), so unless there is code I'm not aware of
> (and
> > please, someone correct me if I'm wrong), I don't think user can upgrade
> to
> > 4.0 at all if they still have KEYS index. They'd have to drop those index
> > first.
> >
> > So If I'm right here, this technically mean removing the KEYS index code
> in
> > 4.0 is fine, since you cannot upgrade in the first place if you have KEYS
> > index. But the more important question for 4.0 imo is what is the upgrade
> > path for users if they have a KEYS index in 3.X?
> >
> > Currently (without code changes), the only available option I can think
> of
> > is that before upgrade to 4.0, users would have to 1) drop their KEYS
> index
> > and then 2) re-create a "normal" (non-KEYS) equivalent index.
> >
> > Are we comfortable with that being the upgrade path for KEYS index?
> >
> > Personally, I'm not sure I am because this is not a seamless upgrade, as
> > between the 1) and 2) above, there is a window where there is no
> accessible
> > index, so if the user application rely on it, it means a period of
> downtime
> > for the application to perform the upgrade. However, if we want a more
> > seamless upgrade, we need to figure something out, and that probably
> > involve non trivial amounts of code and testing. And, playing devil's
> > advocate, KEYS index being so old, maybe nobody that plans to upgrade to
> > 4.0 have them anymore, and maybe it's not worth bothering?
> >
> > So I could use others opinions here.
> >
> > Tl;dr, this ticket raises the point that "Oops, I'm not sure we have
> though
> > through the question of upgrade to 4.0 for KEYS indexes". And tbc, it's
> not
> > directly related to this ticket, only indirectly, but it is still
> something
> > we need to figure out. And I'd say, before 4.0-alpha. But I'm happy to
> > create a separate ticket specific to that question if that helps.
> >
> >
> > Ekaterina
> >
> >>
> >>
> >> It was moved to alpha following the lifecycle document and the latest
> >> discussion on the ML. Sylvain made a review yesterday. To me the biggest
> >> question is about the indexing. (Posted questions on the ticket itself,
> >> waiting for opinion and agreement) For the rest of the ticket I support
> >> Sylvain that it is primarily removal of dead code.
> >>
> >> My understanding is either it lands in alpha before the big testing in
> >> beta or it is delayed for v.5
> >>
> >> Ekaterina
> >>
> >> On Wed, 10 Jun 2020 at 12:59, Joshua McKenzie <jm...@apache.org>
> >> wrote:
> >>
> >>> Where did we land on inclusion of this ticket in 4.0? Since it's now
> >>> flagged as blocking beta rel, I am now very interested in this
> question.
> >>>
> >>> :)
> >>>
> >>>
> >>> On Wed, May 27, 2020 at 7:52 PM Jordan West <jo...@gmail.com>
> wrote:
> >>>
> >>>> On Wed, May 27, 2020 at 1:23 PM Joshua McKenzie <jmckenzie@apache.org
> >
> >>>> wrote:
> >>>>
> >>>>> Maybe. Do we just time box, say we're going to cut an RC and give it
> 4
> >>>>> weeks, if nothing awful surfaces we GA?
> >>>>>
> >>>>
> >>>> I've seen that work well in the past on other projects. I agree with
> the
> >>>> notion that RCs are real candidates for release if no one finds issues
> >>> with
> >>>> them. Ideally we would have as little RCs as possible and have more
> >>>> alphas/betas.
> >>>>
> >>>>>
> >>>>> On Wed, May 27, 2020 at 4:12 PM Brandon Williams <dr...@gmail.com>
> >>>> wrote:
> >>>>>
> >>>>>> Absolutely my understanding.
> >>>>>>
> >>>>>> On Wed, May 27, 2020, 2:49 PM Jeremiah D Jordan <
> >>>>> jeremiah.jordan@gmail.com
> >>>>>>>
> >>>>>> wrote:
> >>>>>>
> >>>>>>>> A clear point to cut RC's doesn't surface from the above for me.
> >>>>>>> Releasing
> >>>>>>>> an RC before broad verification seems wrong, and cutting an RC
> >>>> after
> >>>>>> the
> >>>>>>> 4
> >>>>>>>> points above may as well be GA because it's all known scope.
> >>>>>>>
> >>>>>>> Isn’t the whole point of an RC is that it could be the GA? It is
> >>> a
> >>>>>>> “release candidate”, meaning if no one finds any issues with it,
> >>> that
> >>>>> can
> >>>>>>> them become the release? So that seems like exactly the right
> >>> time
> >>>> to
> >>>>>> make
> >>>>>>> RC releases?
> >>>>>>>
> >>>>>>>> On May 27, 2020, at 2:45 PM, Joshua McKenzie <
> >>> jmckenzie@apache.org
> >>>>>
> >>>>>>> wrote:
> >>>>>>>>
> >>>>>>>> I think we're all on the same page here; I was focusing more on
> >>> the
> >>>>>>> release
> >>>>>>>> lifecycles and sequencing than the entire version cycle. Good to
> >>>>>> broaden
> >>>>>>>> scope I think.
> >>>>>>>>
> >>>>>>>> One thing we're not considering is the separation of API changes
> >>>> from
> >>>>>>> major
> >>>>>>>> changes and how that intersects with release milestones.
> >>>>>>>>
> >>>>>>>> Meaning:
> >>>>>>>> 1. alpha phase
> >>>>>>>> 2. Milestone: API freeze (all API changes pushed to next major)
> >>>>>>>> 3. beta phase
> >>>>>>>> 4. Verification phase (all major disruptive pushed to next
> >>> major)
> >>>>>>>>
> >>>>>>>> A clear point to cut RC's doesn't surface from the above for me.
> >>>>>>> Releasing
> >>>>>>>> an RC before broad verification seems wrong, and cutting an RC
> >>>> after
> >>>>>> the
> >>>>>>> 4
> >>>>>>>> points above may as well be GA because it's all known scope.
> >>>>>>>>
> >>>>>>>> Thoughts?
> >>>>>>>>
> >>>>>>>> On Wed, May 27, 2020 at 3:28 PM Scott Andreas <
> >>>> scott@paradoxica.net>
> >>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> That makes sense to me, yep.
> >>>>>>>>>
> >>>>>>>>> My hope and expectation is that the time required for
> >>>> "verification
> >>>>>>> work"
> >>>>>>>>> will shrink dramatically in the not too distant future -
> >>> ideally
> >>>> to
> >>>>> a
> >>>>>>>>> period of less than a month. In this world, the cost of missing
> >>>> one
> >>>>>>> train
> >>>>>>>>> is reduced to catching the next one.
> >>>>>>>>>
> >>>>>>>>> One of the main goals in shifting focus from "testing" and
> >>> "test
> >>>>>> plans"
> >>>>>>> to
> >>>>>>>>> "test engineering" is automating as many aspects of release
> >>>>>>> qualification
> >>>>>>>>> as possible, with an asymptotic ideal as a function of compute
> >>>>>> capacity
> >>>>>>> and
> >>>>>>>>> time. While such automation will never be complete (it's likely
> >>>> that
> >>>>>>>>> development of new features will/must include qualification
> >>> infra
> >>>>>>> changes
> >>>>>>>>> to exercise them), if we're able to apply the same rigor to
> >>> major
> >>>>>>> releases
> >>>>>>>>> as we are to patchlevel builds with little incremental effort,
> >>> I'd
> >>>>> be
> >>>>>>>>> thrilled.
> >>>>>>>>>
> >>>>>>>>> This is mostly a way of saying:
> >>>>>>>>> – I like the cadence/sequencing Benedict proposes below.
> >>>>>>>>> – I think improvements in test engineering can reduce/eliminate
> >>>>>>>>> invalidation and may increase the scope of what can be a
> >>> candidate
> >>>>> for
> >>>>>>>>> merge on a given branch
> >>>>>>>>> – And if not, the cost of missing the train is lower because
> >>> we'll
> >>>>> be
> >>>>>>> able
> >>>>>>>>> to deliver major releases more often.
> >>>>>>>>>
> >>>>>>>>> Scott
> >>>>>>>>>
> >>>>>>>>> ________________________________________
> >>>>>>>>> From: Jeremiah D Jordan <je...@gmail.com>
> >>>>>>>>> Sent: Wednesday, May 27, 2020 11:54 AM
> >>>>>>>>> To: Cassandra DEV
> >>>>>>>>> Subject: Re: [DISCUSS] CASSANDRA-13994
> >>>>>>>>>
> >>>>>>>>> +1 strongly agree. If we aren’t going to let something go into
> >>>>> 4.0.0
> >>>>>>>>> because it would "invalidate testing” then we can not let such
> >>> a
> >>>>> thing
> >>>>>>> go
> >>>>>>>>> into 4.0.1 unless we plan to re-do said testing for the patch
> >>>>> release.
> >>>>>>>>>
> >>>>>>>>>> On May 27, 2020, at 1:31 PM, Benedict Elliott Smith <
> >>>>>>> benedict@apache.org>
> >>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>> I'm being told this still isn't clear, so let me try in a
> >>>>>> bullet-point
> >>>>>>>>> timeline:
> >>>>>>>>>>
> >>>>>>>>>> * 4.0 Beta
> >>>>>>>>>> * 4.0 Verification Work
> >>>>>>>>>> * [Merge Window]
> >>>>>>>>>> * 4.0 GA
> >>>>>>>>>> * 4.0 Minor Releases
> >>>>>>>>>> * ...
> >>>>>>>>>> * 5.0 Dev
> >>>>>>>>>> * ...
> >>>>>>>>>> * 5.0 Verification Work
> >>>>>>>>>> * GA 5.0
> >>>>>>>>>>
> >>>>>>>>>> I think that anything that is prohibited from "[Merge Window]"
> >>>>>> because
> >>>>>>>>> it invalidates "4.0 Verification Work" must also be prohibited
> >>>> until
> >>>>>>> "5.0
> >>>>>>>>> Dev" because the next equivalent work that can now validate it
> >>>>> occurs
> >>>>>>> only
> >>>>>>>>> at "5.0 Verification Work"
> >>>>>>>>>>
> >>>>>>>>>> On 27/05/2020, 19:05, "Benedict Elliott Smith" <
> >>>>> benedict@apache.org
> >>>>>>>
> >>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>> I'm not sure if I communicated my point very well. I mean
> >>> to
> >>>> say
> >>>>>>>>> that if the reason we are prohibiting a patch to land
> >>> post-beta is
> >>>>>>> because
> >>>>>>>>> it invalidates work we only perform pre-ga, then it probably
> >>>> should
> >>>>>> not
> >>>>>>> be
> >>>>>>>>> permitted to land post-ga either, since it must also invalidate
> >>>> the
> >>>>>> same
> >>>>>>>>> work?
> >>>>>>>>>>
> >>>>>>>>>> That is to say, if we're comfortable with work landing
> >>> post-ga
> >>>>>>>>> because we believe it to be safe to release without our
> >>>>>>> pre-major-release
> >>>>>>>>> verification, we should be comfortable with it landing at any
> >>> time
> >>>>>>> pre-ga
> >>>>>>>>> too. Anything else seems inconsistent to me, and we should
> >>>> examine
> >>>>>> what
> >>>>>>>>> assumptions we're making that permit this inconsistency to
> >>> arise.
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> On 27/05/2020, 18:49, "Joshua McKenzie" <
> >>> jmckenzie@apache.org
> >>>>>
> >>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> because it invalidates our pre-release verification, then it
> >>>>> should
> >>>>>>> not
> >>>>>>>>>>> land
> >>>>>>>>>>
> >>>>>>>>>> until we next perform pre-release verification
> >>>>>>>>>>
> >>>>>>>>>> At least for me there's a little softness around our
> >>>>> collective
> >>>>>>>>> alignment
> >>>>>>>>>> on when pre-release verification takes place. If it's
> >>>> between
> >>>>>>>>> alpha-1 and
> >>>>>>>>>> ga we don't want changes that would invalidate those
> >>>> changes
> >>>>> to
> >>>>>>>>> land during
> >>>>>>>>>> that time frame. Different for beta-1 to ga. We also
> >>> risk
> >>>>>>>>> invalidating
> >>>>>>>>>> testing if we do any of that testing before wherever
> >>> that
> >>>>>> cutoff
> >>>>>>>>> is, and a
> >>>>>>>>>> lack of clarity on that cutoff further muddies those
> >>>> waters.
> >>>>>>>>>>
> >>>>>>>>>> My very loosely held perspective is that beta-1 to ga is
> >>>> the
> >>>>>>>>> window in
> >>>>>>>>>> which we apply the "don't do things that will invalidate
> >>>>>>>>> verification", and
> >>>>>>>>>> we plan to do that verification during the beta phase. I
> >>>>>> *think*
> >>>>>>>>> this is
> >>>>>>>>>> consistent w/the current framing of the lifecycle doc.
> >>> That
> >>>>>> being
> >>>>>>>>> said, I
> >>>>>>>>>> don't have strong religion on this so if we collectively
> >>>> want
> >>>>>> to
> >>>>>>>>> call it
> >>>>>>>>>> "don't majorly disrupt from alpha-1 to ga", we can
> >>>> formalize
> >>>>>> that
> >>>>>>>>> in the
> >>>>>>>>>> docs and go ahead and triage current open scope for 4.0
> >>> and
> >>>>>> move
> >>>>>>>>> things out.
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> On Wed, May 27, 2020 at 12:59 PM Ekaterina Dimitrova <
> >>>>>>>>>> ekaterina.dimitrova@datastax.com> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Thank you all for your input.
> >>>>>>>>>>> I think an important topic is again to revise the lifecycle
> >>> and
> >>>>>> ensure
> >>>>>>>>> we
> >>>>>>>>>>> really have the vision on what is left until beta. I will
> >>> start
> >>>> a
> >>>>>>>>> separate
> >>>>>>>>>>> thread on the flaky tests situation soon.
> >>>>>>>>>>>
> >>>>>>>>>>> For this particular ticket I see a couple of things:
> >>>>>>>>>>> - There are a lot of deletions of already not used code
> >>>>>>>>>>> - I implemented it still in alpha as per our agreement that
> >>> this
> >>>>>> will
> >>>>>>>>> give
> >>>>>>>>>>> us enough time for testing. Probably Dinesh as a reviewer can
> >>>> give
> >>>>>>> some
> >>>>>>>>>>> valuable feedback/opinion on the patch.
> >>>>>>>>>>> - It definitely touches around important places but the
> >>>> important
> >>>>>>> thing
> >>>>>>>>> is
> >>>>>>>>>>> to see how exactly it touches, I think
> >>>>>>>>>>> - Considering it for alpha before the major testing in beta
> >>>> sounds
> >>>>>>>>>>> reasonable to me but I guess it also depends on people
> >>>>> availability
> >>>>>> to
> >>>>>>>>>>> review it in detail and the exact test plans afterwards
> >>>>>>>>>>>
> >>>>>>>>>>> On Wed, 27 May 2020 at 7:14, Benedict Elliott Smith <
> >>>>>>>>> benedict@apache.org>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> I think our "pre-beta" criteria should also be our "not in a
> >>>>> major"
> >>>>>>>>>>>> criteria.
> >>>>>>>>>>>>
> >>>>>>>>>>>> If work is prohibited because it invalidates our pre-release
> >>>>>>>>>>> verification,
> >>>>>>>>>>>> then it should not land until we next perform pre-release
> >>>>>>> verification,
> >>>>>>>>>>>> which only currently happens once per major.
> >>>>>>>>>>>>
> >>>>>>>>>>>> This could mean either landing less in a major, or
> >>> permitting
> >>>>> more
> >>>>>> in
> >>>>>>>>>>> beta
> >>>>>>>>>>>> etc.
> >>>>>>>>>>>>
> >>>>>>>>>>>> On 26/05/2020, 19:24, "Joshua McKenzie" <
> >>> jmckenzie@apache.org
> >>>>>
> >>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>> I think an interesting question that informs when to stop
> >>>>>> accepting
> >>>>>>>>>>>> specific changes in a release is when we expect any
> >>> extensive
> >>>>>>>>>>>> pre-release
> >>>>>>>>>>>> testing to take place.
> >>>>>>>>>>>>
> >>>>>>>>>>>> If we go by our release lifecycle, gutting deprecated code
> >>>>> seems
> >>>>>>>>>>>> compatible
> >>>>>>>>>>>> w/Alpha but I wouldn't endorse merging it into Beta:
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>
> >>>>>>
> >>>>
> https://cwiki.apache.org/confluence/display/CASSANDRA/Release+Lifecycle
> >>>>>>> .
> >>>>>>>>>>>> Since almost all of the 40_quality_testing epic stuff is
> >>> also
> >>>>>> beta
> >>>>>>>>>>>> phase
> >>>>>>>>>>>> and hasn't really taken off yet, it also seems like there
> >>>> will
> >>>>> be
> >>>>>>>>>>>> extensive
> >>>>>>>>>>>> testing after this phase transition.
> >>>>>>>>>>>>
> >>>>>>>>>>>> All that being said, I'd advocate for marking FixVer 4.x
> >>> to
> >>>>>>> indicate
> >>>>>>>>>>>> optionality and disallow merge of tickets like this after
> >>>> we're
> >>>>>>> done
> >>>>>>>>>>>> w/alpha phase in keeping w/our lifecycle doc in general.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Does that make sense? Should we consider revisiting and
> >>>>> revising
> >>>>>>> the
> >>>>>>>>>>>> lifecycle doc re: larger deprecation / changes and cycle
> >>>>> stages?
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Tue, May 26, 2020 at 12:53 PM Oleksandr Petrov <
> >>>>>>>>>>>> oleksandr.petrov@gmail.com> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>>> 1) Would you block the release over this ticket?
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> I would definitely not block the release on this ticket.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> 2) Would you prioritize this ticket over testing?
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Same here, I would prioritise testing.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> 3) Does fixing this ticket make 4.0 a more stable release?
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> I wanted to give some context: I wrote that in August 2018.
> >>>>> While
> >>>>>> I
> >>>>>>>>>>>> still
> >>>>>>>>>>>>> believe it is important to get rid of this code, I'm
> >>>> disinclined
> >>>>>> to
> >>>>>>>>>>>> merge
> >>>>>>>>>>>>> it into 4.0.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Given that the patch is rather big (421 additions and 1,480
> >>>>>>>>>>>> deletions) and
> >>>>>>>>>>>>> touches many important places, including parser, I would be
> >>>>>>>>>>> extremely
> >>>>>>>>>>>>> cautious to merge it that late in release cycle. It would
> >>> be
> >>>>> great
> >>>>>>>>>>>> to also
> >>>>>>>>>>>>> hear arguments that would justify the risk.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Thank you for starting this discussion,
> >>>>>>>>>>>>> -- Alex
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Tue, May 26, 2020 at 5:20 PM Ekaterina Dimitrova <
> >>>>>>>>>>>>> ekaterina.dimitrova@datastax.com> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> Dear all,
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Following the ticket review sent on 12th May I wanted to
> >>>> bring
> >>>>> up
> >>>>>>>>>>>>>> https://issues.apache.org/jira/browse/CASSANDRA-13994:
> >>>> Remove
> >>>>>>>>>>>> COMPACT
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> STORAGE internals before 4.0 release.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> It is already under review by Dinesh Joshi and Alex
> >>> Petrov.
> >>>>> Not a
> >>>>>>>>>>>>>> blocker but already under review.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Below are my responses to the questions brought up.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> 1) Would you block the release over this
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> ticket? - probably not
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> 2) Would you prioritize this ticket over testing? -
> >>> already
> >>>>>>>>>>>>>> implemented but if there are some big changes needed after
> >>>> the
> >>>>>>>>>>>> review,
> >>>>>>>>>>>>>> I doubt it we will want to prioritize over the testing
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> 3) Does fixing
> >>>>>>>>>>>>>> this ticket make 4.0 a more stable release? - I will just
> >>>> cite
> >>>>>>>>>>> Alex
> >>>>>>>>>>>>>> Petrov who reported this Jira and I think the rest of us
> >>>> would
> >>>>>>>>>>>> agree
> >>>>>>>>>>>>>> with him here.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> "I would say it's quite important to clean up compact
> >>> storage
> >>>>>>>>>>>>>> internals in 4.0 before the release. It should have no
> >>>> visible
> >>>>>>>>>>>>>> side-effects, but it'd be very good to have as it
> >>> simplifies
> >>>>>>>>>>>> multiple
> >>>>>>>>>>>>>> code paths."
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Ekaterina Dimitrova
> >>>>>>>>>>>>>> e. ekaterina.dimitrova@datastax.com
> >>>>>>>>>>>>>> w. www.datastax.com
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> --
> >>>>>>>>>>>>> alex p
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>
> >>> ---------------------------------------------------------------------
> >>>>>>>>>>>> To unsubscribe, e-mail:
> >>> dev-unsubscribe@cassandra.apache.org
> >>>>>>>>>>>> For additional commands, e-mail:
> >>> dev-help@cassandra.apache.org
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>
> >>> ---------------------------------------------------------------------
> >>>>>>>>>> To unsubscribe, e-mail:
> >>> dev-unsubscribe@cassandra.apache.org
> >>>>>>>>>> For additional commands, e-mail:
> >>> dev-help@cassandra.apache.org
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>> ---------------------------------------------------------------------
> >>>>>>>>>> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> >>>>>>>>>> For additional commands, e-mail:
> >>> dev-help@cassandra.apache.org
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>> ---------------------------------------------------------------------
> >>>>>>>>> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> >>>>>>>>> For additional commands, e-mail: dev-help@cassandra.apache.org
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>> ---------------------------------------------------------------------
> >>>>>>>>> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> >>>>>>>>> For additional commands, e-mail: dev-help@cassandra.apache.org
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>> ---------------------------------------------------------------------
> >>>>>>> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> >>>>>>> For additional commands, e-mail: dev-help@cassandra.apache.org
> >>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>>
> >>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> For additional commands, e-mail: dev-help@cassandra.apache.org
>
>
Re: [DISCUSS] CASSANDRA-13994
Posted by Dinesh Joshi <dj...@apache.org>.
I might be missing some context here but I thought we did not intend to remove compact storage for 4.0 as it was deemed to be too invasive at this point. Did that decision change?
Dinesh
> On Jun 24, 2020, at 10:48 AM, Ekaterina Dimitrova <ek...@datastax.com> wrote:
>
> Dear all,
> As we talked yesterday during our meeting, I am bringing back to the table
> CASSANDRA-13994 <https://issues.apache.org/jira/browse/CASSANDRA-13994>.
> Description
>
> 4.0 comes without thrift (after CASSANDRA-11115
> <https://issues.apache.org/jira/browse/CASSANDRA-11115>) and COMPACT
> STORAGE (after CASSANDRA-10857
> <https://issues.apache.org/jira/browse/CASSANDRA-10857>), and since Compact
> Storage flags are now disabled, all of the related functionality is useless.
>
> There are still some things to consider:
> 1. One of the system tables (built indexes) was compact. For now, we just
> added value column to it to make sure it's backwards-compatible, but we
> might want to make sure it's just a "normal" table and doesn't have
> redundant columns.
> 2. Compact Tables were building indexes in KEYS mode. Removing it is
> trivial, but this would mean that all built indexes will be defunct. We
> could log a warning for now and ask users to migrate off those for now and
> completely remove it from future releases. It's just a couple of classes
> though.
>
>
> Thank you Sylvain for the first pass of review.
>
> An option, as also Sylvain suggested, is to split the ticket in two tickets
> - clean the Compact storage dead code in this one and open a new ticket to
> handle the indexes.
>
> I am pasting here the latest comment, hopefully this makes it easier for
> the audience:
>
> I have made a first pass of review and offered a few remarks above.
>
> But I think this ticket is hanging up on us deciding whether removing the
> KEYS 2ndary index code is ok or not. And this yields, to me, the question
> of what is the upgrade path to 4.0 for users that still have KEYS index
> (which, reminder, could only be created with Thrift, but could *used* with
> CQL and thus still be around).
>
> Because, while I haven't tested this myself, I suspect we have a hole here.
>
> Namely, KEYS index were compact tables, and 4.0 does not start if there is
> still compact tables. And while for user tables, user are asked to use DROP
> COMPACT STORAGE before upgrading, this cannot be done on KEYS index (there
> is just no syntax to do it), so unless there is code I'm not aware of (and
> please, someone correct me if I'm wrong), I don't think user can upgrade to
> 4.0 at all if they still have KEYS index. They'd have to drop those index
> first.
>
> So If I'm right here, this technically mean removing the KEYS index code in
> 4.0 is fine, since you cannot upgrade in the first place if you have KEYS
> index. But the more important question for 4.0 imo is what is the upgrade
> path for users if they have a KEYS index in 3.X?
>
> Currently (without code changes), the only available option I can think of
> is that before upgrade to 4.0, users would have to 1) drop their KEYS index
> and then 2) re-create a "normal" (non-KEYS) equivalent index.
>
> Are we comfortable with that being the upgrade path for KEYS index?
>
> Personally, I'm not sure I am because this is not a seamless upgrade, as
> between the 1) and 2) above, there is a window where there is no accessible
> index, so if the user application rely on it, it means a period of downtime
> for the application to perform the upgrade. However, if we want a more
> seamless upgrade, we need to figure something out, and that probably
> involve non trivial amounts of code and testing. And, playing devil's
> advocate, KEYS index being so old, maybe nobody that plans to upgrade to
> 4.0 have them anymore, and maybe it's not worth bothering?
>
> So I could use others opinions here.
>
> Tl;dr, this ticket raises the point that "Oops, I'm not sure we have though
> through the question of upgrade to 4.0 for KEYS indexes". And tbc, it's not
> directly related to this ticket, only indirectly, but it is still something
> we need to figure out. And I'd say, before 4.0-alpha. But I'm happy to
> create a separate ticket specific to that question if that helps.
>
>
> Ekaterina
>
>>
>>
>> It was moved to alpha following the lifecycle document and the latest
>> discussion on the ML. Sylvain made a review yesterday. To me the biggest
>> question is about the indexing. (Posted questions on the ticket itself,
>> waiting for opinion and agreement) For the rest of the ticket I support
>> Sylvain that it is primarily removal of dead code.
>>
>> My understanding is either it lands in alpha before the big testing in
>> beta or it is delayed for v.5
>>
>> Ekaterina
>>
>> On Wed, 10 Jun 2020 at 12:59, Joshua McKenzie <jm...@apache.org>
>> wrote:
>>
>>> Where did we land on inclusion of this ticket in 4.0? Since it's now
>>> flagged as blocking beta rel, I am now very interested in this question.
>>>
>>> :)
>>>
>>>
>>> On Wed, May 27, 2020 at 7:52 PM Jordan West <jo...@gmail.com> wrote:
>>>
>>>> On Wed, May 27, 2020 at 1:23 PM Joshua McKenzie <jm...@apache.org>
>>>> wrote:
>>>>
>>>>> Maybe. Do we just time box, say we're going to cut an RC and give it 4
>>>>> weeks, if nothing awful surfaces we GA?
>>>>>
>>>>
>>>> I've seen that work well in the past on other projects. I agree with the
>>>> notion that RCs are real candidates for release if no one finds issues
>>> with
>>>> them. Ideally we would have as little RCs as possible and have more
>>>> alphas/betas.
>>>>
>>>>>
>>>>> On Wed, May 27, 2020 at 4:12 PM Brandon Williams <dr...@gmail.com>
>>>> wrote:
>>>>>
>>>>>> Absolutely my understanding.
>>>>>>
>>>>>> On Wed, May 27, 2020, 2:49 PM Jeremiah D Jordan <
>>>>> jeremiah.jordan@gmail.com
>>>>>>>
>>>>>> wrote:
>>>>>>
>>>>>>>> A clear point to cut RC's doesn't surface from the above for me.
>>>>>>> Releasing
>>>>>>>> an RC before broad verification seems wrong, and cutting an RC
>>>> after
>>>>>> the
>>>>>>> 4
>>>>>>>> points above may as well be GA because it's all known scope.
>>>>>>>
>>>>>>> Isn’t the whole point of an RC is that it could be the GA? It is
>>> a
>>>>>>> “release candidate”, meaning if no one finds any issues with it,
>>> that
>>>>> can
>>>>>>> them become the release? So that seems like exactly the right
>>> time
>>>> to
>>>>>> make
>>>>>>> RC releases?
>>>>>>>
>>>>>>>> On May 27, 2020, at 2:45 PM, Joshua McKenzie <
>>> jmckenzie@apache.org
>>>>>
>>>>>>> wrote:
>>>>>>>>
>>>>>>>> I think we're all on the same page here; I was focusing more on
>>> the
>>>>>>> release
>>>>>>>> lifecycles and sequencing than the entire version cycle. Good to
>>>>>> broaden
>>>>>>>> scope I think.
>>>>>>>>
>>>>>>>> One thing we're not considering is the separation of API changes
>>>> from
>>>>>>> major
>>>>>>>> changes and how that intersects with release milestones.
>>>>>>>>
>>>>>>>> Meaning:
>>>>>>>> 1. alpha phase
>>>>>>>> 2. Milestone: API freeze (all API changes pushed to next major)
>>>>>>>> 3. beta phase
>>>>>>>> 4. Verification phase (all major disruptive pushed to next
>>> major)
>>>>>>>>
>>>>>>>> A clear point to cut RC's doesn't surface from the above for me.
>>>>>>> Releasing
>>>>>>>> an RC before broad verification seems wrong, and cutting an RC
>>>> after
>>>>>> the
>>>>>>> 4
>>>>>>>> points above may as well be GA because it's all known scope.
>>>>>>>>
>>>>>>>> Thoughts?
>>>>>>>>
>>>>>>>> On Wed, May 27, 2020 at 3:28 PM Scott Andreas <
>>>> scott@paradoxica.net>
>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> That makes sense to me, yep.
>>>>>>>>>
>>>>>>>>> My hope and expectation is that the time required for
>>>> "verification
>>>>>>> work"
>>>>>>>>> will shrink dramatically in the not too distant future -
>>> ideally
>>>> to
>>>>> a
>>>>>>>>> period of less than a month. In this world, the cost of missing
>>>> one
>>>>>>> train
>>>>>>>>> is reduced to catching the next one.
>>>>>>>>>
>>>>>>>>> One of the main goals in shifting focus from "testing" and
>>> "test
>>>>>> plans"
>>>>>>> to
>>>>>>>>> "test engineering" is automating as many aspects of release
>>>>>>> qualification
>>>>>>>>> as possible, with an asymptotic ideal as a function of compute
>>>>>> capacity
>>>>>>> and
>>>>>>>>> time. While such automation will never be complete (it's likely
>>>> that
>>>>>>>>> development of new features will/must include qualification
>>> infra
>>>>>>> changes
>>>>>>>>> to exercise them), if we're able to apply the same rigor to
>>> major
>>>>>>> releases
>>>>>>>>> as we are to patchlevel builds with little incremental effort,
>>> I'd
>>>>> be
>>>>>>>>> thrilled.
>>>>>>>>>
>>>>>>>>> This is mostly a way of saying:
>>>>>>>>> – I like the cadence/sequencing Benedict proposes below.
>>>>>>>>> – I think improvements in test engineering can reduce/eliminate
>>>>>>>>> invalidation and may increase the scope of what can be a
>>> candidate
>>>>> for
>>>>>>>>> merge on a given branch
>>>>>>>>> – And if not, the cost of missing the train is lower because
>>> we'll
>>>>> be
>>>>>>> able
>>>>>>>>> to deliver major releases more often.
>>>>>>>>>
>>>>>>>>> Scott
>>>>>>>>>
>>>>>>>>> ________________________________________
>>>>>>>>> From: Jeremiah D Jordan <je...@gmail.com>
>>>>>>>>> Sent: Wednesday, May 27, 2020 11:54 AM
>>>>>>>>> To: Cassandra DEV
>>>>>>>>> Subject: Re: [DISCUSS] CASSANDRA-13994
>>>>>>>>>
>>>>>>>>> +1 strongly agree. If we aren’t going to let something go into
>>>>> 4.0.0
>>>>>>>>> because it would "invalidate testing” then we can not let such
>>> a
>>>>> thing
>>>>>>> go
>>>>>>>>> into 4.0.1 unless we plan to re-do said testing for the patch
>>>>> release.
>>>>>>>>>
>>>>>>>>>> On May 27, 2020, at 1:31 PM, Benedict Elliott Smith <
>>>>>>> benedict@apache.org>
>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> I'm being told this still isn't clear, so let me try in a
>>>>>> bullet-point
>>>>>>>>> timeline:
>>>>>>>>>>
>>>>>>>>>> * 4.0 Beta
>>>>>>>>>> * 4.0 Verification Work
>>>>>>>>>> * [Merge Window]
>>>>>>>>>> * 4.0 GA
>>>>>>>>>> * 4.0 Minor Releases
>>>>>>>>>> * ...
>>>>>>>>>> * 5.0 Dev
>>>>>>>>>> * ...
>>>>>>>>>> * 5.0 Verification Work
>>>>>>>>>> * GA 5.0
>>>>>>>>>>
>>>>>>>>>> I think that anything that is prohibited from "[Merge Window]"
>>>>>> because
>>>>>>>>> it invalidates "4.0 Verification Work" must also be prohibited
>>>> until
>>>>>>> "5.0
>>>>>>>>> Dev" because the next equivalent work that can now validate it
>>>>> occurs
>>>>>>> only
>>>>>>>>> at "5.0 Verification Work"
>>>>>>>>>>
>>>>>>>>>> On 27/05/2020, 19:05, "Benedict Elliott Smith" <
>>>>> benedict@apache.org
>>>>>>>
>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> I'm not sure if I communicated my point very well. I mean
>>> to
>>>> say
>>>>>>>>> that if the reason we are prohibiting a patch to land
>>> post-beta is
>>>>>>> because
>>>>>>>>> it invalidates work we only perform pre-ga, then it probably
>>>> should
>>>>>> not
>>>>>>> be
>>>>>>>>> permitted to land post-ga either, since it must also invalidate
>>>> the
>>>>>> same
>>>>>>>>> work?
>>>>>>>>>>
>>>>>>>>>> That is to say, if we're comfortable with work landing
>>> post-ga
>>>>>>>>> because we believe it to be safe to release without our
>>>>>>> pre-major-release
>>>>>>>>> verification, we should be comfortable with it landing at any
>>> time
>>>>>>> pre-ga
>>>>>>>>> too. Anything else seems inconsistent to me, and we should
>>>> examine
>>>>>> what
>>>>>>>>> assumptions we're making that permit this inconsistency to
>>> arise.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On 27/05/2020, 18:49, "Joshua McKenzie" <
>>> jmckenzie@apache.org
>>>>>
>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> because it invalidates our pre-release verification, then it
>>>>> should
>>>>>>> not
>>>>>>>>>>> land
>>>>>>>>>>
>>>>>>>>>> until we next perform pre-release verification
>>>>>>>>>>
>>>>>>>>>> At least for me there's a little softness around our
>>>>> collective
>>>>>>>>> alignment
>>>>>>>>>> on when pre-release verification takes place. If it's
>>>> between
>>>>>>>>> alpha-1 and
>>>>>>>>>> ga we don't want changes that would invalidate those
>>>> changes
>>>>> to
>>>>>>>>> land during
>>>>>>>>>> that time frame. Different for beta-1 to ga. We also
>>> risk
>>>>>>>>> invalidating
>>>>>>>>>> testing if we do any of that testing before wherever
>>> that
>>>>>> cutoff
>>>>>>>>> is, and a
>>>>>>>>>> lack of clarity on that cutoff further muddies those
>>>> waters.
>>>>>>>>>>
>>>>>>>>>> My very loosely held perspective is that beta-1 to ga is
>>>> the
>>>>>>>>> window in
>>>>>>>>>> which we apply the "don't do things that will invalidate
>>>>>>>>> verification", and
>>>>>>>>>> we plan to do that verification during the beta phase. I
>>>>>> *think*
>>>>>>>>> this is
>>>>>>>>>> consistent w/the current framing of the lifecycle doc.
>>> That
>>>>>> being
>>>>>>>>> said, I
>>>>>>>>>> don't have strong religion on this so if we collectively
>>>> want
>>>>>> to
>>>>>>>>> call it
>>>>>>>>>> "don't majorly disrupt from alpha-1 to ga", we can
>>>> formalize
>>>>>> that
>>>>>>>>> in the
>>>>>>>>>> docs and go ahead and triage current open scope for 4.0
>>> and
>>>>>> move
>>>>>>>>> things out.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Wed, May 27, 2020 at 12:59 PM Ekaterina Dimitrova <
>>>>>>>>>> ekaterina.dimitrova@datastax.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Thank you all for your input.
>>>>>>>>>>> I think an important topic is again to revise the lifecycle
>>> and
>>>>>> ensure
>>>>>>>>> we
>>>>>>>>>>> really have the vision on what is left until beta. I will
>>> start
>>>> a
>>>>>>>>> separate
>>>>>>>>>>> thread on the flaky tests situation soon.
>>>>>>>>>>>
>>>>>>>>>>> For this particular ticket I see a couple of things:
>>>>>>>>>>> - There are a lot of deletions of already not used code
>>>>>>>>>>> - I implemented it still in alpha as per our agreement that
>>> this
>>>>>> will
>>>>>>>>> give
>>>>>>>>>>> us enough time for testing. Probably Dinesh as a reviewer can
>>>> give
>>>>>>> some
>>>>>>>>>>> valuable feedback/opinion on the patch.
>>>>>>>>>>> - It definitely touches around important places but the
>>>> important
>>>>>>> thing
>>>>>>>>> is
>>>>>>>>>>> to see how exactly it touches, I think
>>>>>>>>>>> - Considering it for alpha before the major testing in beta
>>>> sounds
>>>>>>>>>>> reasonable to me but I guess it also depends on people
>>>>> availability
>>>>>> to
>>>>>>>>>>> review it in detail and the exact test plans afterwards
>>>>>>>>>>>
>>>>>>>>>>> On Wed, 27 May 2020 at 7:14, Benedict Elliott Smith <
>>>>>>>>> benedict@apache.org>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> I think our "pre-beta" criteria should also be our "not in a
>>>>> major"
>>>>>>>>>>>> criteria.
>>>>>>>>>>>>
>>>>>>>>>>>> If work is prohibited because it invalidates our pre-release
>>>>>>>>>>> verification,
>>>>>>>>>>>> then it should not land until we next perform pre-release
>>>>>>> verification,
>>>>>>>>>>>> which only currently happens once per major.
>>>>>>>>>>>>
>>>>>>>>>>>> This could mean either landing less in a major, or
>>> permitting
>>>>> more
>>>>>> in
>>>>>>>>>>> beta
>>>>>>>>>>>> etc.
>>>>>>>>>>>>
>>>>>>>>>>>> On 26/05/2020, 19:24, "Joshua McKenzie" <
>>> jmckenzie@apache.org
>>>>>
>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> I think an interesting question that informs when to stop
>>>>>> accepting
>>>>>>>>>>>> specific changes in a release is when we expect any
>>> extensive
>>>>>>>>>>>> pre-release
>>>>>>>>>>>> testing to take place.
>>>>>>>>>>>>
>>>>>>>>>>>> If we go by our release lifecycle, gutting deprecated code
>>>>> seems
>>>>>>>>>>>> compatible
>>>>>>>>>>>> w/Alpha but I wouldn't endorse merging it into Beta:
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>
>>>>>>
>>>> https://cwiki.apache.org/confluence/display/CASSANDRA/Release+Lifecycle
>>>>>>> .
>>>>>>>>>>>> Since almost all of the 40_quality_testing epic stuff is
>>> also
>>>>>> beta
>>>>>>>>>>>> phase
>>>>>>>>>>>> and hasn't really taken off yet, it also seems like there
>>>> will
>>>>> be
>>>>>>>>>>>> extensive
>>>>>>>>>>>> testing after this phase transition.
>>>>>>>>>>>>
>>>>>>>>>>>> All that being said, I'd advocate for marking FixVer 4.x
>>> to
>>>>>>> indicate
>>>>>>>>>>>> optionality and disallow merge of tickets like this after
>>>> we're
>>>>>>> done
>>>>>>>>>>>> w/alpha phase in keeping w/our lifecycle doc in general.
>>>>>>>>>>>>
>>>>>>>>>>>> Does that make sense? Should we consider revisiting and
>>>>> revising
>>>>>>> the
>>>>>>>>>>>> lifecycle doc re: larger deprecation / changes and cycle
>>>>> stages?
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Tue, May 26, 2020 at 12:53 PM Oleksandr Petrov <
>>>>>>>>>>>> oleksandr.petrov@gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>>> 1) Would you block the release over this ticket?
>>>>>>>>>>>>>
>>>>>>>>>>>>> I would definitely not block the release on this ticket.
>>>>>>>>>>>>>
>>>>>>>>>>>>>> 2) Would you prioritize this ticket over testing?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Same here, I would prioritise testing.
>>>>>>>>>>>>>
>>>>>>>>>>>>>> 3) Does fixing this ticket make 4.0 a more stable release?
>>>>>>>>>>>>>
>>>>>>>>>>>>> I wanted to give some context: I wrote that in August 2018.
>>>>> While
>>>>>> I
>>>>>>>>>>>> still
>>>>>>>>>>>>> believe it is important to get rid of this code, I'm
>>>> disinclined
>>>>>> to
>>>>>>>>>>>> merge
>>>>>>>>>>>>> it into 4.0.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Given that the patch is rather big (421 additions and 1,480
>>>>>>>>>>>> deletions) and
>>>>>>>>>>>>> touches many important places, including parser, I would be
>>>>>>>>>>> extremely
>>>>>>>>>>>>> cautious to merge it that late in release cycle. It would
>>> be
>>>>> great
>>>>>>>>>>>> to also
>>>>>>>>>>>>> hear arguments that would justify the risk.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thank you for starting this discussion,
>>>>>>>>>>>>> -- Alex
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Tue, May 26, 2020 at 5:20 PM Ekaterina Dimitrova <
>>>>>>>>>>>>> ekaterina.dimitrova@datastax.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Dear all,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Following the ticket review sent on 12th May I wanted to
>>>> bring
>>>>> up
>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/CASSANDRA-13994:
>>>> Remove
>>>>>>>>>>>> COMPACT
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> STORAGE internals before 4.0 release.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> It is already under review by Dinesh Joshi and Alex
>>> Petrov.
>>>>> Not a
>>>>>>>>>>>>>> blocker but already under review.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Below are my responses to the questions brought up.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 1) Would you block the release over this
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> ticket? - probably not
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 2) Would you prioritize this ticket over testing? -
>>> already
>>>>>>>>>>>>>> implemented but if there are some big changes needed after
>>>> the
>>>>>>>>>>>> review,
>>>>>>>>>>>>>> I doubt it we will want to prioritize over the testing
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 3) Does fixing
>>>>>>>>>>>>>> this ticket make 4.0 a more stable release? - I will just
>>>> cite
>>>>>>>>>>> Alex
>>>>>>>>>>>>>> Petrov who reported this Jira and I think the rest of us
>>>> would
>>>>>>>>>>>> agree
>>>>>>>>>>>>>> with him here.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> "I would say it's quite important to clean up compact
>>> storage
>>>>>>>>>>>>>> internals in 4.0 before the release. It should have no
>>>> visible
>>>>>>>>>>>>>> side-effects, but it'd be very good to have as it
>>> simplifies
>>>>>>>>>>>> multiple
>>>>>>>>>>>>>> code paths."
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Ekaterina Dimitrova
>>>>>>>>>>>>>> e. ekaterina.dimitrova@datastax.com
>>>>>>>>>>>>>> w. www.datastax.com
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> alex p
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>
>>> ---------------------------------------------------------------------
>>>>>>>>>>>> To unsubscribe, e-mail:
>>> dev-unsubscribe@cassandra.apache.org
>>>>>>>>>>>> For additional commands, e-mail:
>>> dev-help@cassandra.apache.org
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>
>>> ---------------------------------------------------------------------
>>>>>>>>>> To unsubscribe, e-mail:
>>> dev-unsubscribe@cassandra.apache.org
>>>>>>>>>> For additional commands, e-mail:
>>> dev-help@cassandra.apache.org
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>> ---------------------------------------------------------------------
>>>>>>>>>> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
>>>>>>>>>> For additional commands, e-mail:
>>> dev-help@cassandra.apache.org
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>> ---------------------------------------------------------------------
>>>>>>>>> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
>>>>>>>>> For additional commands, e-mail: dev-help@cassandra.apache.org
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>> ---------------------------------------------------------------------
>>>>>>>>> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
>>>>>>>>> For additional commands, e-mail: dev-help@cassandra.apache.org
>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>> ---------------------------------------------------------------------
>>>>>>> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
>>>>>>> For additional commands, e-mail: dev-help@cassandra.apache.org
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
For additional commands, e-mail: dev-help@cassandra.apache.org
Re: [DISCUSS] CASSANDRA-13994
Posted by Ekaterina Dimitrova <ek...@datastax.com>.
Dear all,
As we talked yesterday during our meeting, I am bringing back to the table
CASSANDRA-13994 <https://issues.apache.org/jira/browse/CASSANDRA-13994>.
Description
4.0 comes without thrift (after CASSANDRA-11115
<https://issues.apache.org/jira/browse/CASSANDRA-11115>) and COMPACT
STORAGE (after CASSANDRA-10857
<https://issues.apache.org/jira/browse/CASSANDRA-10857>), and since Compact
Storage flags are now disabled, all of the related functionality is useless.
There are still some things to consider:
1. One of the system tables (built indexes) was compact. For now, we just
added value column to it to make sure it's backwards-compatible, but we
might want to make sure it's just a "normal" table and doesn't have
redundant columns.
2. Compact Tables were building indexes in KEYS mode. Removing it is
trivial, but this would mean that all built indexes will be defunct. We
could log a warning for now and ask users to migrate off those for now and
completely remove it from future releases. It's just a couple of classes
though.
Thank you Sylvain for the first pass of review.
An option, as also Sylvain suggested, is to split the ticket in two tickets
- clean the Compact storage dead code in this one and open a new ticket to
handle the indexes.
I am pasting here the latest comment, hopefully this makes it easier for
the audience:
I have made a first pass of review and offered a few remarks above.
But I think this ticket is hanging up on us deciding whether removing the
KEYS 2ndary index code is ok or not. And this yields, to me, the question
of what is the upgrade path to 4.0 for users that still have KEYS index
(which, reminder, could only be created with Thrift, but could *used* with
CQL and thus still be around).
Because, while I haven't tested this myself, I suspect we have a hole here.
Namely, KEYS index were compact tables, and 4.0 does not start if there is
still compact tables. And while for user tables, user are asked to use DROP
COMPACT STORAGE before upgrading, this cannot be done on KEYS index (there
is just no syntax to do it), so unless there is code I'm not aware of (and
please, someone correct me if I'm wrong), I don't think user can upgrade to
4.0 at all if they still have KEYS index. They'd have to drop those index
first.
So If I'm right here, this technically mean removing the KEYS index code in
4.0 is fine, since you cannot upgrade in the first place if you have KEYS
index. But the more important question for 4.0 imo is what is the upgrade
path for users if they have a KEYS index in 3.X?
Currently (without code changes), the only available option I can think of
is that before upgrade to 4.0, users would have to 1) drop their KEYS index
and then 2) re-create a "normal" (non-KEYS) equivalent index.
Are we comfortable with that being the upgrade path for KEYS index?
Personally, I'm not sure I am because this is not a seamless upgrade, as
between the 1) and 2) above, there is a window where there is no accessible
index, so if the user application rely on it, it means a period of downtime
for the application to perform the upgrade. However, if we want a more
seamless upgrade, we need to figure something out, and that probably
involve non trivial amounts of code and testing. And, playing devil's
advocate, KEYS index being so old, maybe nobody that plans to upgrade to
4.0 have them anymore, and maybe it's not worth bothering?
So I could use others opinions here.
Tl;dr, this ticket raises the point that "Oops, I'm not sure we have though
through the question of upgrade to 4.0 for KEYS indexes". And tbc, it's not
directly related to this ticket, only indirectly, but it is still something
we need to figure out. And I'd say, before 4.0-alpha. But I'm happy to
create a separate ticket specific to that question if that helps.
Ekaterina
>
>
> It was moved to alpha following the lifecycle document and the latest
> discussion on the ML. Sylvain made a review yesterday. To me the biggest
> question is about the indexing. (Posted questions on the ticket itself,
> waiting for opinion and agreement) For the rest of the ticket I support
> Sylvain that it is primarily removal of dead code.
>
> My understanding is either it lands in alpha before the big testing in
> beta or it is delayed for v.5
>
> Ekaterina
>
> On Wed, 10 Jun 2020 at 12:59, Joshua McKenzie <jm...@apache.org>
> wrote:
>
>> Where did we land on inclusion of this ticket in 4.0? Since it's now
>> flagged as blocking beta rel, I am now very interested in this question.
>>
>> :)
>>
>>
>> On Wed, May 27, 2020 at 7:52 PM Jordan West <jo...@gmail.com> wrote:
>>
>> > On Wed, May 27, 2020 at 1:23 PM Joshua McKenzie <jm...@apache.org>
>> > wrote:
>> >
>> > > Maybe. Do we just time box, say we're going to cut an RC and give it 4
>> > > weeks, if nothing awful surfaces we GA?
>> > >
>> >
>> > I've seen that work well in the past on other projects. I agree with the
>> > notion that RCs are real candidates for release if no one finds issues
>> with
>> > them. Ideally we would have as little RCs as possible and have more
>> > alphas/betas.
>> >
>> > >
>> > > On Wed, May 27, 2020 at 4:12 PM Brandon Williams <dr...@gmail.com>
>> > wrote:
>> > >
>> > > > Absolutely my understanding.
>> > > >
>> > > > On Wed, May 27, 2020, 2:49 PM Jeremiah D Jordan <
>> > > jeremiah.jordan@gmail.com
>> > > > >
>> > > > wrote:
>> > > >
>> > > > > > A clear point to cut RC's doesn't surface from the above for me.
>> > > > > Releasing
>> > > > > > an RC before broad verification seems wrong, and cutting an RC
>> > after
>> > > > the
>> > > > > 4
>> > > > > > points above may as well be GA because it's all known scope.
>> > > > >
>> > > > > Isn’t the whole point of an RC is that it could be the GA? It is
>> a
>> > > > > “release candidate”, meaning if no one finds any issues with it,
>> that
>> > > can
>> > > > > them become the release? So that seems like exactly the right
>> time
>> > to
>> > > > make
>> > > > > RC releases?
>> > > > >
>> > > > > > On May 27, 2020, at 2:45 PM, Joshua McKenzie <
>> jmckenzie@apache.org
>> > >
>> > > > > wrote:
>> > > > > >
>> > > > > > I think we're all on the same page here; I was focusing more on
>> the
>> > > > > release
>> > > > > > lifecycles and sequencing than the entire version cycle. Good to
>> > > > broaden
>> > > > > > scope I think.
>> > > > > >
>> > > > > > One thing we're not considering is the separation of API changes
>> > from
>> > > > > major
>> > > > > > changes and how that intersects with release milestones.
>> > > > > >
>> > > > > > Meaning:
>> > > > > > 1. alpha phase
>> > > > > > 2. Milestone: API freeze (all API changes pushed to next major)
>> > > > > > 3. beta phase
>> > > > > > 4. Verification phase (all major disruptive pushed to next
>> major)
>> > > > > >
>> > > > > > A clear point to cut RC's doesn't surface from the above for me.
>> > > > > Releasing
>> > > > > > an RC before broad verification seems wrong, and cutting an RC
>> > after
>> > > > the
>> > > > > 4
>> > > > > > points above may as well be GA because it's all known scope.
>> > > > > >
>> > > > > > Thoughts?
>> > > > > >
>> > > > > > On Wed, May 27, 2020 at 3:28 PM Scott Andreas <
>> > scott@paradoxica.net>
>> > > > > wrote:
>> > > > > >
>> > > > > >> That makes sense to me, yep.
>> > > > > >>
>> > > > > >> My hope and expectation is that the time required for
>> > "verification
>> > > > > work"
>> > > > > >> will shrink dramatically in the not too distant future -
>> ideally
>> > to
>> > > a
>> > > > > >> period of less than a month. In this world, the cost of missing
>> > one
>> > > > > train
>> > > > > >> is reduced to catching the next one.
>> > > > > >>
>> > > > > >> One of the main goals in shifting focus from "testing" and
>> "test
>> > > > plans"
>> > > > > to
>> > > > > >> "test engineering" is automating as many aspects of release
>> > > > > qualification
>> > > > > >> as possible, with an asymptotic ideal as a function of compute
>> > > > capacity
>> > > > > and
>> > > > > >> time. While such automation will never be complete (it's likely
>> > that
>> > > > > >> development of new features will/must include qualification
>> infra
>> > > > > changes
>> > > > > >> to exercise them), if we're able to apply the same rigor to
>> major
>> > > > > releases
>> > > > > >> as we are to patchlevel builds with little incremental effort,
>> I'd
>> > > be
>> > > > > >> thrilled.
>> > > > > >>
>> > > > > >> This is mostly a way of saying:
>> > > > > >> – I like the cadence/sequencing Benedict proposes below.
>> > > > > >> – I think improvements in test engineering can reduce/eliminate
>> > > > > >> invalidation and may increase the scope of what can be a
>> candidate
>> > > for
>> > > > > >> merge on a given branch
>> > > > > >> – And if not, the cost of missing the train is lower because
>> we'll
>> > > be
>> > > > > able
>> > > > > >> to deliver major releases more often.
>> > > > > >>
>> > > > > >> Scott
>> > > > > >>
>> > > > > >> ________________________________________
>> > > > > >> From: Jeremiah D Jordan <je...@gmail.com>
>> > > > > >> Sent: Wednesday, May 27, 2020 11:54 AM
>> > > > > >> To: Cassandra DEV
>> > > > > >> Subject: Re: [DISCUSS] CASSANDRA-13994
>> > > > > >>
>> > > > > >> +1 strongly agree. If we aren’t going to let something go into
>> > > 4.0.0
>> > > > > >> because it would "invalidate testing” then we can not let such
>> a
>> > > thing
>> > > > > go
>> > > > > >> into 4.0.1 unless we plan to re-do said testing for the patch
>> > > release.
>> > > > > >>
>> > > > > >>> On May 27, 2020, at 1:31 PM, Benedict Elliott Smith <
>> > > > > benedict@apache.org>
>> > > > > >> wrote:
>> > > > > >>>
>> > > > > >>> I'm being told this still isn't clear, so let me try in a
>> > > > bullet-point
>> > > > > >> timeline:
>> > > > > >>>
>> > > > > >>> * 4.0 Beta
>> > > > > >>> * 4.0 Verification Work
>> > > > > >>> * [Merge Window]
>> > > > > >>> * 4.0 GA
>> > > > > >>> * 4.0 Minor Releases
>> > > > > >>> * ...
>> > > > > >>> * 5.0 Dev
>> > > > > >>> * ...
>> > > > > >>> * 5.0 Verification Work
>> > > > > >>> * GA 5.0
>> > > > > >>>
>> > > > > >>> I think that anything that is prohibited from "[Merge Window]"
>> > > > because
>> > > > > >> it invalidates "4.0 Verification Work" must also be prohibited
>> > until
>> > > > > "5.0
>> > > > > >> Dev" because the next equivalent work that can now validate it
>> > > occurs
>> > > > > only
>> > > > > >> at "5.0 Verification Work"
>> > > > > >>>
>> > > > > >>> On 27/05/2020, 19:05, "Benedict Elliott Smith" <
>> > > benedict@apache.org
>> > > > >
>> > > > > >> wrote:
>> > > > > >>>
>> > > > > >>> I'm not sure if I communicated my point very well. I mean
>> to
>> > say
>> > > > > >> that if the reason we are prohibiting a patch to land
>> post-beta is
>> > > > > because
>> > > > > >> it invalidates work we only perform pre-ga, then it probably
>> > should
>> > > > not
>> > > > > be
>> > > > > >> permitted to land post-ga either, since it must also invalidate
>> > the
>> > > > same
>> > > > > >> work?
>> > > > > >>>
>> > > > > >>> That is to say, if we're comfortable with work landing
>> post-ga
>> > > > > >> because we believe it to be safe to release without our
>> > > > > pre-major-release
>> > > > > >> verification, we should be comfortable with it landing at any
>> time
>> > > > > pre-ga
>> > > > > >> too. Anything else seems inconsistent to me, and we should
>> > examine
>> > > > what
>> > > > > >> assumptions we're making that permit this inconsistency to
>> arise.
>> > > > > >>>
>> > > > > >>>
>> > > > > >>> On 27/05/2020, 18:49, "Joshua McKenzie" <
>> jmckenzie@apache.org
>> > >
>> > > > > >> wrote:
>> > > > > >>>
>> > > > > >>>>
>> > > > > >>>> because it invalidates our pre-release verification, then it
>> > > should
>> > > > > not
>> > > > > >>>> land
>> > > > > >>>
>> > > > > >>> until we next perform pre-release verification
>> > > > > >>>
>> > > > > >>> At least for me there's a little softness around our
>> > > collective
>> > > > > >> alignment
>> > > > > >>> on when pre-release verification takes place. If it's
>> > between
>> > > > > >> alpha-1 and
>> > > > > >>> ga we don't want changes that would invalidate those
>> > changes
>> > > to
>> > > > > >> land during
>> > > > > >>> that time frame. Different for beta-1 to ga. We also
>> risk
>> > > > > >> invalidating
>> > > > > >>> testing if we do any of that testing before wherever
>> that
>> > > > cutoff
>> > > > > >> is, and a
>> > > > > >>> lack of clarity on that cutoff further muddies those
>> > waters.
>> > > > > >>>
>> > > > > >>> My very loosely held perspective is that beta-1 to ga is
>> > the
>> > > > > >> window in
>> > > > > >>> which we apply the "don't do things that will invalidate
>> > > > > >> verification", and
>> > > > > >>> we plan to do that verification during the beta phase. I
>> > > > *think*
>> > > > > >> this is
>> > > > > >>> consistent w/the current framing of the lifecycle doc.
>> That
>> > > > being
>> > > > > >> said, I
>> > > > > >>> don't have strong religion on this so if we collectively
>> > want
>> > > > to
>> > > > > >> call it
>> > > > > >>> "don't majorly disrupt from alpha-1 to ga", we can
>> > formalize
>> > > > that
>> > > > > >> in the
>> > > > > >>> docs and go ahead and triage current open scope for 4.0
>> and
>> > > > move
>> > > > > >> things out.
>> > > > > >>>
>> > > > > >>>
>> > > > > >>>
>> > > > > >>> On Wed, May 27, 2020 at 12:59 PM Ekaterina Dimitrova <
>> > > > > >>> ekaterina.dimitrova@datastax.com> wrote:
>> > > > > >>>
>> > > > > >>>> Thank you all for your input.
>> > > > > >>>> I think an important topic is again to revise the lifecycle
>> and
>> > > > ensure
>> > > > > >> we
>> > > > > >>>> really have the vision on what is left until beta. I will
>> start
>> > a
>> > > > > >> separate
>> > > > > >>>> thread on the flaky tests situation soon.
>> > > > > >>>>
>> > > > > >>>> For this particular ticket I see a couple of things:
>> > > > > >>>> - There are a lot of deletions of already not used code
>> > > > > >>>> - I implemented it still in alpha as per our agreement that
>> this
>> > > > will
>> > > > > >> give
>> > > > > >>>> us enough time for testing. Probably Dinesh as a reviewer can
>> > give
>> > > > > some
>> > > > > >>>> valuable feedback/opinion on the patch.
>> > > > > >>>> - It definitely touches around important places but the
>> > important
>> > > > > thing
>> > > > > >> is
>> > > > > >>>> to see how exactly it touches, I think
>> > > > > >>>> - Considering it for alpha before the major testing in beta
>> > sounds
>> > > > > >>>> reasonable to me but I guess it also depends on people
>> > > availability
>> > > > to
>> > > > > >>>> review it in detail and the exact test plans afterwards
>> > > > > >>>>
>> > > > > >>>> On Wed, 27 May 2020 at 7:14, Benedict Elliott Smith <
>> > > > > >> benedict@apache.org>
>> > > > > >>>> wrote:
>> > > > > >>>>
>> > > > > >>>>> I think our "pre-beta" criteria should also be our "not in a
>> > > major"
>> > > > > >>>>> criteria.
>> > > > > >>>>>
>> > > > > >>>>> If work is prohibited because it invalidates our pre-release
>> > > > > >>>> verification,
>> > > > > >>>>> then it should not land until we next perform pre-release
>> > > > > verification,
>> > > > > >>>>> which only currently happens once per major.
>> > > > > >>>>>
>> > > > > >>>>> This could mean either landing less in a major, or
>> permitting
>> > > more
>> > > > in
>> > > > > >>>> beta
>> > > > > >>>>> etc.
>> > > > > >>>>>
>> > > > > >>>>> On 26/05/2020, 19:24, "Joshua McKenzie" <
>> jmckenzie@apache.org
>> > >
>> > > > > wrote:
>> > > > > >>>>>
>> > > > > >>>>> I think an interesting question that informs when to stop
>> > > > accepting
>> > > > > >>>>> specific changes in a release is when we expect any
>> extensive
>> > > > > >>>>> pre-release
>> > > > > >>>>> testing to take place.
>> > > > > >>>>>
>> > > > > >>>>> If we go by our release lifecycle, gutting deprecated code
>> > > seems
>> > > > > >>>>> compatible
>> > > > > >>>>> w/Alpha but I wouldn't endorse merging it into Beta:
>> > > > > >>>>>
>> > > > > >>>>>
>> > > > > >>
>> > > >
>> > https://cwiki.apache.org/confluence/display/CASSANDRA/Release+Lifecycle
>> > > > > .
>> > > > > >>>>> Since almost all of the 40_quality_testing epic stuff is
>> also
>> > > > beta
>> > > > > >>>>> phase
>> > > > > >>>>> and hasn't really taken off yet, it also seems like there
>> > will
>> > > be
>> > > > > >>>>> extensive
>> > > > > >>>>> testing after this phase transition.
>> > > > > >>>>>
>> > > > > >>>>> All that being said, I'd advocate for marking FixVer 4.x
>> to
>> > > > > indicate
>> > > > > >>>>> optionality and disallow merge of tickets like this after
>> > we're
>> > > > > done
>> > > > > >>>>> w/alpha phase in keeping w/our lifecycle doc in general.
>> > > > > >>>>>
>> > > > > >>>>> Does that make sense? Should we consider revisiting and
>> > > revising
>> > > > > the
>> > > > > >>>>> lifecycle doc re: larger deprecation / changes and cycle
>> > > stages?
>> > > > > >>>>>
>> > > > > >>>>>
>> > > > > >>>>>
>> > > > > >>>>> On Tue, May 26, 2020 at 12:53 PM Oleksandr Petrov <
>> > > > > >>>>> oleksandr.petrov@gmail.com> wrote:
>> > > > > >>>>>
>> > > > > >>>>>>> 1) Would you block the release over this ticket?
>> > > > > >>>>>>
>> > > > > >>>>>> I would definitely not block the release on this ticket.
>> > > > > >>>>>>
>> > > > > >>>>>>> 2) Would you prioritize this ticket over testing?
>> > > > > >>>>>>
>> > > > > >>>>>> Same here, I would prioritise testing.
>> > > > > >>>>>>
>> > > > > >>>>>>> 3) Does fixing this ticket make 4.0 a more stable release?
>> > > > > >>>>>>
>> > > > > >>>>>> I wanted to give some context: I wrote that in August 2018.
>> > > While
>> > > > I
>> > > > > >>>>> still
>> > > > > >>>>>> believe it is important to get rid of this code, I'm
>> > disinclined
>> > > > to
>> > > > > >>>>> merge
>> > > > > >>>>>> it into 4.0.
>> > > > > >>>>>>
>> > > > > >>>>>> Given that the patch is rather big (421 additions and 1,480
>> > > > > >>>>> deletions) and
>> > > > > >>>>>> touches many important places, including parser, I would be
>> > > > > >>>> extremely
>> > > > > >>>>>> cautious to merge it that late in release cycle. It would
>> be
>> > > great
>> > > > > >>>>> to also
>> > > > > >>>>>> hear arguments that would justify the risk.
>> > > > > >>>>>>
>> > > > > >>>>>> Thank you for starting this discussion,
>> > > > > >>>>>> -- Alex
>> > > > > >>>>>>
>> > > > > >>>>>>
>> > > > > >>>>>>
>> > > > > >>>>>> On Tue, May 26, 2020 at 5:20 PM Ekaterina Dimitrova <
>> > > > > >>>>>> ekaterina.dimitrova@datastax.com> wrote:
>> > > > > >>>>>>
>> > > > > >>>>>>> Dear all,
>> > > > > >>>>>>>
>> > > > > >>>>>>> Following the ticket review sent on 12th May I wanted to
>> > bring
>> > > up
>> > > > > >>>>>>> https://issues.apache.org/jira/browse/CASSANDRA-13994:
>> > Remove
>> > > > > >>>>> COMPACT
>> > > > > >>>>>>>
>> > > > > >>>>>>> STORAGE internals before 4.0 release.
>> > > > > >>>>>>>
>> > > > > >>>>>>> It is already under review by Dinesh Joshi and Alex
>> Petrov.
>> > > Not a
>> > > > > >>>>>>> blocker but already under review.
>> > > > > >>>>>>>
>> > > > > >>>>>>> Below are my responses to the questions brought up.
>> > > > > >>>>>>>
>> > > > > >>>>>>>
>> > > > > >>>>>>> 1) Would you block the release over this
>> > > > > >>>>>>>
>> > > > > >>>>>>> ticket? - probably not
>> > > > > >>>>>>>
>> > > > > >>>>>>> 2) Would you prioritize this ticket over testing? -
>> already
>> > > > > >>>>>>> implemented but if there are some big changes needed after
>> > the
>> > > > > >>>>> review,
>> > > > > >>>>>>> I doubt it we will want to prioritize over the testing
>> > > > > >>>>>>>
>> > > > > >>>>>>> 3) Does fixing
>> > > > > >>>>>>> this ticket make 4.0 a more stable release? - I will just
>> > cite
>> > > > > >>>> Alex
>> > > > > >>>>>>> Petrov who reported this Jira and I think the rest of us
>> > would
>> > > > > >>>>> agree
>> > > > > >>>>>>> with him here.
>> > > > > >>>>>>>
>> > > > > >>>>>>> "I would say it's quite important to clean up compact
>> storage
>> > > > > >>>>>>> internals in 4.0 before the release. It should have no
>> > visible
>> > > > > >>>>>>> side-effects, but it'd be very good to have as it
>> simplifies
>> > > > > >>>>> multiple
>> > > > > >>>>>>> code paths."
>> > > > > >>>>>>>
>> > > > > >>>>>>>
>> > > > > >>>>>>> Ekaterina Dimitrova
>> > > > > >>>>>>> e. ekaterina.dimitrova@datastax.com
>> > > > > >>>>>>> w. www.datastax.com
>> > > > > >>>>>>>
>> > > > > >>>>>>
>> > > > > >>>>>>
>> > > > > >>>>>> --
>> > > > > >>>>>> alex p
>> > > > > >>>>>>
>> > > > > >>>>>
>> > > > > >>>>>
>> > > > > >>>>>
>> > > > > >>>>>
>> > > >
>> ---------------------------------------------------------------------
>> > > > > >>>>> To unsubscribe, e-mail:
>> dev-unsubscribe@cassandra.apache.org
>> > > > > >>>>> For additional commands, e-mail:
>> dev-help@cassandra.apache.org
>> > > > > >>>>>
>> > > > > >>>>>
>> > > > > >>>>
>> > > > > >>>
>> > > > > >>>
>> > > > > >>>
>> > > > > >>>
>> > > >
>> ---------------------------------------------------------------------
>> > > > > >>> To unsubscribe, e-mail:
>> dev-unsubscribe@cassandra.apache.org
>> > > > > >>> For additional commands, e-mail:
>> dev-help@cassandra.apache.org
>> > > > > >>>
>> > > > > >>>
>> > > > > >>>
>> > > > > >>>
>> > > > > >>>
>> > > ---------------------------------------------------------------------
>> > > > > >>> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
>> > > > > >>> For additional commands, e-mail:
>> dev-help@cassandra.apache.org
>> > > > > >>>
>> > > > > >>
>> > > > > >>
>> > > > > >>
>> > > ---------------------------------------------------------------------
>> > > > > >> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
>> > > > > >> For additional commands, e-mail: dev-help@cassandra.apache.org
>> > > > > >>
>> > > > > >>
>> > > > > >>
>> > > ---------------------------------------------------------------------
>> > > > > >> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
>> > > > > >> For additional commands, e-mail: dev-help@cassandra.apache.org
>> > > > > >>
>> > > > > >>
>> > > > >
>> > > > >
>> > > > >
>> ---------------------------------------------------------------------
>> > > > > To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
>> > > > > For additional commands, e-mail: dev-help@cassandra.apache.org
>> > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>
Re: [DISCUSS] CASSANDRA-13994
Posted by Ekaterina Dimitrova <e....@gmail.com>.
It was moved to alpha following the lifecycle document and the latest
discussion on the ML. Sylvain made a review yesterday. To me the biggest
question is about the indexing. (Posted questions on the ticket itself,
waiting for opinion and agreement) For the rest of the ticket I support
Sylvain that it is primarily removal of dead code.
My understanding is either it lands in alpha before the big testing in beta
or it is delayed for v.5
Ekaterina
On Wed, 10 Jun 2020 at 12:59, Joshua McKenzie <jm...@apache.org> wrote:
> Where did we land on inclusion of this ticket in 4.0? Since it's now
> flagged as blocking beta rel, I am now very interested in this question.
>
> :)
>
>
> On Wed, May 27, 2020 at 7:52 PM Jordan West <jo...@gmail.com> wrote:
>
> > On Wed, May 27, 2020 at 1:23 PM Joshua McKenzie <jm...@apache.org>
> > wrote:
> >
> > > Maybe. Do we just time box, say we're going to cut an RC and give it 4
> > > weeks, if nothing awful surfaces we GA?
> > >
> >
> > I've seen that work well in the past on other projects. I agree with the
> > notion that RCs are real candidates for release if no one finds issues
> with
> > them. Ideally we would have as little RCs as possible and have more
> > alphas/betas.
> >
> > >
> > > On Wed, May 27, 2020 at 4:12 PM Brandon Williams <dr...@gmail.com>
> > wrote:
> > >
> > > > Absolutely my understanding.
> > > >
> > > > On Wed, May 27, 2020, 2:49 PM Jeremiah D Jordan <
> > > jeremiah.jordan@gmail.com
> > > > >
> > > > wrote:
> > > >
> > > > > > A clear point to cut RC's doesn't surface from the above for me.
> > > > > Releasing
> > > > > > an RC before broad verification seems wrong, and cutting an RC
> > after
> > > > the
> > > > > 4
> > > > > > points above may as well be GA because it's all known scope.
> > > > >
> > > > > Isn’t the whole point of an RC is that it could be the GA? It is a
> > > > > “release candidate”, meaning if no one finds any issues with it,
> that
> > > can
> > > > > them become the release? So that seems like exactly the right time
> > to
> > > > make
> > > > > RC releases?
> > > > >
> > > > > > On May 27, 2020, at 2:45 PM, Joshua McKenzie <
> jmckenzie@apache.org
> > >
> > > > > wrote:
> > > > > >
> > > > > > I think we're all on the same page here; I was focusing more on
> the
> > > > > release
> > > > > > lifecycles and sequencing than the entire version cycle. Good to
> > > > broaden
> > > > > > scope I think.
> > > > > >
> > > > > > One thing we're not considering is the separation of API changes
> > from
> > > > > major
> > > > > > changes and how that intersects with release milestones.
> > > > > >
> > > > > > Meaning:
> > > > > > 1. alpha phase
> > > > > > 2. Milestone: API freeze (all API changes pushed to next major)
> > > > > > 3. beta phase
> > > > > > 4. Verification phase (all major disruptive pushed to next major)
> > > > > >
> > > > > > A clear point to cut RC's doesn't surface from the above for me.
> > > > > Releasing
> > > > > > an RC before broad verification seems wrong, and cutting an RC
> > after
> > > > the
> > > > > 4
> > > > > > points above may as well be GA because it's all known scope.
> > > > > >
> > > > > > Thoughts?
> > > > > >
> > > > > > On Wed, May 27, 2020 at 3:28 PM Scott Andreas <
> > scott@paradoxica.net>
> > > > > wrote:
> > > > > >
> > > > > >> That makes sense to me, yep.
> > > > > >>
> > > > > >> My hope and expectation is that the time required for
> > "verification
> > > > > work"
> > > > > >> will shrink dramatically in the not too distant future - ideally
> > to
> > > a
> > > > > >> period of less than a month. In this world, the cost of missing
> > one
> > > > > train
> > > > > >> is reduced to catching the next one.
> > > > > >>
> > > > > >> One of the main goals in shifting focus from "testing" and "test
> > > > plans"
> > > > > to
> > > > > >> "test engineering" is automating as many aspects of release
> > > > > qualification
> > > > > >> as possible, with an asymptotic ideal as a function of compute
> > > > capacity
> > > > > and
> > > > > >> time. While such automation will never be complete (it's likely
> > that
> > > > > >> development of new features will/must include qualification
> infra
> > > > > changes
> > > > > >> to exercise them), if we're able to apply the same rigor to
> major
> > > > > releases
> > > > > >> as we are to patchlevel builds with little incremental effort,
> I'd
> > > be
> > > > > >> thrilled.
> > > > > >>
> > > > > >> This is mostly a way of saying:
> > > > > >> – I like the cadence/sequencing Benedict proposes below.
> > > > > >> – I think improvements in test engineering can reduce/eliminate
> > > > > >> invalidation and may increase the scope of what can be a
> candidate
> > > for
> > > > > >> merge on a given branch
> > > > > >> – And if not, the cost of missing the train is lower because
> we'll
> > > be
> > > > > able
> > > > > >> to deliver major releases more often.
> > > > > >>
> > > > > >> Scott
> > > > > >>
> > > > > >> ________________________________________
> > > > > >> From: Jeremiah D Jordan <je...@gmail.com>
> > > > > >> Sent: Wednesday, May 27, 2020 11:54 AM
> > > > > >> To: Cassandra DEV
> > > > > >> Subject: Re: [DISCUSS] CASSANDRA-13994
> > > > > >>
> > > > > >> +1 strongly agree. If we aren’t going to let something go into
> > > 4.0.0
> > > > > >> because it would "invalidate testing” then we can not let such a
> > > thing
> > > > > go
> > > > > >> into 4.0.1 unless we plan to re-do said testing for the patch
> > > release.
> > > > > >>
> > > > > >>> On May 27, 2020, at 1:31 PM, Benedict Elliott Smith <
> > > > > benedict@apache.org>
> > > > > >> wrote:
> > > > > >>>
> > > > > >>> I'm being told this still isn't clear, so let me try in a
> > > > bullet-point
> > > > > >> timeline:
> > > > > >>>
> > > > > >>> * 4.0 Beta
> > > > > >>> * 4.0 Verification Work
> > > > > >>> * [Merge Window]
> > > > > >>> * 4.0 GA
> > > > > >>> * 4.0 Minor Releases
> > > > > >>> * ...
> > > > > >>> * 5.0 Dev
> > > > > >>> * ...
> > > > > >>> * 5.0 Verification Work
> > > > > >>> * GA 5.0
> > > > > >>>
> > > > > >>> I think that anything that is prohibited from "[Merge Window]"
> > > > because
> > > > > >> it invalidates "4.0 Verification Work" must also be prohibited
> > until
> > > > > "5.0
> > > > > >> Dev" because the next equivalent work that can now validate it
> > > occurs
> > > > > only
> > > > > >> at "5.0 Verification Work"
> > > > > >>>
> > > > > >>> On 27/05/2020, 19:05, "Benedict Elliott Smith" <
> > > benedict@apache.org
> > > > >
> > > > > >> wrote:
> > > > > >>>
> > > > > >>> I'm not sure if I communicated my point very well. I mean to
> > say
> > > > > >> that if the reason we are prohibiting a patch to land post-beta
> is
> > > > > because
> > > > > >> it invalidates work we only perform pre-ga, then it probably
> > should
> > > > not
> > > > > be
> > > > > >> permitted to land post-ga either, since it must also invalidate
> > the
> > > > same
> > > > > >> work?
> > > > > >>>
> > > > > >>> That is to say, if we're comfortable with work landing
> post-ga
> > > > > >> because we believe it to be safe to release without our
> > > > > pre-major-release
> > > > > >> verification, we should be comfortable with it landing at any
> time
> > > > > pre-ga
> > > > > >> too. Anything else seems inconsistent to me, and we should
> > examine
> > > > what
> > > > > >> assumptions we're making that permit this inconsistency to
> arise.
> > > > > >>>
> > > > > >>>
> > > > > >>> On 27/05/2020, 18:49, "Joshua McKenzie" <
> jmckenzie@apache.org
> > >
> > > > > >> wrote:
> > > > > >>>
> > > > > >>>>
> > > > > >>>> because it invalidates our pre-release verification, then it
> > > should
> > > > > not
> > > > > >>>> land
> > > > > >>>
> > > > > >>> until we next perform pre-release verification
> > > > > >>>
> > > > > >>> At least for me there's a little softness around our
> > > collective
> > > > > >> alignment
> > > > > >>> on when pre-release verification takes place. If it's
> > between
> > > > > >> alpha-1 and
> > > > > >>> ga we don't want changes that would invalidate those
> > changes
> > > to
> > > > > >> land during
> > > > > >>> that time frame. Different for beta-1 to ga. We also risk
> > > > > >> invalidating
> > > > > >>> testing if we do any of that testing before wherever that
> > > > cutoff
> > > > > >> is, and a
> > > > > >>> lack of clarity on that cutoff further muddies those
> > waters.
> > > > > >>>
> > > > > >>> My very loosely held perspective is that beta-1 to ga is
> > the
> > > > > >> window in
> > > > > >>> which we apply the "don't do things that will invalidate
> > > > > >> verification", and
> > > > > >>> we plan to do that verification during the beta phase. I
> > > > *think*
> > > > > >> this is
> > > > > >>> consistent w/the current framing of the lifecycle doc.
> That
> > > > being
> > > > > >> said, I
> > > > > >>> don't have strong religion on this so if we collectively
> > want
> > > > to
> > > > > >> call it
> > > > > >>> "don't majorly disrupt from alpha-1 to ga", we can
> > formalize
> > > > that
> > > > > >> in the
> > > > > >>> docs and go ahead and triage current open scope for 4.0
> and
> > > > move
> > > > > >> things out.
> > > > > >>>
> > > > > >>>
> > > > > >>>
> > > > > >>> On Wed, May 27, 2020 at 12:59 PM Ekaterina Dimitrova <
> > > > > >>> ekaterina.dimitrova@datastax.com> wrote:
> > > > > >>>
> > > > > >>>> Thank you all for your input.
> > > > > >>>> I think an important topic is again to revise the lifecycle
> and
> > > > ensure
> > > > > >> we
> > > > > >>>> really have the vision on what is left until beta. I will
> start
> > a
> > > > > >> separate
> > > > > >>>> thread on the flaky tests situation soon.
> > > > > >>>>
> > > > > >>>> For this particular ticket I see a couple of things:
> > > > > >>>> - There are a lot of deletions of already not used code
> > > > > >>>> - I implemented it still in alpha as per our agreement that
> this
> > > > will
> > > > > >> give
> > > > > >>>> us enough time for testing. Probably Dinesh as a reviewer can
> > give
> > > > > some
> > > > > >>>> valuable feedback/opinion on the patch.
> > > > > >>>> - It definitely touches around important places but the
> > important
> > > > > thing
> > > > > >> is
> > > > > >>>> to see how exactly it touches, I think
> > > > > >>>> - Considering it for alpha before the major testing in beta
> > sounds
> > > > > >>>> reasonable to me but I guess it also depends on people
> > > availability
> > > > to
> > > > > >>>> review it in detail and the exact test plans afterwards
> > > > > >>>>
> > > > > >>>> On Wed, 27 May 2020 at 7:14, Benedict Elliott Smith <
> > > > > >> benedict@apache.org>
> > > > > >>>> wrote:
> > > > > >>>>
> > > > > >>>>> I think our "pre-beta" criteria should also be our "not in a
> > > major"
> > > > > >>>>> criteria.
> > > > > >>>>>
> > > > > >>>>> If work is prohibited because it invalidates our pre-release
> > > > > >>>> verification,
> > > > > >>>>> then it should not land until we next perform pre-release
> > > > > verification,
> > > > > >>>>> which only currently happens once per major.
> > > > > >>>>>
> > > > > >>>>> This could mean either landing less in a major, or permitting
> > > more
> > > > in
> > > > > >>>> beta
> > > > > >>>>> etc.
> > > > > >>>>>
> > > > > >>>>> On 26/05/2020, 19:24, "Joshua McKenzie" <
> jmckenzie@apache.org
> > >
> > > > > wrote:
> > > > > >>>>>
> > > > > >>>>> I think an interesting question that informs when to stop
> > > > accepting
> > > > > >>>>> specific changes in a release is when we expect any
> extensive
> > > > > >>>>> pre-release
> > > > > >>>>> testing to take place.
> > > > > >>>>>
> > > > > >>>>> If we go by our release lifecycle, gutting deprecated code
> > > seems
> > > > > >>>>> compatible
> > > > > >>>>> w/Alpha but I wouldn't endorse merging it into Beta:
> > > > > >>>>>
> > > > > >>>>>
> > > > > >>
> > > >
> > https://cwiki.apache.org/confluence/display/CASSANDRA/Release+Lifecycle
> > > > > .
> > > > > >>>>> Since almost all of the 40_quality_testing epic stuff is
> also
> > > > beta
> > > > > >>>>> phase
> > > > > >>>>> and hasn't really taken off yet, it also seems like there
> > will
> > > be
> > > > > >>>>> extensive
> > > > > >>>>> testing after this phase transition.
> > > > > >>>>>
> > > > > >>>>> All that being said, I'd advocate for marking FixVer 4.x to
> > > > > indicate
> > > > > >>>>> optionality and disallow merge of tickets like this after
> > we're
> > > > > done
> > > > > >>>>> w/alpha phase in keeping w/our lifecycle doc in general.
> > > > > >>>>>
> > > > > >>>>> Does that make sense? Should we consider revisiting and
> > > revising
> > > > > the
> > > > > >>>>> lifecycle doc re: larger deprecation / changes and cycle
> > > stages?
> > > > > >>>>>
> > > > > >>>>>
> > > > > >>>>>
> > > > > >>>>> On Tue, May 26, 2020 at 12:53 PM Oleksandr Petrov <
> > > > > >>>>> oleksandr.petrov@gmail.com> wrote:
> > > > > >>>>>
> > > > > >>>>>>> 1) Would you block the release over this ticket?
> > > > > >>>>>>
> > > > > >>>>>> I would definitely not block the release on this ticket.
> > > > > >>>>>>
> > > > > >>>>>>> 2) Would you prioritize this ticket over testing?
> > > > > >>>>>>
> > > > > >>>>>> Same here, I would prioritise testing.
> > > > > >>>>>>
> > > > > >>>>>>> 3) Does fixing this ticket make 4.0 a more stable release?
> > > > > >>>>>>
> > > > > >>>>>> I wanted to give some context: I wrote that in August 2018.
> > > While
> > > > I
> > > > > >>>>> still
> > > > > >>>>>> believe it is important to get rid of this code, I'm
> > disinclined
> > > > to
> > > > > >>>>> merge
> > > > > >>>>>> it into 4.0.
> > > > > >>>>>>
> > > > > >>>>>> Given that the patch is rather big (421 additions and 1,480
> > > > > >>>>> deletions) and
> > > > > >>>>>> touches many important places, including parser, I would be
> > > > > >>>> extremely
> > > > > >>>>>> cautious to merge it that late in release cycle. It would be
> > > great
> > > > > >>>>> to also
> > > > > >>>>>> hear arguments that would justify the risk.
> > > > > >>>>>>
> > > > > >>>>>> Thank you for starting this discussion,
> > > > > >>>>>> -- Alex
> > > > > >>>>>>
> > > > > >>>>>>
> > > > > >>>>>>
> > > > > >>>>>> On Tue, May 26, 2020 at 5:20 PM Ekaterina Dimitrova <
> > > > > >>>>>> ekaterina.dimitrova@datastax.com> wrote:
> > > > > >>>>>>
> > > > > >>>>>>> Dear all,
> > > > > >>>>>>>
> > > > > >>>>>>> Following the ticket review sent on 12th May I wanted to
> > bring
> > > up
> > > > > >>>>>>> https://issues.apache.org/jira/browse/CASSANDRA-13994:
> > Remove
> > > > > >>>>> COMPACT
> > > > > >>>>>>>
> > > > > >>>>>>> STORAGE internals before 4.0 release.
> > > > > >>>>>>>
> > > > > >>>>>>> It is already under review by Dinesh Joshi and Alex Petrov.
> > > Not a
> > > > > >>>>>>> blocker but already under review.
> > > > > >>>>>>>
> > > > > >>>>>>> Below are my responses to the questions brought up.
> > > > > >>>>>>>
> > > > > >>>>>>>
> > > > > >>>>>>> 1) Would you block the release over this
> > > > > >>>>>>>
> > > > > >>>>>>> ticket? - probably not
> > > > > >>>>>>>
> > > > > >>>>>>> 2) Would you prioritize this ticket over testing? - already
> > > > > >>>>>>> implemented but if there are some big changes needed after
> > the
> > > > > >>>>> review,
> > > > > >>>>>>> I doubt it we will want to prioritize over the testing
> > > > > >>>>>>>
> > > > > >>>>>>> 3) Does fixing
> > > > > >>>>>>> this ticket make 4.0 a more stable release? - I will just
> > cite
> > > > > >>>> Alex
> > > > > >>>>>>> Petrov who reported this Jira and I think the rest of us
> > would
> > > > > >>>>> agree
> > > > > >>>>>>> with him here.
> > > > > >>>>>>>
> > > > > >>>>>>> "I would say it's quite important to clean up compact
> storage
> > > > > >>>>>>> internals in 4.0 before the release. It should have no
> > visible
> > > > > >>>>>>> side-effects, but it'd be very good to have as it
> simplifies
> > > > > >>>>> multiple
> > > > > >>>>>>> code paths."
> > > > > >>>>>>>
> > > > > >>>>>>>
> > > > > >>>>>>> Ekaterina Dimitrova
> > > > > >>>>>>> e. ekaterina.dimitrova@datastax.com
> > > > > >>>>>>> w. www.datastax.com
> > > > > >>>>>>>
> > > > > >>>>>>
> > > > > >>>>>>
> > > > > >>>>>> --
> > > > > >>>>>> alex p
> > > > > >>>>>>
> > > > > >>>>>
> > > > > >>>>>
> > > > > >>>>>
> > > > > >>>>>
> > > > ---------------------------------------------------------------------
> > > > > >>>>> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> > > > > >>>>> For additional commands, e-mail:
> dev-help@cassandra.apache.org
> > > > > >>>>>
> > > > > >>>>>
> > > > > >>>>
> > > > > >>>
> > > > > >>>
> > > > > >>>
> > > > > >>>
> > > >
> ---------------------------------------------------------------------
> > > > > >>> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> > > > > >>> For additional commands, e-mail:
> dev-help@cassandra.apache.org
> > > > > >>>
> > > > > >>>
> > > > > >>>
> > > > > >>>
> > > > > >>>
> > > ---------------------------------------------------------------------
> > > > > >>> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> > > > > >>> For additional commands, e-mail: dev-help@cassandra.apache.org
> > > > > >>>
> > > > > >>
> > > > > >>
> > > > > >>
> > > ---------------------------------------------------------------------
> > > > > >> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> > > > > >> For additional commands, e-mail: dev-help@cassandra.apache.org
> > > > > >>
> > > > > >>
> > > > > >>
> > > ---------------------------------------------------------------------
> > > > > >> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> > > > > >> For additional commands, e-mail: dev-help@cassandra.apache.org
> > > > > >>
> > > > > >>
> > > > >
> > > > >
> > > > >
> ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> > > > > For additional commands, e-mail: dev-help@cassandra.apache.org
> > > > >
> > > > >
> > > >
> > >
> >
>
Re: [DISCUSS] CASSANDRA-13994
Posted by Joshua McKenzie <jm...@apache.org>.
Where did we land on inclusion of this ticket in 4.0? Since it's now
flagged as blocking beta rel, I am now very interested in this question.
:)
On Wed, May 27, 2020 at 7:52 PM Jordan West <jo...@gmail.com> wrote:
> On Wed, May 27, 2020 at 1:23 PM Joshua McKenzie <jm...@apache.org>
> wrote:
>
> > Maybe. Do we just time box, say we're going to cut an RC and give it 4
> > weeks, if nothing awful surfaces we GA?
> >
>
> I've seen that work well in the past on other projects. I agree with the
> notion that RCs are real candidates for release if no one finds issues with
> them. Ideally we would have as little RCs as possible and have more
> alphas/betas.
>
> >
> > On Wed, May 27, 2020 at 4:12 PM Brandon Williams <dr...@gmail.com>
> wrote:
> >
> > > Absolutely my understanding.
> > >
> > > On Wed, May 27, 2020, 2:49 PM Jeremiah D Jordan <
> > jeremiah.jordan@gmail.com
> > > >
> > > wrote:
> > >
> > > > > A clear point to cut RC's doesn't surface from the above for me.
> > > > Releasing
> > > > > an RC before broad verification seems wrong, and cutting an RC
> after
> > > the
> > > > 4
> > > > > points above may as well be GA because it's all known scope.
> > > >
> > > > Isn’t the whole point of an RC is that it could be the GA? It is a
> > > > “release candidate”, meaning if no one finds any issues with it, that
> > can
> > > > them become the release? So that seems like exactly the right time
> to
> > > make
> > > > RC releases?
> > > >
> > > > > On May 27, 2020, at 2:45 PM, Joshua McKenzie <jmckenzie@apache.org
> >
> > > > wrote:
> > > > >
> > > > > I think we're all on the same page here; I was focusing more on the
> > > > release
> > > > > lifecycles and sequencing than the entire version cycle. Good to
> > > broaden
> > > > > scope I think.
> > > > >
> > > > > One thing we're not considering is the separation of API changes
> from
> > > > major
> > > > > changes and how that intersects with release milestones.
> > > > >
> > > > > Meaning:
> > > > > 1. alpha phase
> > > > > 2. Milestone: API freeze (all API changes pushed to next major)
> > > > > 3. beta phase
> > > > > 4. Verification phase (all major disruptive pushed to next major)
> > > > >
> > > > > A clear point to cut RC's doesn't surface from the above for me.
> > > > Releasing
> > > > > an RC before broad verification seems wrong, and cutting an RC
> after
> > > the
> > > > 4
> > > > > points above may as well be GA because it's all known scope.
> > > > >
> > > > > Thoughts?
> > > > >
> > > > > On Wed, May 27, 2020 at 3:28 PM Scott Andreas <
> scott@paradoxica.net>
> > > > wrote:
> > > > >
> > > > >> That makes sense to me, yep.
> > > > >>
> > > > >> My hope and expectation is that the time required for
> "verification
> > > > work"
> > > > >> will shrink dramatically in the not too distant future - ideally
> to
> > a
> > > > >> period of less than a month. In this world, the cost of missing
> one
> > > > train
> > > > >> is reduced to catching the next one.
> > > > >>
> > > > >> One of the main goals in shifting focus from "testing" and "test
> > > plans"
> > > > to
> > > > >> "test engineering" is automating as many aspects of release
> > > > qualification
> > > > >> as possible, with an asymptotic ideal as a function of compute
> > > capacity
> > > > and
> > > > >> time. While such automation will never be complete (it's likely
> that
> > > > >> development of new features will/must include qualification infra
> > > > changes
> > > > >> to exercise them), if we're able to apply the same rigor to major
> > > > releases
> > > > >> as we are to patchlevel builds with little incremental effort, I'd
> > be
> > > > >> thrilled.
> > > > >>
> > > > >> This is mostly a way of saying:
> > > > >> – I like the cadence/sequencing Benedict proposes below.
> > > > >> – I think improvements in test engineering can reduce/eliminate
> > > > >> invalidation and may increase the scope of what can be a candidate
> > for
> > > > >> merge on a given branch
> > > > >> – And if not, the cost of missing the train is lower because we'll
> > be
> > > > able
> > > > >> to deliver major releases more often.
> > > > >>
> > > > >> Scott
> > > > >>
> > > > >> ________________________________________
> > > > >> From: Jeremiah D Jordan <je...@gmail.com>
> > > > >> Sent: Wednesday, May 27, 2020 11:54 AM
> > > > >> To: Cassandra DEV
> > > > >> Subject: Re: [DISCUSS] CASSANDRA-13994
> > > > >>
> > > > >> +1 strongly agree. If we aren’t going to let something go into
> > 4.0.0
> > > > >> because it would "invalidate testing” then we can not let such a
> > thing
> > > > go
> > > > >> into 4.0.1 unless we plan to re-do said testing for the patch
> > release.
> > > > >>
> > > > >>> On May 27, 2020, at 1:31 PM, Benedict Elliott Smith <
> > > > benedict@apache.org>
> > > > >> wrote:
> > > > >>>
> > > > >>> I'm being told this still isn't clear, so let me try in a
> > > bullet-point
> > > > >> timeline:
> > > > >>>
> > > > >>> * 4.0 Beta
> > > > >>> * 4.0 Verification Work
> > > > >>> * [Merge Window]
> > > > >>> * 4.0 GA
> > > > >>> * 4.0 Minor Releases
> > > > >>> * ...
> > > > >>> * 5.0 Dev
> > > > >>> * ...
> > > > >>> * 5.0 Verification Work
> > > > >>> * GA 5.0
> > > > >>>
> > > > >>> I think that anything that is prohibited from "[Merge Window]"
> > > because
> > > > >> it invalidates "4.0 Verification Work" must also be prohibited
> until
> > > > "5.0
> > > > >> Dev" because the next equivalent work that can now validate it
> > occurs
> > > > only
> > > > >> at "5.0 Verification Work"
> > > > >>>
> > > > >>> On 27/05/2020, 19:05, "Benedict Elliott Smith" <
> > benedict@apache.org
> > > >
> > > > >> wrote:
> > > > >>>
> > > > >>> I'm not sure if I communicated my point very well. I mean to
> say
> > > > >> that if the reason we are prohibiting a patch to land post-beta is
> > > > because
> > > > >> it invalidates work we only perform pre-ga, then it probably
> should
> > > not
> > > > be
> > > > >> permitted to land post-ga either, since it must also invalidate
> the
> > > same
> > > > >> work?
> > > > >>>
> > > > >>> That is to say, if we're comfortable with work landing post-ga
> > > > >> because we believe it to be safe to release without our
> > > > pre-major-release
> > > > >> verification, we should be comfortable with it landing at any time
> > > > pre-ga
> > > > >> too. Anything else seems inconsistent to me, and we should
> examine
> > > what
> > > > >> assumptions we're making that permit this inconsistency to arise.
> > > > >>>
> > > > >>>
> > > > >>> On 27/05/2020, 18:49, "Joshua McKenzie" <jmckenzie@apache.org
> >
> > > > >> wrote:
> > > > >>>
> > > > >>>>
> > > > >>>> because it invalidates our pre-release verification, then it
> > should
> > > > not
> > > > >>>> land
> > > > >>>
> > > > >>> until we next perform pre-release verification
> > > > >>>
> > > > >>> At least for me there's a little softness around our
> > collective
> > > > >> alignment
> > > > >>> on when pre-release verification takes place. If it's
> between
> > > > >> alpha-1 and
> > > > >>> ga we don't want changes that would invalidate those
> changes
> > to
> > > > >> land during
> > > > >>> that time frame. Different for beta-1 to ga. We also risk
> > > > >> invalidating
> > > > >>> testing if we do any of that testing before wherever that
> > > cutoff
> > > > >> is, and a
> > > > >>> lack of clarity on that cutoff further muddies those
> waters.
> > > > >>>
> > > > >>> My very loosely held perspective is that beta-1 to ga is
> the
> > > > >> window in
> > > > >>> which we apply the "don't do things that will invalidate
> > > > >> verification", and
> > > > >>> we plan to do that verification during the beta phase. I
> > > *think*
> > > > >> this is
> > > > >>> consistent w/the current framing of the lifecycle doc. That
> > > being
> > > > >> said, I
> > > > >>> don't have strong religion on this so if we collectively
> want
> > > to
> > > > >> call it
> > > > >>> "don't majorly disrupt from alpha-1 to ga", we can
> formalize
> > > that
> > > > >> in the
> > > > >>> docs and go ahead and triage current open scope for 4.0 and
> > > move
> > > > >> things out.
> > > > >>>
> > > > >>>
> > > > >>>
> > > > >>> On Wed, May 27, 2020 at 12:59 PM Ekaterina Dimitrova <
> > > > >>> ekaterina.dimitrova@datastax.com> wrote:
> > > > >>>
> > > > >>>> Thank you all for your input.
> > > > >>>> I think an important topic is again to revise the lifecycle and
> > > ensure
> > > > >> we
> > > > >>>> really have the vision on what is left until beta. I will start
> a
> > > > >> separate
> > > > >>>> thread on the flaky tests situation soon.
> > > > >>>>
> > > > >>>> For this particular ticket I see a couple of things:
> > > > >>>> - There are a lot of deletions of already not used code
> > > > >>>> - I implemented it still in alpha as per our agreement that this
> > > will
> > > > >> give
> > > > >>>> us enough time for testing. Probably Dinesh as a reviewer can
> give
> > > > some
> > > > >>>> valuable feedback/opinion on the patch.
> > > > >>>> - It definitely touches around important places but the
> important
> > > > thing
> > > > >> is
> > > > >>>> to see how exactly it touches, I think
> > > > >>>> - Considering it for alpha before the major testing in beta
> sounds
> > > > >>>> reasonable to me but I guess it also depends on people
> > availability
> > > to
> > > > >>>> review it in detail and the exact test plans afterwards
> > > > >>>>
> > > > >>>> On Wed, 27 May 2020 at 7:14, Benedict Elliott Smith <
> > > > >> benedict@apache.org>
> > > > >>>> wrote:
> > > > >>>>
> > > > >>>>> I think our "pre-beta" criteria should also be our "not in a
> > major"
> > > > >>>>> criteria.
> > > > >>>>>
> > > > >>>>> If work is prohibited because it invalidates our pre-release
> > > > >>>> verification,
> > > > >>>>> then it should not land until we next perform pre-release
> > > > verification,
> > > > >>>>> which only currently happens once per major.
> > > > >>>>>
> > > > >>>>> This could mean either landing less in a major, or permitting
> > more
> > > in
> > > > >>>> beta
> > > > >>>>> etc.
> > > > >>>>>
> > > > >>>>> On 26/05/2020, 19:24, "Joshua McKenzie" <jmckenzie@apache.org
> >
> > > > wrote:
> > > > >>>>>
> > > > >>>>> I think an interesting question that informs when to stop
> > > accepting
> > > > >>>>> specific changes in a release is when we expect any extensive
> > > > >>>>> pre-release
> > > > >>>>> testing to take place.
> > > > >>>>>
> > > > >>>>> If we go by our release lifecycle, gutting deprecated code
> > seems
> > > > >>>>> compatible
> > > > >>>>> w/Alpha but I wouldn't endorse merging it into Beta:
> > > > >>>>>
> > > > >>>>>
> > > > >>
> > >
> https://cwiki.apache.org/confluence/display/CASSANDRA/Release+Lifecycle
> > > > .
> > > > >>>>> Since almost all of the 40_quality_testing epic stuff is also
> > > beta
> > > > >>>>> phase
> > > > >>>>> and hasn't really taken off yet, it also seems like there
> will
> > be
> > > > >>>>> extensive
> > > > >>>>> testing after this phase transition.
> > > > >>>>>
> > > > >>>>> All that being said, I'd advocate for marking FixVer 4.x to
> > > > indicate
> > > > >>>>> optionality and disallow merge of tickets like this after
> we're
> > > > done
> > > > >>>>> w/alpha phase in keeping w/our lifecycle doc in general.
> > > > >>>>>
> > > > >>>>> Does that make sense? Should we consider revisiting and
> > revising
> > > > the
> > > > >>>>> lifecycle doc re: larger deprecation / changes and cycle
> > stages?
> > > > >>>>>
> > > > >>>>>
> > > > >>>>>
> > > > >>>>> On Tue, May 26, 2020 at 12:53 PM Oleksandr Petrov <
> > > > >>>>> oleksandr.petrov@gmail.com> wrote:
> > > > >>>>>
> > > > >>>>>>> 1) Would you block the release over this ticket?
> > > > >>>>>>
> > > > >>>>>> I would definitely not block the release on this ticket.
> > > > >>>>>>
> > > > >>>>>>> 2) Would you prioritize this ticket over testing?
> > > > >>>>>>
> > > > >>>>>> Same here, I would prioritise testing.
> > > > >>>>>>
> > > > >>>>>>> 3) Does fixing this ticket make 4.0 a more stable release?
> > > > >>>>>>
> > > > >>>>>> I wanted to give some context: I wrote that in August 2018.
> > While
> > > I
> > > > >>>>> still
> > > > >>>>>> believe it is important to get rid of this code, I'm
> disinclined
> > > to
> > > > >>>>> merge
> > > > >>>>>> it into 4.0.
> > > > >>>>>>
> > > > >>>>>> Given that the patch is rather big (421 additions and 1,480
> > > > >>>>> deletions) and
> > > > >>>>>> touches many important places, including parser, I would be
> > > > >>>> extremely
> > > > >>>>>> cautious to merge it that late in release cycle. It would be
> > great
> > > > >>>>> to also
> > > > >>>>>> hear arguments that would justify the risk.
> > > > >>>>>>
> > > > >>>>>> Thank you for starting this discussion,
> > > > >>>>>> -- Alex
> > > > >>>>>>
> > > > >>>>>>
> > > > >>>>>>
> > > > >>>>>> On Tue, May 26, 2020 at 5:20 PM Ekaterina Dimitrova <
> > > > >>>>>> ekaterina.dimitrova@datastax.com> wrote:
> > > > >>>>>>
> > > > >>>>>>> Dear all,
> > > > >>>>>>>
> > > > >>>>>>> Following the ticket review sent on 12th May I wanted to
> bring
> > up
> > > > >>>>>>> https://issues.apache.org/jira/browse/CASSANDRA-13994:
> Remove
> > > > >>>>> COMPACT
> > > > >>>>>>>
> > > > >>>>>>> STORAGE internals before 4.0 release.
> > > > >>>>>>>
> > > > >>>>>>> It is already under review by Dinesh Joshi and Alex Petrov.
> > Not a
> > > > >>>>>>> blocker but already under review.
> > > > >>>>>>>
> > > > >>>>>>> Below are my responses to the questions brought up.
> > > > >>>>>>>
> > > > >>>>>>>
> > > > >>>>>>> 1) Would you block the release over this
> > > > >>>>>>>
> > > > >>>>>>> ticket? - probably not
> > > > >>>>>>>
> > > > >>>>>>> 2) Would you prioritize this ticket over testing? - already
> > > > >>>>>>> implemented but if there are some big changes needed after
> the
> > > > >>>>> review,
> > > > >>>>>>> I doubt it we will want to prioritize over the testing
> > > > >>>>>>>
> > > > >>>>>>> 3) Does fixing
> > > > >>>>>>> this ticket make 4.0 a more stable release? - I will just
> cite
> > > > >>>> Alex
> > > > >>>>>>> Petrov who reported this Jira and I think the rest of us
> would
> > > > >>>>> agree
> > > > >>>>>>> with him here.
> > > > >>>>>>>
> > > > >>>>>>> "I would say it's quite important to clean up compact storage
> > > > >>>>>>> internals in 4.0 before the release. It should have no
> visible
> > > > >>>>>>> side-effects, but it'd be very good to have as it simplifies
> > > > >>>>> multiple
> > > > >>>>>>> code paths."
> > > > >>>>>>>
> > > > >>>>>>>
> > > > >>>>>>> Ekaterina Dimitrova
> > > > >>>>>>> e. ekaterina.dimitrova@datastax.com
> > > > >>>>>>> w. www.datastax.com
> > > > >>>>>>>
> > > > >>>>>>
> > > > >>>>>>
> > > > >>>>>> --
> > > > >>>>>> alex p
> > > > >>>>>>
> > > > >>>>>
> > > > >>>>>
> > > > >>>>>
> > > > >>>>>
> > > ---------------------------------------------------------------------
> > > > >>>>> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> > > > >>>>> For additional commands, e-mail: dev-help@cassandra.apache.org
> > > > >>>>>
> > > > >>>>>
> > > > >>>>
> > > > >>>
> > > > >>>
> > > > >>>
> > > > >>>
> > > ---------------------------------------------------------------------
> > > > >>> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> > > > >>> For additional commands, e-mail: dev-help@cassandra.apache.org
> > > > >>>
> > > > >>>
> > > > >>>
> > > > >>>
> > > > >>>
> > ---------------------------------------------------------------------
> > > > >>> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> > > > >>> For additional commands, e-mail: dev-help@cassandra.apache.org
> > > > >>>
> > > > >>
> > > > >>
> > > > >>
> > ---------------------------------------------------------------------
> > > > >> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> > > > >> For additional commands, e-mail: dev-help@cassandra.apache.org
> > > > >>
> > > > >>
> > > > >>
> > ---------------------------------------------------------------------
> > > > >> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> > > > >> For additional commands, e-mail: dev-help@cassandra.apache.org
> > > > >>
> > > > >>
> > > >
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> > > > For additional commands, e-mail: dev-help@cassandra.apache.org
> > > >
> > > >
> > >
> >
>
Re: [DISCUSS] CASSANDRA-13994
Posted by Jordan West <jo...@gmail.com>.
On Wed, May 27, 2020 at 1:23 PM Joshua McKenzie <jm...@apache.org>
wrote:
> Maybe. Do we just time box, say we're going to cut an RC and give it 4
> weeks, if nothing awful surfaces we GA?
>
I've seen that work well in the past on other projects. I agree with the
notion that RCs are real candidates for release if no one finds issues with
them. Ideally we would have as little RCs as possible and have more
alphas/betas.
>
> On Wed, May 27, 2020 at 4:12 PM Brandon Williams <dr...@gmail.com> wrote:
>
> > Absolutely my understanding.
> >
> > On Wed, May 27, 2020, 2:49 PM Jeremiah D Jordan <
> jeremiah.jordan@gmail.com
> > >
> > wrote:
> >
> > > > A clear point to cut RC's doesn't surface from the above for me.
> > > Releasing
> > > > an RC before broad verification seems wrong, and cutting an RC after
> > the
> > > 4
> > > > points above may as well be GA because it's all known scope.
> > >
> > > Isn’t the whole point of an RC is that it could be the GA? It is a
> > > “release candidate”, meaning if no one finds any issues with it, that
> can
> > > them become the release? So that seems like exactly the right time to
> > make
> > > RC releases?
> > >
> > > > On May 27, 2020, at 2:45 PM, Joshua McKenzie <jm...@apache.org>
> > > wrote:
> > > >
> > > > I think we're all on the same page here; I was focusing more on the
> > > release
> > > > lifecycles and sequencing than the entire version cycle. Good to
> > broaden
> > > > scope I think.
> > > >
> > > > One thing we're not considering is the separation of API changes from
> > > major
> > > > changes and how that intersects with release milestones.
> > > >
> > > > Meaning:
> > > > 1. alpha phase
> > > > 2. Milestone: API freeze (all API changes pushed to next major)
> > > > 3. beta phase
> > > > 4. Verification phase (all major disruptive pushed to next major)
> > > >
> > > > A clear point to cut RC's doesn't surface from the above for me.
> > > Releasing
> > > > an RC before broad verification seems wrong, and cutting an RC after
> > the
> > > 4
> > > > points above may as well be GA because it's all known scope.
> > > >
> > > > Thoughts?
> > > >
> > > > On Wed, May 27, 2020 at 3:28 PM Scott Andreas <sc...@paradoxica.net>
> > > wrote:
> > > >
> > > >> That makes sense to me, yep.
> > > >>
> > > >> My hope and expectation is that the time required for "verification
> > > work"
> > > >> will shrink dramatically in the not too distant future - ideally to
> a
> > > >> period of less than a month. In this world, the cost of missing one
> > > train
> > > >> is reduced to catching the next one.
> > > >>
> > > >> One of the main goals in shifting focus from "testing" and "test
> > plans"
> > > to
> > > >> "test engineering" is automating as many aspects of release
> > > qualification
> > > >> as possible, with an asymptotic ideal as a function of compute
> > capacity
> > > and
> > > >> time. While such automation will never be complete (it's likely that
> > > >> development of new features will/must include qualification infra
> > > changes
> > > >> to exercise them), if we're able to apply the same rigor to major
> > > releases
> > > >> as we are to patchlevel builds with little incremental effort, I'd
> be
> > > >> thrilled.
> > > >>
> > > >> This is mostly a way of saying:
> > > >> – I like the cadence/sequencing Benedict proposes below.
> > > >> – I think improvements in test engineering can reduce/eliminate
> > > >> invalidation and may increase the scope of what can be a candidate
> for
> > > >> merge on a given branch
> > > >> – And if not, the cost of missing the train is lower because we'll
> be
> > > able
> > > >> to deliver major releases more often.
> > > >>
> > > >> Scott
> > > >>
> > > >> ________________________________________
> > > >> From: Jeremiah D Jordan <je...@gmail.com>
> > > >> Sent: Wednesday, May 27, 2020 11:54 AM
> > > >> To: Cassandra DEV
> > > >> Subject: Re: [DISCUSS] CASSANDRA-13994
> > > >>
> > > >> +1 strongly agree. If we aren’t going to let something go into
> 4.0.0
> > > >> because it would "invalidate testing” then we can not let such a
> thing
> > > go
> > > >> into 4.0.1 unless we plan to re-do said testing for the patch
> release.
> > > >>
> > > >>> On May 27, 2020, at 1:31 PM, Benedict Elliott Smith <
> > > benedict@apache.org>
> > > >> wrote:
> > > >>>
> > > >>> I'm being told this still isn't clear, so let me try in a
> > bullet-point
> > > >> timeline:
> > > >>>
> > > >>> * 4.0 Beta
> > > >>> * 4.0 Verification Work
> > > >>> * [Merge Window]
> > > >>> * 4.0 GA
> > > >>> * 4.0 Minor Releases
> > > >>> * ...
> > > >>> * 5.0 Dev
> > > >>> * ...
> > > >>> * 5.0 Verification Work
> > > >>> * GA 5.0
> > > >>>
> > > >>> I think that anything that is prohibited from "[Merge Window]"
> > because
> > > >> it invalidates "4.0 Verification Work" must also be prohibited until
> > > "5.0
> > > >> Dev" because the next equivalent work that can now validate it
> occurs
> > > only
> > > >> at "5.0 Verification Work"
> > > >>>
> > > >>> On 27/05/2020, 19:05, "Benedict Elliott Smith" <
> benedict@apache.org
> > >
> > > >> wrote:
> > > >>>
> > > >>> I'm not sure if I communicated my point very well. I mean to say
> > > >> that if the reason we are prohibiting a patch to land post-beta is
> > > because
> > > >> it invalidates work we only perform pre-ga, then it probably should
> > not
> > > be
> > > >> permitted to land post-ga either, since it must also invalidate the
> > same
> > > >> work?
> > > >>>
> > > >>> That is to say, if we're comfortable with work landing post-ga
> > > >> because we believe it to be safe to release without our
> > > pre-major-release
> > > >> verification, we should be comfortable with it landing at any time
> > > pre-ga
> > > >> too. Anything else seems inconsistent to me, and we should examine
> > what
> > > >> assumptions we're making that permit this inconsistency to arise.
> > > >>>
> > > >>>
> > > >>> On 27/05/2020, 18:49, "Joshua McKenzie" <jm...@apache.org>
> > > >> wrote:
> > > >>>
> > > >>>>
> > > >>>> because it invalidates our pre-release verification, then it
> should
> > > not
> > > >>>> land
> > > >>>
> > > >>> until we next perform pre-release verification
> > > >>>
> > > >>> At least for me there's a little softness around our
> collective
> > > >> alignment
> > > >>> on when pre-release verification takes place. If it's between
> > > >> alpha-1 and
> > > >>> ga we don't want changes that would invalidate those changes
> to
> > > >> land during
> > > >>> that time frame. Different for beta-1 to ga. We also risk
> > > >> invalidating
> > > >>> testing if we do any of that testing before wherever that
> > cutoff
> > > >> is, and a
> > > >>> lack of clarity on that cutoff further muddies those waters.
> > > >>>
> > > >>> My very loosely held perspective is that beta-1 to ga is the
> > > >> window in
> > > >>> which we apply the "don't do things that will invalidate
> > > >> verification", and
> > > >>> we plan to do that verification during the beta phase. I
> > *think*
> > > >> this is
> > > >>> consistent w/the current framing of the lifecycle doc. That
> > being
> > > >> said, I
> > > >>> don't have strong religion on this so if we collectively want
> > to
> > > >> call it
> > > >>> "don't majorly disrupt from alpha-1 to ga", we can formalize
> > that
> > > >> in the
> > > >>> docs and go ahead and triage current open scope for 4.0 and
> > move
> > > >> things out.
> > > >>>
> > > >>>
> > > >>>
> > > >>> On Wed, May 27, 2020 at 12:59 PM Ekaterina Dimitrova <
> > > >>> ekaterina.dimitrova@datastax.com> wrote:
> > > >>>
> > > >>>> Thank you all for your input.
> > > >>>> I think an important topic is again to revise the lifecycle and
> > ensure
> > > >> we
> > > >>>> really have the vision on what is left until beta. I will start a
> > > >> separate
> > > >>>> thread on the flaky tests situation soon.
> > > >>>>
> > > >>>> For this particular ticket I see a couple of things:
> > > >>>> - There are a lot of deletions of already not used code
> > > >>>> - I implemented it still in alpha as per our agreement that this
> > will
> > > >> give
> > > >>>> us enough time for testing. Probably Dinesh as a reviewer can give
> > > some
> > > >>>> valuable feedback/opinion on the patch.
> > > >>>> - It definitely touches around important places but the important
> > > thing
> > > >> is
> > > >>>> to see how exactly it touches, I think
> > > >>>> - Considering it for alpha before the major testing in beta sounds
> > > >>>> reasonable to me but I guess it also depends on people
> availability
> > to
> > > >>>> review it in detail and the exact test plans afterwards
> > > >>>>
> > > >>>> On Wed, 27 May 2020 at 7:14, Benedict Elliott Smith <
> > > >> benedict@apache.org>
> > > >>>> wrote:
> > > >>>>
> > > >>>>> I think our "pre-beta" criteria should also be our "not in a
> major"
> > > >>>>> criteria.
> > > >>>>>
> > > >>>>> If work is prohibited because it invalidates our pre-release
> > > >>>> verification,
> > > >>>>> then it should not land until we next perform pre-release
> > > verification,
> > > >>>>> which only currently happens once per major.
> > > >>>>>
> > > >>>>> This could mean either landing less in a major, or permitting
> more
> > in
> > > >>>> beta
> > > >>>>> etc.
> > > >>>>>
> > > >>>>> On 26/05/2020, 19:24, "Joshua McKenzie" <jm...@apache.org>
> > > wrote:
> > > >>>>>
> > > >>>>> I think an interesting question that informs when to stop
> > accepting
> > > >>>>> specific changes in a release is when we expect any extensive
> > > >>>>> pre-release
> > > >>>>> testing to take place.
> > > >>>>>
> > > >>>>> If we go by our release lifecycle, gutting deprecated code
> seems
> > > >>>>> compatible
> > > >>>>> w/Alpha but I wouldn't endorse merging it into Beta:
> > > >>>>>
> > > >>>>>
> > > >>
> > https://cwiki.apache.org/confluence/display/CASSANDRA/Release+Lifecycle
> > > .
> > > >>>>> Since almost all of the 40_quality_testing epic stuff is also
> > beta
> > > >>>>> phase
> > > >>>>> and hasn't really taken off yet, it also seems like there will
> be
> > > >>>>> extensive
> > > >>>>> testing after this phase transition.
> > > >>>>>
> > > >>>>> All that being said, I'd advocate for marking FixVer 4.x to
> > > indicate
> > > >>>>> optionality and disallow merge of tickets like this after we're
> > > done
> > > >>>>> w/alpha phase in keeping w/our lifecycle doc in general.
> > > >>>>>
> > > >>>>> Does that make sense? Should we consider revisiting and
> revising
> > > the
> > > >>>>> lifecycle doc re: larger deprecation / changes and cycle
> stages?
> > > >>>>>
> > > >>>>>
> > > >>>>>
> > > >>>>> On Tue, May 26, 2020 at 12:53 PM Oleksandr Petrov <
> > > >>>>> oleksandr.petrov@gmail.com> wrote:
> > > >>>>>
> > > >>>>>>> 1) Would you block the release over this ticket?
> > > >>>>>>
> > > >>>>>> I would definitely not block the release on this ticket.
> > > >>>>>>
> > > >>>>>>> 2) Would you prioritize this ticket over testing?
> > > >>>>>>
> > > >>>>>> Same here, I would prioritise testing.
> > > >>>>>>
> > > >>>>>>> 3) Does fixing this ticket make 4.0 a more stable release?
> > > >>>>>>
> > > >>>>>> I wanted to give some context: I wrote that in August 2018.
> While
> > I
> > > >>>>> still
> > > >>>>>> believe it is important to get rid of this code, I'm disinclined
> > to
> > > >>>>> merge
> > > >>>>>> it into 4.0.
> > > >>>>>>
> > > >>>>>> Given that the patch is rather big (421 additions and 1,480
> > > >>>>> deletions) and
> > > >>>>>> touches many important places, including parser, I would be
> > > >>>> extremely
> > > >>>>>> cautious to merge it that late in release cycle. It would be
> great
> > > >>>>> to also
> > > >>>>>> hear arguments that would justify the risk.
> > > >>>>>>
> > > >>>>>> Thank you for starting this discussion,
> > > >>>>>> -- Alex
> > > >>>>>>
> > > >>>>>>
> > > >>>>>>
> > > >>>>>> On Tue, May 26, 2020 at 5:20 PM Ekaterina Dimitrova <
> > > >>>>>> ekaterina.dimitrova@datastax.com> wrote:
> > > >>>>>>
> > > >>>>>>> Dear all,
> > > >>>>>>>
> > > >>>>>>> Following the ticket review sent on 12th May I wanted to bring
> up
> > > >>>>>>> https://issues.apache.org/jira/browse/CASSANDRA-13994: Remove
> > > >>>>> COMPACT
> > > >>>>>>>
> > > >>>>>>> STORAGE internals before 4.0 release.
> > > >>>>>>>
> > > >>>>>>> It is already under review by Dinesh Joshi and Alex Petrov.
> Not a
> > > >>>>>>> blocker but already under review.
> > > >>>>>>>
> > > >>>>>>> Below are my responses to the questions brought up.
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>> 1) Would you block the release over this
> > > >>>>>>>
> > > >>>>>>> ticket? - probably not
> > > >>>>>>>
> > > >>>>>>> 2) Would you prioritize this ticket over testing? - already
> > > >>>>>>> implemented but if there are some big changes needed after the
> > > >>>>> review,
> > > >>>>>>> I doubt it we will want to prioritize over the testing
> > > >>>>>>>
> > > >>>>>>> 3) Does fixing
> > > >>>>>>> this ticket make 4.0 a more stable release? - I will just cite
> > > >>>> Alex
> > > >>>>>>> Petrov who reported this Jira and I think the rest of us would
> > > >>>>> agree
> > > >>>>>>> with him here.
> > > >>>>>>>
> > > >>>>>>> "I would say it's quite important to clean up compact storage
> > > >>>>>>> internals in 4.0 before the release. It should have no visible
> > > >>>>>>> side-effects, but it'd be very good to have as it simplifies
> > > >>>>> multiple
> > > >>>>>>> code paths."
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>> Ekaterina Dimitrova
> > > >>>>>>> e. ekaterina.dimitrova@datastax.com
> > > >>>>>>> w. www.datastax.com
> > > >>>>>>>
> > > >>>>>>
> > > >>>>>>
> > > >>>>>> --
> > > >>>>>> alex p
> > > >>>>>>
> > > >>>>>
> > > >>>>>
> > > >>>>>
> > > >>>>>
> > ---------------------------------------------------------------------
> > > >>>>> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> > > >>>>> For additional commands, e-mail: dev-help@cassandra.apache.org
> > > >>>>>
> > > >>>>>
> > > >>>>
> > > >>>
> > > >>>
> > > >>>
> > > >>>
> > ---------------------------------------------------------------------
> > > >>> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> > > >>> For additional commands, e-mail: dev-help@cassandra.apache.org
> > > >>>
> > > >>>
> > > >>>
> > > >>>
> > > >>>
> ---------------------------------------------------------------------
> > > >>> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> > > >>> For additional commands, e-mail: dev-help@cassandra.apache.org
> > > >>>
> > > >>
> > > >>
> > > >>
> ---------------------------------------------------------------------
> > > >> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> > > >> For additional commands, e-mail: dev-help@cassandra.apache.org
> > > >>
> > > >>
> > > >>
> ---------------------------------------------------------------------
> > > >> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> > > >> For additional commands, e-mail: dev-help@cassandra.apache.org
> > > >>
> > > >>
> > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> > > For additional commands, e-mail: dev-help@cassandra.apache.org
> > >
> > >
> >
>
Re: [DISCUSS] CASSANDRA-13994
Posted by Joshua McKenzie <jm...@apache.org>.
Maybe. Do we just time box, say we're going to cut an RC and give it 4
weeks, if nothing awful surfaces we GA?
On Wed, May 27, 2020 at 4:12 PM Brandon Williams <dr...@gmail.com> wrote:
> Absolutely my understanding.
>
> On Wed, May 27, 2020, 2:49 PM Jeremiah D Jordan <jeremiah.jordan@gmail.com
> >
> wrote:
>
> > > A clear point to cut RC's doesn't surface from the above for me.
> > Releasing
> > > an RC before broad verification seems wrong, and cutting an RC after
> the
> > 4
> > > points above may as well be GA because it's all known scope.
> >
> > Isn’t the whole point of an RC is that it could be the GA? It is a
> > “release candidate”, meaning if no one finds any issues with it, that can
> > them become the release? So that seems like exactly the right time to
> make
> > RC releases?
> >
> > > On May 27, 2020, at 2:45 PM, Joshua McKenzie <jm...@apache.org>
> > wrote:
> > >
> > > I think we're all on the same page here; I was focusing more on the
> > release
> > > lifecycles and sequencing than the entire version cycle. Good to
> broaden
> > > scope I think.
> > >
> > > One thing we're not considering is the separation of API changes from
> > major
> > > changes and how that intersects with release milestones.
> > >
> > > Meaning:
> > > 1. alpha phase
> > > 2. Milestone: API freeze (all API changes pushed to next major)
> > > 3. beta phase
> > > 4. Verification phase (all major disruptive pushed to next major)
> > >
> > > A clear point to cut RC's doesn't surface from the above for me.
> > Releasing
> > > an RC before broad verification seems wrong, and cutting an RC after
> the
> > 4
> > > points above may as well be GA because it's all known scope.
> > >
> > > Thoughts?
> > >
> > > On Wed, May 27, 2020 at 3:28 PM Scott Andreas <sc...@paradoxica.net>
> > wrote:
> > >
> > >> That makes sense to me, yep.
> > >>
> > >> My hope and expectation is that the time required for "verification
> > work"
> > >> will shrink dramatically in the not too distant future - ideally to a
> > >> period of less than a month. In this world, the cost of missing one
> > train
> > >> is reduced to catching the next one.
> > >>
> > >> One of the main goals in shifting focus from "testing" and "test
> plans"
> > to
> > >> "test engineering" is automating as many aspects of release
> > qualification
> > >> as possible, with an asymptotic ideal as a function of compute
> capacity
> > and
> > >> time. While such automation will never be complete (it's likely that
> > >> development of new features will/must include qualification infra
> > changes
> > >> to exercise them), if we're able to apply the same rigor to major
> > releases
> > >> as we are to patchlevel builds with little incremental effort, I'd be
> > >> thrilled.
> > >>
> > >> This is mostly a way of saying:
> > >> – I like the cadence/sequencing Benedict proposes below.
> > >> – I think improvements in test engineering can reduce/eliminate
> > >> invalidation and may increase the scope of what can be a candidate for
> > >> merge on a given branch
> > >> – And if not, the cost of missing the train is lower because we'll be
> > able
> > >> to deliver major releases more often.
> > >>
> > >> Scott
> > >>
> > >> ________________________________________
> > >> From: Jeremiah D Jordan <je...@gmail.com>
> > >> Sent: Wednesday, May 27, 2020 11:54 AM
> > >> To: Cassandra DEV
> > >> Subject: Re: [DISCUSS] CASSANDRA-13994
> > >>
> > >> +1 strongly agree. If we aren’t going to let something go into 4.0.0
> > >> because it would "invalidate testing” then we can not let such a thing
> > go
> > >> into 4.0.1 unless we plan to re-do said testing for the patch release.
> > >>
> > >>> On May 27, 2020, at 1:31 PM, Benedict Elliott Smith <
> > benedict@apache.org>
> > >> wrote:
> > >>>
> > >>> I'm being told this still isn't clear, so let me try in a
> bullet-point
> > >> timeline:
> > >>>
> > >>> * 4.0 Beta
> > >>> * 4.0 Verification Work
> > >>> * [Merge Window]
> > >>> * 4.0 GA
> > >>> * 4.0 Minor Releases
> > >>> * ...
> > >>> * 5.0 Dev
> > >>> * ...
> > >>> * 5.0 Verification Work
> > >>> * GA 5.0
> > >>>
> > >>> I think that anything that is prohibited from "[Merge Window]"
> because
> > >> it invalidates "4.0 Verification Work" must also be prohibited until
> > "5.0
> > >> Dev" because the next equivalent work that can now validate it occurs
> > only
> > >> at "5.0 Verification Work"
> > >>>
> > >>> On 27/05/2020, 19:05, "Benedict Elliott Smith" <benedict@apache.org
> >
> > >> wrote:
> > >>>
> > >>> I'm not sure if I communicated my point very well. I mean to say
> > >> that if the reason we are prohibiting a patch to land post-beta is
> > because
> > >> it invalidates work we only perform pre-ga, then it probably should
> not
> > be
> > >> permitted to land post-ga either, since it must also invalidate the
> same
> > >> work?
> > >>>
> > >>> That is to say, if we're comfortable with work landing post-ga
> > >> because we believe it to be safe to release without our
> > pre-major-release
> > >> verification, we should be comfortable with it landing at any time
> > pre-ga
> > >> too. Anything else seems inconsistent to me, and we should examine
> what
> > >> assumptions we're making that permit this inconsistency to arise.
> > >>>
> > >>>
> > >>> On 27/05/2020, 18:49, "Joshua McKenzie" <jm...@apache.org>
> > >> wrote:
> > >>>
> > >>>>
> > >>>> because it invalidates our pre-release verification, then it should
> > not
> > >>>> land
> > >>>
> > >>> until we next perform pre-release verification
> > >>>
> > >>> At least for me there's a little softness around our collective
> > >> alignment
> > >>> on when pre-release verification takes place. If it's between
> > >> alpha-1 and
> > >>> ga we don't want changes that would invalidate those changes to
> > >> land during
> > >>> that time frame. Different for beta-1 to ga. We also risk
> > >> invalidating
> > >>> testing if we do any of that testing before wherever that
> cutoff
> > >> is, and a
> > >>> lack of clarity on that cutoff further muddies those waters.
> > >>>
> > >>> My very loosely held perspective is that beta-1 to ga is the
> > >> window in
> > >>> which we apply the "don't do things that will invalidate
> > >> verification", and
> > >>> we plan to do that verification during the beta phase. I
> *think*
> > >> this is
> > >>> consistent w/the current framing of the lifecycle doc. That
> being
> > >> said, I
> > >>> don't have strong religion on this so if we collectively want
> to
> > >> call it
> > >>> "don't majorly disrupt from alpha-1 to ga", we can formalize
> that
> > >> in the
> > >>> docs and go ahead and triage current open scope for 4.0 and
> move
> > >> things out.
> > >>>
> > >>>
> > >>>
> > >>> On Wed, May 27, 2020 at 12:59 PM Ekaterina Dimitrova <
> > >>> ekaterina.dimitrova@datastax.com> wrote:
> > >>>
> > >>>> Thank you all for your input.
> > >>>> I think an important topic is again to revise the lifecycle and
> ensure
> > >> we
> > >>>> really have the vision on what is left until beta. I will start a
> > >> separate
> > >>>> thread on the flaky tests situation soon.
> > >>>>
> > >>>> For this particular ticket I see a couple of things:
> > >>>> - There are a lot of deletions of already not used code
> > >>>> - I implemented it still in alpha as per our agreement that this
> will
> > >> give
> > >>>> us enough time for testing. Probably Dinesh as a reviewer can give
> > some
> > >>>> valuable feedback/opinion on the patch.
> > >>>> - It definitely touches around important places but the important
> > thing
> > >> is
> > >>>> to see how exactly it touches, I think
> > >>>> - Considering it for alpha before the major testing in beta sounds
> > >>>> reasonable to me but I guess it also depends on people availability
> to
> > >>>> review it in detail and the exact test plans afterwards
> > >>>>
> > >>>> On Wed, 27 May 2020 at 7:14, Benedict Elliott Smith <
> > >> benedict@apache.org>
> > >>>> wrote:
> > >>>>
> > >>>>> I think our "pre-beta" criteria should also be our "not in a major"
> > >>>>> criteria.
> > >>>>>
> > >>>>> If work is prohibited because it invalidates our pre-release
> > >>>> verification,
> > >>>>> then it should not land until we next perform pre-release
> > verification,
> > >>>>> which only currently happens once per major.
> > >>>>>
> > >>>>> This could mean either landing less in a major, or permitting more
> in
> > >>>> beta
> > >>>>> etc.
> > >>>>>
> > >>>>> On 26/05/2020, 19:24, "Joshua McKenzie" <jm...@apache.org>
> > wrote:
> > >>>>>
> > >>>>> I think an interesting question that informs when to stop
> accepting
> > >>>>> specific changes in a release is when we expect any extensive
> > >>>>> pre-release
> > >>>>> testing to take place.
> > >>>>>
> > >>>>> If we go by our release lifecycle, gutting deprecated code seems
> > >>>>> compatible
> > >>>>> w/Alpha but I wouldn't endorse merging it into Beta:
> > >>>>>
> > >>>>>
> > >>
> https://cwiki.apache.org/confluence/display/CASSANDRA/Release+Lifecycle
> > .
> > >>>>> Since almost all of the 40_quality_testing epic stuff is also
> beta
> > >>>>> phase
> > >>>>> and hasn't really taken off yet, it also seems like there will be
> > >>>>> extensive
> > >>>>> testing after this phase transition.
> > >>>>>
> > >>>>> All that being said, I'd advocate for marking FixVer 4.x to
> > indicate
> > >>>>> optionality and disallow merge of tickets like this after we're
> > done
> > >>>>> w/alpha phase in keeping w/our lifecycle doc in general.
> > >>>>>
> > >>>>> Does that make sense? Should we consider revisiting and revising
> > the
> > >>>>> lifecycle doc re: larger deprecation / changes and cycle stages?
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>> On Tue, May 26, 2020 at 12:53 PM Oleksandr Petrov <
> > >>>>> oleksandr.petrov@gmail.com> wrote:
> > >>>>>
> > >>>>>>> 1) Would you block the release over this ticket?
> > >>>>>>
> > >>>>>> I would definitely not block the release on this ticket.
> > >>>>>>
> > >>>>>>> 2) Would you prioritize this ticket over testing?
> > >>>>>>
> > >>>>>> Same here, I would prioritise testing.
> > >>>>>>
> > >>>>>>> 3) Does fixing this ticket make 4.0 a more stable release?
> > >>>>>>
> > >>>>>> I wanted to give some context: I wrote that in August 2018. While
> I
> > >>>>> still
> > >>>>>> believe it is important to get rid of this code, I'm disinclined
> to
> > >>>>> merge
> > >>>>>> it into 4.0.
> > >>>>>>
> > >>>>>> Given that the patch is rather big (421 additions and 1,480
> > >>>>> deletions) and
> > >>>>>> touches many important places, including parser, I would be
> > >>>> extremely
> > >>>>>> cautious to merge it that late in release cycle. It would be great
> > >>>>> to also
> > >>>>>> hear arguments that would justify the risk.
> > >>>>>>
> > >>>>>> Thank you for starting this discussion,
> > >>>>>> -- Alex
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>> On Tue, May 26, 2020 at 5:20 PM Ekaterina Dimitrova <
> > >>>>>> ekaterina.dimitrova@datastax.com> wrote:
> > >>>>>>
> > >>>>>>> Dear all,
> > >>>>>>>
> > >>>>>>> Following the ticket review sent on 12th May I wanted to bring up
> > >>>>>>> https://issues.apache.org/jira/browse/CASSANDRA-13994: Remove
> > >>>>> COMPACT
> > >>>>>>>
> > >>>>>>> STORAGE internals before 4.0 release.
> > >>>>>>>
> > >>>>>>> It is already under review by Dinesh Joshi and Alex Petrov. Not a
> > >>>>>>> blocker but already under review.
> > >>>>>>>
> > >>>>>>> Below are my responses to the questions brought up.
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> 1) Would you block the release over this
> > >>>>>>>
> > >>>>>>> ticket? - probably not
> > >>>>>>>
> > >>>>>>> 2) Would you prioritize this ticket over testing? - already
> > >>>>>>> implemented but if there are some big changes needed after the
> > >>>>> review,
> > >>>>>>> I doubt it we will want to prioritize over the testing
> > >>>>>>>
> > >>>>>>> 3) Does fixing
> > >>>>>>> this ticket make 4.0 a more stable release? - I will just cite
> > >>>> Alex
> > >>>>>>> Petrov who reported this Jira and I think the rest of us would
> > >>>>> agree
> > >>>>>>> with him here.
> > >>>>>>>
> > >>>>>>> "I would say it's quite important to clean up compact storage
> > >>>>>>> internals in 4.0 before the release. It should have no visible
> > >>>>>>> side-effects, but it'd be very good to have as it simplifies
> > >>>>> multiple
> > >>>>>>> code paths."
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> Ekaterina Dimitrova
> > >>>>>>> e. ekaterina.dimitrova@datastax.com
> > >>>>>>> w. www.datastax.com
> > >>>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>> --
> > >>>>>> alex p
> > >>>>>>
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>>
> ---------------------------------------------------------------------
> > >>>>> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> > >>>>> For additional commands, e-mail: dev-help@cassandra.apache.org
> > >>>>>
> > >>>>>
> > >>>>
> > >>>
> > >>>
> > >>>
> > >>>
> ---------------------------------------------------------------------
> > >>> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> > >>> For additional commands, e-mail: dev-help@cassandra.apache.org
> > >>>
> > >>>
> > >>>
> > >>>
> > >>> ---------------------------------------------------------------------
> > >>> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> > >>> For additional commands, e-mail: dev-help@cassandra.apache.org
> > >>>
> > >>
> > >>
> > >> ---------------------------------------------------------------------
> > >> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> > >> For additional commands, e-mail: dev-help@cassandra.apache.org
> > >>
> > >>
> > >> ---------------------------------------------------------------------
> > >> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> > >> For additional commands, e-mail: dev-help@cassandra.apache.org
> > >>
> > >>
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> > For additional commands, e-mail: dev-help@cassandra.apache.org
> >
> >
>
Re: [DISCUSS] CASSANDRA-13994
Posted by Brandon Williams <dr...@gmail.com>.
Absolutely my understanding.
On Wed, May 27, 2020, 2:49 PM Jeremiah D Jordan <je...@gmail.com>
wrote:
> > A clear point to cut RC's doesn't surface from the above for me.
> Releasing
> > an RC before broad verification seems wrong, and cutting an RC after the
> 4
> > points above may as well be GA because it's all known scope.
>
> Isn’t the whole point of an RC is that it could be the GA? It is a
> “release candidate”, meaning if no one finds any issues with it, that can
> them become the release? So that seems like exactly the right time to make
> RC releases?
>
> > On May 27, 2020, at 2:45 PM, Joshua McKenzie <jm...@apache.org>
> wrote:
> >
> > I think we're all on the same page here; I was focusing more on the
> release
> > lifecycles and sequencing than the entire version cycle. Good to broaden
> > scope I think.
> >
> > One thing we're not considering is the separation of API changes from
> major
> > changes and how that intersects with release milestones.
> >
> > Meaning:
> > 1. alpha phase
> > 2. Milestone: API freeze (all API changes pushed to next major)
> > 3. beta phase
> > 4. Verification phase (all major disruptive pushed to next major)
> >
> > A clear point to cut RC's doesn't surface from the above for me.
> Releasing
> > an RC before broad verification seems wrong, and cutting an RC after the
> 4
> > points above may as well be GA because it's all known scope.
> >
> > Thoughts?
> >
> > On Wed, May 27, 2020 at 3:28 PM Scott Andreas <sc...@paradoxica.net>
> wrote:
> >
> >> That makes sense to me, yep.
> >>
> >> My hope and expectation is that the time required for "verification
> work"
> >> will shrink dramatically in the not too distant future - ideally to a
> >> period of less than a month. In this world, the cost of missing one
> train
> >> is reduced to catching the next one.
> >>
> >> One of the main goals in shifting focus from "testing" and "test plans"
> to
> >> "test engineering" is automating as many aspects of release
> qualification
> >> as possible, with an asymptotic ideal as a function of compute capacity
> and
> >> time. While such automation will never be complete (it's likely that
> >> development of new features will/must include qualification infra
> changes
> >> to exercise them), if we're able to apply the same rigor to major
> releases
> >> as we are to patchlevel builds with little incremental effort, I'd be
> >> thrilled.
> >>
> >> This is mostly a way of saying:
> >> – I like the cadence/sequencing Benedict proposes below.
> >> – I think improvements in test engineering can reduce/eliminate
> >> invalidation and may increase the scope of what can be a candidate for
> >> merge on a given branch
> >> – And if not, the cost of missing the train is lower because we'll be
> able
> >> to deliver major releases more often.
> >>
> >> Scott
> >>
> >> ________________________________________
> >> From: Jeremiah D Jordan <je...@gmail.com>
> >> Sent: Wednesday, May 27, 2020 11:54 AM
> >> To: Cassandra DEV
> >> Subject: Re: [DISCUSS] CASSANDRA-13994
> >>
> >> +1 strongly agree. If we aren’t going to let something go into 4.0.0
> >> because it would "invalidate testing” then we can not let such a thing
> go
> >> into 4.0.1 unless we plan to re-do said testing for the patch release.
> >>
> >>> On May 27, 2020, at 1:31 PM, Benedict Elliott Smith <
> benedict@apache.org>
> >> wrote:
> >>>
> >>> I'm being told this still isn't clear, so let me try in a bullet-point
> >> timeline:
> >>>
> >>> * 4.0 Beta
> >>> * 4.0 Verification Work
> >>> * [Merge Window]
> >>> * 4.0 GA
> >>> * 4.0 Minor Releases
> >>> * ...
> >>> * 5.0 Dev
> >>> * ...
> >>> * 5.0 Verification Work
> >>> * GA 5.0
> >>>
> >>> I think that anything that is prohibited from "[Merge Window]" because
> >> it invalidates "4.0 Verification Work" must also be prohibited until
> "5.0
> >> Dev" because the next equivalent work that can now validate it occurs
> only
> >> at "5.0 Verification Work"
> >>>
> >>> On 27/05/2020, 19:05, "Benedict Elliott Smith" <be...@apache.org>
> >> wrote:
> >>>
> >>> I'm not sure if I communicated my point very well. I mean to say
> >> that if the reason we are prohibiting a patch to land post-beta is
> because
> >> it invalidates work we only perform pre-ga, then it probably should not
> be
> >> permitted to land post-ga either, since it must also invalidate the same
> >> work?
> >>>
> >>> That is to say, if we're comfortable with work landing post-ga
> >> because we believe it to be safe to release without our
> pre-major-release
> >> verification, we should be comfortable with it landing at any time
> pre-ga
> >> too. Anything else seems inconsistent to me, and we should examine what
> >> assumptions we're making that permit this inconsistency to arise.
> >>>
> >>>
> >>> On 27/05/2020, 18:49, "Joshua McKenzie" <jm...@apache.org>
> >> wrote:
> >>>
> >>>>
> >>>> because it invalidates our pre-release verification, then it should
> not
> >>>> land
> >>>
> >>> until we next perform pre-release verification
> >>>
> >>> At least for me there's a little softness around our collective
> >> alignment
> >>> on when pre-release verification takes place. If it's between
> >> alpha-1 and
> >>> ga we don't want changes that would invalidate those changes to
> >> land during
> >>> that time frame. Different for beta-1 to ga. We also risk
> >> invalidating
> >>> testing if we do any of that testing before wherever that cutoff
> >> is, and a
> >>> lack of clarity on that cutoff further muddies those waters.
> >>>
> >>> My very loosely held perspective is that beta-1 to ga is the
> >> window in
> >>> which we apply the "don't do things that will invalidate
> >> verification", and
> >>> we plan to do that verification during the beta phase. I *think*
> >> this is
> >>> consistent w/the current framing of the lifecycle doc. That being
> >> said, I
> >>> don't have strong religion on this so if we collectively want to
> >> call it
> >>> "don't majorly disrupt from alpha-1 to ga", we can formalize that
> >> in the
> >>> docs and go ahead and triage current open scope for 4.0 and move
> >> things out.
> >>>
> >>>
> >>>
> >>> On Wed, May 27, 2020 at 12:59 PM Ekaterina Dimitrova <
> >>> ekaterina.dimitrova@datastax.com> wrote:
> >>>
> >>>> Thank you all for your input.
> >>>> I think an important topic is again to revise the lifecycle and ensure
> >> we
> >>>> really have the vision on what is left until beta. I will start a
> >> separate
> >>>> thread on the flaky tests situation soon.
> >>>>
> >>>> For this particular ticket I see a couple of things:
> >>>> - There are a lot of deletions of already not used code
> >>>> - I implemented it still in alpha as per our agreement that this will
> >> give
> >>>> us enough time for testing. Probably Dinesh as a reviewer can give
> some
> >>>> valuable feedback/opinion on the patch.
> >>>> - It definitely touches around important places but the important
> thing
> >> is
> >>>> to see how exactly it touches, I think
> >>>> - Considering it for alpha before the major testing in beta sounds
> >>>> reasonable to me but I guess it also depends on people availability to
> >>>> review it in detail and the exact test plans afterwards
> >>>>
> >>>> On Wed, 27 May 2020 at 7:14, Benedict Elliott Smith <
> >> benedict@apache.org>
> >>>> wrote:
> >>>>
> >>>>> I think our "pre-beta" criteria should also be our "not in a major"
> >>>>> criteria.
> >>>>>
> >>>>> If work is prohibited because it invalidates our pre-release
> >>>> verification,
> >>>>> then it should not land until we next perform pre-release
> verification,
> >>>>> which only currently happens once per major.
> >>>>>
> >>>>> This could mean either landing less in a major, or permitting more in
> >>>> beta
> >>>>> etc.
> >>>>>
> >>>>> On 26/05/2020, 19:24, "Joshua McKenzie" <jm...@apache.org>
> wrote:
> >>>>>
> >>>>> I think an interesting question that informs when to stop accepting
> >>>>> specific changes in a release is when we expect any extensive
> >>>>> pre-release
> >>>>> testing to take place.
> >>>>>
> >>>>> If we go by our release lifecycle, gutting deprecated code seems
> >>>>> compatible
> >>>>> w/Alpha but I wouldn't endorse merging it into Beta:
> >>>>>
> >>>>>
> >> https://cwiki.apache.org/confluence/display/CASSANDRA/Release+Lifecycle
> .
> >>>>> Since almost all of the 40_quality_testing epic stuff is also beta
> >>>>> phase
> >>>>> and hasn't really taken off yet, it also seems like there will be
> >>>>> extensive
> >>>>> testing after this phase transition.
> >>>>>
> >>>>> All that being said, I'd advocate for marking FixVer 4.x to
> indicate
> >>>>> optionality and disallow merge of tickets like this after we're
> done
> >>>>> w/alpha phase in keeping w/our lifecycle doc in general.
> >>>>>
> >>>>> Does that make sense? Should we consider revisiting and revising
> the
> >>>>> lifecycle doc re: larger deprecation / changes and cycle stages?
> >>>>>
> >>>>>
> >>>>>
> >>>>> On Tue, May 26, 2020 at 12:53 PM Oleksandr Petrov <
> >>>>> oleksandr.petrov@gmail.com> wrote:
> >>>>>
> >>>>>>> 1) Would you block the release over this ticket?
> >>>>>>
> >>>>>> I would definitely not block the release on this ticket.
> >>>>>>
> >>>>>>> 2) Would you prioritize this ticket over testing?
> >>>>>>
> >>>>>> Same here, I would prioritise testing.
> >>>>>>
> >>>>>>> 3) Does fixing this ticket make 4.0 a more stable release?
> >>>>>>
> >>>>>> I wanted to give some context: I wrote that in August 2018. While I
> >>>>> still
> >>>>>> believe it is important to get rid of this code, I'm disinclined to
> >>>>> merge
> >>>>>> it into 4.0.
> >>>>>>
> >>>>>> Given that the patch is rather big (421 additions and 1,480
> >>>>> deletions) and
> >>>>>> touches many important places, including parser, I would be
> >>>> extremely
> >>>>>> cautious to merge it that late in release cycle. It would be great
> >>>>> to also
> >>>>>> hear arguments that would justify the risk.
> >>>>>>
> >>>>>> Thank you for starting this discussion,
> >>>>>> -- Alex
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> On Tue, May 26, 2020 at 5:20 PM Ekaterina Dimitrova <
> >>>>>> ekaterina.dimitrova@datastax.com> wrote:
> >>>>>>
> >>>>>>> Dear all,
> >>>>>>>
> >>>>>>> Following the ticket review sent on 12th May I wanted to bring up
> >>>>>>> https://issues.apache.org/jira/browse/CASSANDRA-13994: Remove
> >>>>> COMPACT
> >>>>>>>
> >>>>>>> STORAGE internals before 4.0 release.
> >>>>>>>
> >>>>>>> It is already under review by Dinesh Joshi and Alex Petrov. Not a
> >>>>>>> blocker but already under review.
> >>>>>>>
> >>>>>>> Below are my responses to the questions brought up.
> >>>>>>>
> >>>>>>>
> >>>>>>> 1) Would you block the release over this
> >>>>>>>
> >>>>>>> ticket? - probably not
> >>>>>>>
> >>>>>>> 2) Would you prioritize this ticket over testing? - already
> >>>>>>> implemented but if there are some big changes needed after the
> >>>>> review,
> >>>>>>> I doubt it we will want to prioritize over the testing
> >>>>>>>
> >>>>>>> 3) Does fixing
> >>>>>>> this ticket make 4.0 a more stable release? - I will just cite
> >>>> Alex
> >>>>>>> Petrov who reported this Jira and I think the rest of us would
> >>>>> agree
> >>>>>>> with him here.
> >>>>>>>
> >>>>>>> "I would say it's quite important to clean up compact storage
> >>>>>>> internals in 4.0 before the release. It should have no visible
> >>>>>>> side-effects, but it'd be very good to have as it simplifies
> >>>>> multiple
> >>>>>>> code paths."
> >>>>>>>
> >>>>>>>
> >>>>>>> Ekaterina Dimitrova
> >>>>>>> e. ekaterina.dimitrova@datastax.com
> >>>>>>> w. www.datastax.com
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> alex p
> >>>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> ---------------------------------------------------------------------
> >>>>> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> >>>>> For additional commands, e-mail: dev-help@cassandra.apache.org
> >>>>>
> >>>>>
> >>>>
> >>>
> >>>
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> >>> For additional commands, e-mail: dev-help@cassandra.apache.org
> >>>
> >>>
> >>>
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> >>> For additional commands, e-mail: dev-help@cassandra.apache.org
> >>>
> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> >> For additional commands, e-mail: dev-help@cassandra.apache.org
> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> >> For additional commands, e-mail: dev-help@cassandra.apache.org
> >>
> >>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> For additional commands, e-mail: dev-help@cassandra.apache.org
>
>
Re: [DISCUSS] CASSANDRA-13994
Posted by Jeremiah D Jordan <je...@gmail.com>.
> A clear point to cut RC's doesn't surface from the above for me. Releasing
> an RC before broad verification seems wrong, and cutting an RC after the 4
> points above may as well be GA because it's all known scope.
Isn’t the whole point of an RC is that it could be the GA? It is a “release candidate”, meaning if no one finds any issues with it, that can them become the release? So that seems like exactly the right time to make RC releases?
> On May 27, 2020, at 2:45 PM, Joshua McKenzie <jm...@apache.org> wrote:
>
> I think we're all on the same page here; I was focusing more on the release
> lifecycles and sequencing than the entire version cycle. Good to broaden
> scope I think.
>
> One thing we're not considering is the separation of API changes from major
> changes and how that intersects with release milestones.
>
> Meaning:
> 1. alpha phase
> 2. Milestone: API freeze (all API changes pushed to next major)
> 3. beta phase
> 4. Verification phase (all major disruptive pushed to next major)
>
> A clear point to cut RC's doesn't surface from the above for me. Releasing
> an RC before broad verification seems wrong, and cutting an RC after the 4
> points above may as well be GA because it's all known scope.
>
> Thoughts?
>
> On Wed, May 27, 2020 at 3:28 PM Scott Andreas <sc...@paradoxica.net> wrote:
>
>> That makes sense to me, yep.
>>
>> My hope and expectation is that the time required for "verification work"
>> will shrink dramatically in the not too distant future - ideally to a
>> period of less than a month. In this world, the cost of missing one train
>> is reduced to catching the next one.
>>
>> One of the main goals in shifting focus from "testing" and "test plans" to
>> "test engineering" is automating as many aspects of release qualification
>> as possible, with an asymptotic ideal as a function of compute capacity and
>> time. While such automation will never be complete (it's likely that
>> development of new features will/must include qualification infra changes
>> to exercise them), if we're able to apply the same rigor to major releases
>> as we are to patchlevel builds with little incremental effort, I'd be
>> thrilled.
>>
>> This is mostly a way of saying:
>> – I like the cadence/sequencing Benedict proposes below.
>> – I think improvements in test engineering can reduce/eliminate
>> invalidation and may increase the scope of what can be a candidate for
>> merge on a given branch
>> – And if not, the cost of missing the train is lower because we'll be able
>> to deliver major releases more often.
>>
>> Scott
>>
>> ________________________________________
>> From: Jeremiah D Jordan <je...@gmail.com>
>> Sent: Wednesday, May 27, 2020 11:54 AM
>> To: Cassandra DEV
>> Subject: Re: [DISCUSS] CASSANDRA-13994
>>
>> +1 strongly agree. If we aren’t going to let something go into 4.0.0
>> because it would "invalidate testing” then we can not let such a thing go
>> into 4.0.1 unless we plan to re-do said testing for the patch release.
>>
>>> On May 27, 2020, at 1:31 PM, Benedict Elliott Smith <be...@apache.org>
>> wrote:
>>>
>>> I'm being told this still isn't clear, so let me try in a bullet-point
>> timeline:
>>>
>>> * 4.0 Beta
>>> * 4.0 Verification Work
>>> * [Merge Window]
>>> * 4.0 GA
>>> * 4.0 Minor Releases
>>> * ...
>>> * 5.0 Dev
>>> * ...
>>> * 5.0 Verification Work
>>> * GA 5.0
>>>
>>> I think that anything that is prohibited from "[Merge Window]" because
>> it invalidates "4.0 Verification Work" must also be prohibited until "5.0
>> Dev" because the next equivalent work that can now validate it occurs only
>> at "5.0 Verification Work"
>>>
>>> On 27/05/2020, 19:05, "Benedict Elliott Smith" <be...@apache.org>
>> wrote:
>>>
>>> I'm not sure if I communicated my point very well. I mean to say
>> that if the reason we are prohibiting a patch to land post-beta is because
>> it invalidates work we only perform pre-ga, then it probably should not be
>> permitted to land post-ga either, since it must also invalidate the same
>> work?
>>>
>>> That is to say, if we're comfortable with work landing post-ga
>> because we believe it to be safe to release without our pre-major-release
>> verification, we should be comfortable with it landing at any time pre-ga
>> too. Anything else seems inconsistent to me, and we should examine what
>> assumptions we're making that permit this inconsistency to arise.
>>>
>>>
>>> On 27/05/2020, 18:49, "Joshua McKenzie" <jm...@apache.org>
>> wrote:
>>>
>>>>
>>>> because it invalidates our pre-release verification, then it should not
>>>> land
>>>
>>> until we next perform pre-release verification
>>>
>>> At least for me there's a little softness around our collective
>> alignment
>>> on when pre-release verification takes place. If it's between
>> alpha-1 and
>>> ga we don't want changes that would invalidate those changes to
>> land during
>>> that time frame. Different for beta-1 to ga. We also risk
>> invalidating
>>> testing if we do any of that testing before wherever that cutoff
>> is, and a
>>> lack of clarity on that cutoff further muddies those waters.
>>>
>>> My very loosely held perspective is that beta-1 to ga is the
>> window in
>>> which we apply the "don't do things that will invalidate
>> verification", and
>>> we plan to do that verification during the beta phase. I *think*
>> this is
>>> consistent w/the current framing of the lifecycle doc. That being
>> said, I
>>> don't have strong religion on this so if we collectively want to
>> call it
>>> "don't majorly disrupt from alpha-1 to ga", we can formalize that
>> in the
>>> docs and go ahead and triage current open scope for 4.0 and move
>> things out.
>>>
>>>
>>>
>>> On Wed, May 27, 2020 at 12:59 PM Ekaterina Dimitrova <
>>> ekaterina.dimitrova@datastax.com> wrote:
>>>
>>>> Thank you all for your input.
>>>> I think an important topic is again to revise the lifecycle and ensure
>> we
>>>> really have the vision on what is left until beta. I will start a
>> separate
>>>> thread on the flaky tests situation soon.
>>>>
>>>> For this particular ticket I see a couple of things:
>>>> - There are a lot of deletions of already not used code
>>>> - I implemented it still in alpha as per our agreement that this will
>> give
>>>> us enough time for testing. Probably Dinesh as a reviewer can give some
>>>> valuable feedback/opinion on the patch.
>>>> - It definitely touches around important places but the important thing
>> is
>>>> to see how exactly it touches, I think
>>>> - Considering it for alpha before the major testing in beta sounds
>>>> reasonable to me but I guess it also depends on people availability to
>>>> review it in detail and the exact test plans afterwards
>>>>
>>>> On Wed, 27 May 2020 at 7:14, Benedict Elliott Smith <
>> benedict@apache.org>
>>>> wrote:
>>>>
>>>>> I think our "pre-beta" criteria should also be our "not in a major"
>>>>> criteria.
>>>>>
>>>>> If work is prohibited because it invalidates our pre-release
>>>> verification,
>>>>> then it should not land until we next perform pre-release verification,
>>>>> which only currently happens once per major.
>>>>>
>>>>> This could mean either landing less in a major, or permitting more in
>>>> beta
>>>>> etc.
>>>>>
>>>>> On 26/05/2020, 19:24, "Joshua McKenzie" <jm...@apache.org> wrote:
>>>>>
>>>>> I think an interesting question that informs when to stop accepting
>>>>> specific changes in a release is when we expect any extensive
>>>>> pre-release
>>>>> testing to take place.
>>>>>
>>>>> If we go by our release lifecycle, gutting deprecated code seems
>>>>> compatible
>>>>> w/Alpha but I wouldn't endorse merging it into Beta:
>>>>>
>>>>>
>> https://cwiki.apache.org/confluence/display/CASSANDRA/Release+Lifecycle.
>>>>> Since almost all of the 40_quality_testing epic stuff is also beta
>>>>> phase
>>>>> and hasn't really taken off yet, it also seems like there will be
>>>>> extensive
>>>>> testing after this phase transition.
>>>>>
>>>>> All that being said, I'd advocate for marking FixVer 4.x to indicate
>>>>> optionality and disallow merge of tickets like this after we're done
>>>>> w/alpha phase in keeping w/our lifecycle doc in general.
>>>>>
>>>>> Does that make sense? Should we consider revisiting and revising the
>>>>> lifecycle doc re: larger deprecation / changes and cycle stages?
>>>>>
>>>>>
>>>>>
>>>>> On Tue, May 26, 2020 at 12:53 PM Oleksandr Petrov <
>>>>> oleksandr.petrov@gmail.com> wrote:
>>>>>
>>>>>>> 1) Would you block the release over this ticket?
>>>>>>
>>>>>> I would definitely not block the release on this ticket.
>>>>>>
>>>>>>> 2) Would you prioritize this ticket over testing?
>>>>>>
>>>>>> Same here, I would prioritise testing.
>>>>>>
>>>>>>> 3) Does fixing this ticket make 4.0 a more stable release?
>>>>>>
>>>>>> I wanted to give some context: I wrote that in August 2018. While I
>>>>> still
>>>>>> believe it is important to get rid of this code, I'm disinclined to
>>>>> merge
>>>>>> it into 4.0.
>>>>>>
>>>>>> Given that the patch is rather big (421 additions and 1,480
>>>>> deletions) and
>>>>>> touches many important places, including parser, I would be
>>>> extremely
>>>>>> cautious to merge it that late in release cycle. It would be great
>>>>> to also
>>>>>> hear arguments that would justify the risk.
>>>>>>
>>>>>> Thank you for starting this discussion,
>>>>>> -- Alex
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Tue, May 26, 2020 at 5:20 PM Ekaterina Dimitrova <
>>>>>> ekaterina.dimitrova@datastax.com> wrote:
>>>>>>
>>>>>>> Dear all,
>>>>>>>
>>>>>>> Following the ticket review sent on 12th May I wanted to bring up
>>>>>>> https://issues.apache.org/jira/browse/CASSANDRA-13994: Remove
>>>>> COMPACT
>>>>>>>
>>>>>>> STORAGE internals before 4.0 release.
>>>>>>>
>>>>>>> It is already under review by Dinesh Joshi and Alex Petrov. Not a
>>>>>>> blocker but already under review.
>>>>>>>
>>>>>>> Below are my responses to the questions brought up.
>>>>>>>
>>>>>>>
>>>>>>> 1) Would you block the release over this
>>>>>>>
>>>>>>> ticket? - probably not
>>>>>>>
>>>>>>> 2) Would you prioritize this ticket over testing? - already
>>>>>>> implemented but if there are some big changes needed after the
>>>>> review,
>>>>>>> I doubt it we will want to prioritize over the testing
>>>>>>>
>>>>>>> 3) Does fixing
>>>>>>> this ticket make 4.0 a more stable release? - I will just cite
>>>> Alex
>>>>>>> Petrov who reported this Jira and I think the rest of us would
>>>>> agree
>>>>>>> with him here.
>>>>>>>
>>>>>>> "I would say it's quite important to clean up compact storage
>>>>>>> internals in 4.0 before the release. It should have no visible
>>>>>>> side-effects, but it'd be very good to have as it simplifies
>>>>> multiple
>>>>>>> code paths."
>>>>>>>
>>>>>>>
>>>>>>> Ekaterina Dimitrova
>>>>>>> e. ekaterina.dimitrova@datastax.com
>>>>>>> w. www.datastax.com
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> alex p
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
>>>>> For additional commands, e-mail: dev-help@cassandra.apache.org
>>>>>
>>>>>
>>>>
>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
>>> For additional commands, e-mail: dev-help@cassandra.apache.org
>>>
>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
>>> For additional commands, e-mail: dev-help@cassandra.apache.org
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
>> For additional commands, e-mail: dev-help@cassandra.apache.org
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
>> For additional commands, e-mail: dev-help@cassandra.apache.org
>>
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
For additional commands, e-mail: dev-help@cassandra.apache.org
Re: [DISCUSS] CASSANDRA-13994
Posted by Joshua McKenzie <jm...@apache.org>.
I think we're all on the same page here; I was focusing more on the release
lifecycles and sequencing than the entire version cycle. Good to broaden
scope I think.
One thing we're not considering is the separation of API changes from major
changes and how that intersects with release milestones.
Meaning:
1. alpha phase
2. Milestone: API freeze (all API changes pushed to next major)
3. beta phase
4. Verification phase (all major disruptive pushed to next major)
A clear point to cut RC's doesn't surface from the above for me. Releasing
an RC before broad verification seems wrong, and cutting an RC after the 4
points above may as well be GA because it's all known scope.
Thoughts?
On Wed, May 27, 2020 at 3:28 PM Scott Andreas <sc...@paradoxica.net> wrote:
> That makes sense to me, yep.
>
> My hope and expectation is that the time required for "verification work"
> will shrink dramatically in the not too distant future - ideally to a
> period of less than a month. In this world, the cost of missing one train
> is reduced to catching the next one.
>
> One of the main goals in shifting focus from "testing" and "test plans" to
> "test engineering" is automating as many aspects of release qualification
> as possible, with an asymptotic ideal as a function of compute capacity and
> time. While such automation will never be complete (it's likely that
> development of new features will/must include qualification infra changes
> to exercise them), if we're able to apply the same rigor to major releases
> as we are to patchlevel builds with little incremental effort, I'd be
> thrilled.
>
> This is mostly a way of saying:
> – I like the cadence/sequencing Benedict proposes below.
> – I think improvements in test engineering can reduce/eliminate
> invalidation and may increase the scope of what can be a candidate for
> merge on a given branch
> – And if not, the cost of missing the train is lower because we'll be able
> to deliver major releases more often.
>
> Scott
>
> ________________________________________
> From: Jeremiah D Jordan <je...@gmail.com>
> Sent: Wednesday, May 27, 2020 11:54 AM
> To: Cassandra DEV
> Subject: Re: [DISCUSS] CASSANDRA-13994
>
> +1 strongly agree. If we aren’t going to let something go into 4.0.0
> because it would "invalidate testing” then we can not let such a thing go
> into 4.0.1 unless we plan to re-do said testing for the patch release.
>
> > On May 27, 2020, at 1:31 PM, Benedict Elliott Smith <be...@apache.org>
> wrote:
> >
> > I'm being told this still isn't clear, so let me try in a bullet-point
> timeline:
> >
> > * 4.0 Beta
> > * 4.0 Verification Work
> > * [Merge Window]
> > * 4.0 GA
> > * 4.0 Minor Releases
> > * ...
> > * 5.0 Dev
> > * ...
> > * 5.0 Verification Work
> > * GA 5.0
> >
> > I think that anything that is prohibited from "[Merge Window]" because
> it invalidates "4.0 Verification Work" must also be prohibited until "5.0
> Dev" because the next equivalent work that can now validate it occurs only
> at "5.0 Verification Work"
> >
> > On 27/05/2020, 19:05, "Benedict Elliott Smith" <be...@apache.org>
> wrote:
> >
> > I'm not sure if I communicated my point very well. I mean to say
> that if the reason we are prohibiting a patch to land post-beta is because
> it invalidates work we only perform pre-ga, then it probably should not be
> permitted to land post-ga either, since it must also invalidate the same
> work?
> >
> > That is to say, if we're comfortable with work landing post-ga
> because we believe it to be safe to release without our pre-major-release
> verification, we should be comfortable with it landing at any time pre-ga
> too. Anything else seems inconsistent to me, and we should examine what
> assumptions we're making that permit this inconsistency to arise.
> >
> >
> > On 27/05/2020, 18:49, "Joshua McKenzie" <jm...@apache.org>
> wrote:
> >
> >>
> >> because it invalidates our pre-release verification, then it should not
> >> land
> >
> > until we next perform pre-release verification
> >
> > At least for me there's a little softness around our collective
> alignment
> > on when pre-release verification takes place. If it's between
> alpha-1 and
> > ga we don't want changes that would invalidate those changes to
> land during
> > that time frame. Different for beta-1 to ga. We also risk
> invalidating
> > testing if we do any of that testing before wherever that cutoff
> is, and a
> > lack of clarity on that cutoff further muddies those waters.
> >
> > My very loosely held perspective is that beta-1 to ga is the
> window in
> > which we apply the "don't do things that will invalidate
> verification", and
> > we plan to do that verification during the beta phase. I *think*
> this is
> > consistent w/the current framing of the lifecycle doc. That being
> said, I
> > don't have strong religion on this so if we collectively want to
> call it
> > "don't majorly disrupt from alpha-1 to ga", we can formalize that
> in the
> > docs and go ahead and triage current open scope for 4.0 and move
> things out.
> >
> >
> >
> > On Wed, May 27, 2020 at 12:59 PM Ekaterina Dimitrova <
> > ekaterina.dimitrova@datastax.com> wrote:
> >
> >> Thank you all for your input.
> >> I think an important topic is again to revise the lifecycle and ensure
> we
> >> really have the vision on what is left until beta. I will start a
> separate
> >> thread on the flaky tests situation soon.
> >>
> >> For this particular ticket I see a couple of things:
> >> - There are a lot of deletions of already not used code
> >> - I implemented it still in alpha as per our agreement that this will
> give
> >> us enough time for testing. Probably Dinesh as a reviewer can give some
> >> valuable feedback/opinion on the patch.
> >> - It definitely touches around important places but the important thing
> is
> >> to see how exactly it touches, I think
> >> - Considering it for alpha before the major testing in beta sounds
> >> reasonable to me but I guess it also depends on people availability to
> >> review it in detail and the exact test plans afterwards
> >>
> >> On Wed, 27 May 2020 at 7:14, Benedict Elliott Smith <
> benedict@apache.org>
> >> wrote:
> >>
> >>> I think our "pre-beta" criteria should also be our "not in a major"
> >>> criteria.
> >>>
> >>> If work is prohibited because it invalidates our pre-release
> >> verification,
> >>> then it should not land until we next perform pre-release verification,
> >>> which only currently happens once per major.
> >>>
> >>> This could mean either landing less in a major, or permitting more in
> >> beta
> >>> etc.
> >>>
> >>> On 26/05/2020, 19:24, "Joshua McKenzie" <jm...@apache.org> wrote:
> >>>
> >>> I think an interesting question that informs when to stop accepting
> >>> specific changes in a release is when we expect any extensive
> >>> pre-release
> >>> testing to take place.
> >>>
> >>> If we go by our release lifecycle, gutting deprecated code seems
> >>> compatible
> >>> w/Alpha but I wouldn't endorse merging it into Beta:
> >>>
> >>>
> https://cwiki.apache.org/confluence/display/CASSANDRA/Release+Lifecycle.
> >>> Since almost all of the 40_quality_testing epic stuff is also beta
> >>> phase
> >>> and hasn't really taken off yet, it also seems like there will be
> >>> extensive
> >>> testing after this phase transition.
> >>>
> >>> All that being said, I'd advocate for marking FixVer 4.x to indicate
> >>> optionality and disallow merge of tickets like this after we're done
> >>> w/alpha phase in keeping w/our lifecycle doc in general.
> >>>
> >>> Does that make sense? Should we consider revisiting and revising the
> >>> lifecycle doc re: larger deprecation / changes and cycle stages?
> >>>
> >>>
> >>>
> >>> On Tue, May 26, 2020 at 12:53 PM Oleksandr Petrov <
> >>> oleksandr.petrov@gmail.com> wrote:
> >>>
> >>>>> 1) Would you block the release over this ticket?
> >>>>
> >>>> I would definitely not block the release on this ticket.
> >>>>
> >>>>> 2) Would you prioritize this ticket over testing?
> >>>>
> >>>> Same here, I would prioritise testing.
> >>>>
> >>>>> 3) Does fixing this ticket make 4.0 a more stable release?
> >>>>
> >>>> I wanted to give some context: I wrote that in August 2018. While I
> >>> still
> >>>> believe it is important to get rid of this code, I'm disinclined to
> >>> merge
> >>>> it into 4.0.
> >>>>
> >>>> Given that the patch is rather big (421 additions and 1,480
> >>> deletions) and
> >>>> touches many important places, including parser, I would be
> >> extremely
> >>>> cautious to merge it that late in release cycle. It would be great
> >>> to also
> >>>> hear arguments that would justify the risk.
> >>>>
> >>>> Thank you for starting this discussion,
> >>>> -- Alex
> >>>>
> >>>>
> >>>>
> >>>> On Tue, May 26, 2020 at 5:20 PM Ekaterina Dimitrova <
> >>>> ekaterina.dimitrova@datastax.com> wrote:
> >>>>
> >>>>> Dear all,
> >>>>>
> >>>>> Following the ticket review sent on 12th May I wanted to bring up
> >>>>> https://issues.apache.org/jira/browse/CASSANDRA-13994: Remove
> >>> COMPACT
> >>>>>
> >>>>> STORAGE internals before 4.0 release.
> >>>>>
> >>>>> It is already under review by Dinesh Joshi and Alex Petrov. Not a
> >>>>> blocker but already under review.
> >>>>>
> >>>>> Below are my responses to the questions brought up.
> >>>>>
> >>>>>
> >>>>> 1) Would you block the release over this
> >>>>>
> >>>>> ticket? - probably not
> >>>>>
> >>>>> 2) Would you prioritize this ticket over testing? - already
> >>>>> implemented but if there are some big changes needed after the
> >>> review,
> >>>>> I doubt it we will want to prioritize over the testing
> >>>>>
> >>>>> 3) Does fixing
> >>>>> this ticket make 4.0 a more stable release? - I will just cite
> >> Alex
> >>>>> Petrov who reported this Jira and I think the rest of us would
> >>> agree
> >>>>> with him here.
> >>>>>
> >>>>> "I would say it's quite important to clean up compact storage
> >>>>> internals in 4.0 before the release. It should have no visible
> >>>>> side-effects, but it'd be very good to have as it simplifies
> >>> multiple
> >>>>> code paths."
> >>>>>
> >>>>>
> >>>>> Ekaterina Dimitrova
> >>>>> e. ekaterina.dimitrova@datastax.com
> >>>>> w. www.datastax.com
> >>>>>
> >>>>
> >>>>
> >>>> --
> >>>> alex p
> >>>>
> >>>
> >>>
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> >>> For additional commands, e-mail: dev-help@cassandra.apache.org
> >>>
> >>>
> >>
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> > For additional commands, e-mail: dev-help@cassandra.apache.org
> >
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> > For additional commands, e-mail: dev-help@cassandra.apache.org
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> For additional commands, e-mail: dev-help@cassandra.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> For additional commands, e-mail: dev-help@cassandra.apache.org
>
>
Re: [DISCUSS] CASSANDRA-13994
Posted by Scott Andreas <sc...@paradoxica.net>.
That makes sense to me, yep.
My hope and expectation is that the time required for "verification work" will shrink dramatically in the not too distant future - ideally to a period of less than a month. In this world, the cost of missing one train is reduced to catching the next one.
One of the main goals in shifting focus from "testing" and "test plans" to "test engineering" is automating as many aspects of release qualification as possible, with an asymptotic ideal as a function of compute capacity and time. While such automation will never be complete (it's likely that development of new features will/must include qualification infra changes to exercise them), if we're able to apply the same rigor to major releases as we are to patchlevel builds with little incremental effort, I'd be thrilled.
This is mostly a way of saying:
– I like the cadence/sequencing Benedict proposes below.
– I think improvements in test engineering can reduce/eliminate invalidation and may increase the scope of what can be a candidate for merge on a given branch
– And if not, the cost of missing the train is lower because we'll be able to deliver major releases more often.
Scott
________________________________________
From: Jeremiah D Jordan <je...@gmail.com>
Sent: Wednesday, May 27, 2020 11:54 AM
To: Cassandra DEV
Subject: Re: [DISCUSS] CASSANDRA-13994
+1 strongly agree. If we aren’t going to let something go into 4.0.0 because it would "invalidate testing” then we can not let such a thing go into 4.0.1 unless we plan to re-do said testing for the patch release.
> On May 27, 2020, at 1:31 PM, Benedict Elliott Smith <be...@apache.org> wrote:
>
> I'm being told this still isn't clear, so let me try in a bullet-point timeline:
>
> * 4.0 Beta
> * 4.0 Verification Work
> * [Merge Window]
> * 4.0 GA
> * 4.0 Minor Releases
> * ...
> * 5.0 Dev
> * ...
> * 5.0 Verification Work
> * GA 5.0
>
> I think that anything that is prohibited from "[Merge Window]" because it invalidates "4.0 Verification Work" must also be prohibited until "5.0 Dev" because the next equivalent work that can now validate it occurs only at "5.0 Verification Work"
>
> On 27/05/2020, 19:05, "Benedict Elliott Smith" <be...@apache.org> wrote:
>
> I'm not sure if I communicated my point very well. I mean to say that if the reason we are prohibiting a patch to land post-beta is because it invalidates work we only perform pre-ga, then it probably should not be permitted to land post-ga either, since it must also invalidate the same work?
>
> That is to say, if we're comfortable with work landing post-ga because we believe it to be safe to release without our pre-major-release verification, we should be comfortable with it landing at any time pre-ga too. Anything else seems inconsistent to me, and we should examine what assumptions we're making that permit this inconsistency to arise.
>
>
> On 27/05/2020, 18:49, "Joshua McKenzie" <jm...@apache.org> wrote:
>
>>
>> because it invalidates our pre-release verification, then it should not
>> land
>
> until we next perform pre-release verification
>
> At least for me there's a little softness around our collective alignment
> on when pre-release verification takes place. If it's between alpha-1 and
> ga we don't want changes that would invalidate those changes to land during
> that time frame. Different for beta-1 to ga. We also risk invalidating
> testing if we do any of that testing before wherever that cutoff is, and a
> lack of clarity on that cutoff further muddies those waters.
>
> My very loosely held perspective is that beta-1 to ga is the window in
> which we apply the "don't do things that will invalidate verification", and
> we plan to do that verification during the beta phase. I *think* this is
> consistent w/the current framing of the lifecycle doc. That being said, I
> don't have strong religion on this so if we collectively want to call it
> "don't majorly disrupt from alpha-1 to ga", we can formalize that in the
> docs and go ahead and triage current open scope for 4.0 and move things out.
>
>
>
> On Wed, May 27, 2020 at 12:59 PM Ekaterina Dimitrova <
> ekaterina.dimitrova@datastax.com> wrote:
>
>> Thank you all for your input.
>> I think an important topic is again to revise the lifecycle and ensure we
>> really have the vision on what is left until beta. I will start a separate
>> thread on the flaky tests situation soon.
>>
>> For this particular ticket I see a couple of things:
>> - There are a lot of deletions of already not used code
>> - I implemented it still in alpha as per our agreement that this will give
>> us enough time for testing. Probably Dinesh as a reviewer can give some
>> valuable feedback/opinion on the patch.
>> - It definitely touches around important places but the important thing is
>> to see how exactly it touches, I think
>> - Considering it for alpha before the major testing in beta sounds
>> reasonable to me but I guess it also depends on people availability to
>> review it in detail and the exact test plans afterwards
>>
>> On Wed, 27 May 2020 at 7:14, Benedict Elliott Smith <be...@apache.org>
>> wrote:
>>
>>> I think our "pre-beta" criteria should also be our "not in a major"
>>> criteria.
>>>
>>> If work is prohibited because it invalidates our pre-release
>> verification,
>>> then it should not land until we next perform pre-release verification,
>>> which only currently happens once per major.
>>>
>>> This could mean either landing less in a major, or permitting more in
>> beta
>>> etc.
>>>
>>> On 26/05/2020, 19:24, "Joshua McKenzie" <jm...@apache.org> wrote:
>>>
>>> I think an interesting question that informs when to stop accepting
>>> specific changes in a release is when we expect any extensive
>>> pre-release
>>> testing to take place.
>>>
>>> If we go by our release lifecycle, gutting deprecated code seems
>>> compatible
>>> w/Alpha but I wouldn't endorse merging it into Beta:
>>>
>>> https://cwiki.apache.org/confluence/display/CASSANDRA/Release+Lifecycle.
>>> Since almost all of the 40_quality_testing epic stuff is also beta
>>> phase
>>> and hasn't really taken off yet, it also seems like there will be
>>> extensive
>>> testing after this phase transition.
>>>
>>> All that being said, I'd advocate for marking FixVer 4.x to indicate
>>> optionality and disallow merge of tickets like this after we're done
>>> w/alpha phase in keeping w/our lifecycle doc in general.
>>>
>>> Does that make sense? Should we consider revisiting and revising the
>>> lifecycle doc re: larger deprecation / changes and cycle stages?
>>>
>>>
>>>
>>> On Tue, May 26, 2020 at 12:53 PM Oleksandr Petrov <
>>> oleksandr.petrov@gmail.com> wrote:
>>>
>>>>> 1) Would you block the release over this ticket?
>>>>
>>>> I would definitely not block the release on this ticket.
>>>>
>>>>> 2) Would you prioritize this ticket over testing?
>>>>
>>>> Same here, I would prioritise testing.
>>>>
>>>>> 3) Does fixing this ticket make 4.0 a more stable release?
>>>>
>>>> I wanted to give some context: I wrote that in August 2018. While I
>>> still
>>>> believe it is important to get rid of this code, I'm disinclined to
>>> merge
>>>> it into 4.0.
>>>>
>>>> Given that the patch is rather big (421 additions and 1,480
>>> deletions) and
>>>> touches many important places, including parser, I would be
>> extremely
>>>> cautious to merge it that late in release cycle. It would be great
>>> to also
>>>> hear arguments that would justify the risk.
>>>>
>>>> Thank you for starting this discussion,
>>>> -- Alex
>>>>
>>>>
>>>>
>>>> On Tue, May 26, 2020 at 5:20 PM Ekaterina Dimitrova <
>>>> ekaterina.dimitrova@datastax.com> wrote:
>>>>
>>>>> Dear all,
>>>>>
>>>>> Following the ticket review sent on 12th May I wanted to bring up
>>>>> https://issues.apache.org/jira/browse/CASSANDRA-13994: Remove
>>> COMPACT
>>>>>
>>>>> STORAGE internals before 4.0 release.
>>>>>
>>>>> It is already under review by Dinesh Joshi and Alex Petrov. Not a
>>>>> blocker but already under review.
>>>>>
>>>>> Below are my responses to the questions brought up.
>>>>>
>>>>>
>>>>> 1) Would you block the release over this
>>>>>
>>>>> ticket? - probably not
>>>>>
>>>>> 2) Would you prioritize this ticket over testing? - already
>>>>> implemented but if there are some big changes needed after the
>>> review,
>>>>> I doubt it we will want to prioritize over the testing
>>>>>
>>>>> 3) Does fixing
>>>>> this ticket make 4.0 a more stable release? - I will just cite
>> Alex
>>>>> Petrov who reported this Jira and I think the rest of us would
>>> agree
>>>>> with him here.
>>>>>
>>>>> "I would say it's quite important to clean up compact storage
>>>>> internals in 4.0 before the release. It should have no visible
>>>>> side-effects, but it'd be very good to have as it simplifies
>>> multiple
>>>>> code paths."
>>>>>
>>>>>
>>>>> Ekaterina Dimitrova
>>>>> e. ekaterina.dimitrova@datastax.com
>>>>> w. www.datastax.com
>>>>>
>>>>
>>>>
>>>> --
>>>> alex p
>>>>
>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
>>> For additional commands, e-mail: dev-help@cassandra.apache.org
>>>
>>>
>>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> For additional commands, e-mail: dev-help@cassandra.apache.org
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> For additional commands, e-mail: dev-help@cassandra.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
For additional commands, e-mail: dev-help@cassandra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
For additional commands, e-mail: dev-help@cassandra.apache.org
Re: [DISCUSS] CASSANDRA-13994
Posted by Jeremiah D Jordan <je...@gmail.com>.
+1 strongly agree. If we aren’t going to let something go into 4.0.0 because it would "invalidate testing” then we can not let such a thing go into 4.0.1 unless we plan to re-do said testing for the patch release.
> On May 27, 2020, at 1:31 PM, Benedict Elliott Smith <be...@apache.org> wrote:
>
> I'm being told this still isn't clear, so let me try in a bullet-point timeline:
>
> * 4.0 Beta
> * 4.0 Verification Work
> * [Merge Window]
> * 4.0 GA
> * 4.0 Minor Releases
> * ...
> * 5.0 Dev
> * ...
> * 5.0 Verification Work
> * GA 5.0
>
> I think that anything that is prohibited from "[Merge Window]" because it invalidates "4.0 Verification Work" must also be prohibited until "5.0 Dev" because the next equivalent work that can now validate it occurs only at "5.0 Verification Work"
>
> On 27/05/2020, 19:05, "Benedict Elliott Smith" <be...@apache.org> wrote:
>
> I'm not sure if I communicated my point very well. I mean to say that if the reason we are prohibiting a patch to land post-beta is because it invalidates work we only perform pre-ga, then it probably should not be permitted to land post-ga either, since it must also invalidate the same work?
>
> That is to say, if we're comfortable with work landing post-ga because we believe it to be safe to release without our pre-major-release verification, we should be comfortable with it landing at any time pre-ga too. Anything else seems inconsistent to me, and we should examine what assumptions we're making that permit this inconsistency to arise.
>
>
> On 27/05/2020, 18:49, "Joshua McKenzie" <jm...@apache.org> wrote:
>
>>
>> because it invalidates our pre-release verification, then it should not
>> land
>
> until we next perform pre-release verification
>
> At least for me there's a little softness around our collective alignment
> on when pre-release verification takes place. If it's between alpha-1 and
> ga we don't want changes that would invalidate those changes to land during
> that time frame. Different for beta-1 to ga. We also risk invalidating
> testing if we do any of that testing before wherever that cutoff is, and a
> lack of clarity on that cutoff further muddies those waters.
>
> My very loosely held perspective is that beta-1 to ga is the window in
> which we apply the "don't do things that will invalidate verification", and
> we plan to do that verification during the beta phase. I *think* this is
> consistent w/the current framing of the lifecycle doc. That being said, I
> don't have strong religion on this so if we collectively want to call it
> "don't majorly disrupt from alpha-1 to ga", we can formalize that in the
> docs and go ahead and triage current open scope for 4.0 and move things out.
>
>
>
> On Wed, May 27, 2020 at 12:59 PM Ekaterina Dimitrova <
> ekaterina.dimitrova@datastax.com> wrote:
>
>> Thank you all for your input.
>> I think an important topic is again to revise the lifecycle and ensure we
>> really have the vision on what is left until beta. I will start a separate
>> thread on the flaky tests situation soon.
>>
>> For this particular ticket I see a couple of things:
>> - There are a lot of deletions of already not used code
>> - I implemented it still in alpha as per our agreement that this will give
>> us enough time for testing. Probably Dinesh as a reviewer can give some
>> valuable feedback/opinion on the patch.
>> - It definitely touches around important places but the important thing is
>> to see how exactly it touches, I think
>> - Considering it for alpha before the major testing in beta sounds
>> reasonable to me but I guess it also depends on people availability to
>> review it in detail and the exact test plans afterwards
>>
>> On Wed, 27 May 2020 at 7:14, Benedict Elliott Smith <be...@apache.org>
>> wrote:
>>
>>> I think our "pre-beta" criteria should also be our "not in a major"
>>> criteria.
>>>
>>> If work is prohibited because it invalidates our pre-release
>> verification,
>>> then it should not land until we next perform pre-release verification,
>>> which only currently happens once per major.
>>>
>>> This could mean either landing less in a major, or permitting more in
>> beta
>>> etc.
>>>
>>> On 26/05/2020, 19:24, "Joshua McKenzie" <jm...@apache.org> wrote:
>>>
>>> I think an interesting question that informs when to stop accepting
>>> specific changes in a release is when we expect any extensive
>>> pre-release
>>> testing to take place.
>>>
>>> If we go by our release lifecycle, gutting deprecated code seems
>>> compatible
>>> w/Alpha but I wouldn't endorse merging it into Beta:
>>>
>>> https://cwiki.apache.org/confluence/display/CASSANDRA/Release+Lifecycle.
>>> Since almost all of the 40_quality_testing epic stuff is also beta
>>> phase
>>> and hasn't really taken off yet, it also seems like there will be
>>> extensive
>>> testing after this phase transition.
>>>
>>> All that being said, I'd advocate for marking FixVer 4.x to indicate
>>> optionality and disallow merge of tickets like this after we're done
>>> w/alpha phase in keeping w/our lifecycle doc in general.
>>>
>>> Does that make sense? Should we consider revisiting and revising the
>>> lifecycle doc re: larger deprecation / changes and cycle stages?
>>>
>>>
>>>
>>> On Tue, May 26, 2020 at 12:53 PM Oleksandr Petrov <
>>> oleksandr.petrov@gmail.com> wrote:
>>>
>>>>> 1) Would you block the release over this ticket?
>>>>
>>>> I would definitely not block the release on this ticket.
>>>>
>>>>> 2) Would you prioritize this ticket over testing?
>>>>
>>>> Same here, I would prioritise testing.
>>>>
>>>>> 3) Does fixing this ticket make 4.0 a more stable release?
>>>>
>>>> I wanted to give some context: I wrote that in August 2018. While I
>>> still
>>>> believe it is important to get rid of this code, I'm disinclined to
>>> merge
>>>> it into 4.0.
>>>>
>>>> Given that the patch is rather big (421 additions and 1,480
>>> deletions) and
>>>> touches many important places, including parser, I would be
>> extremely
>>>> cautious to merge it that late in release cycle. It would be great
>>> to also
>>>> hear arguments that would justify the risk.
>>>>
>>>> Thank you for starting this discussion,
>>>> -- Alex
>>>>
>>>>
>>>>
>>>> On Tue, May 26, 2020 at 5:20 PM Ekaterina Dimitrova <
>>>> ekaterina.dimitrova@datastax.com> wrote:
>>>>
>>>>> Dear all,
>>>>>
>>>>> Following the ticket review sent on 12th May I wanted to bring up
>>>>> https://issues.apache.org/jira/browse/CASSANDRA-13994: Remove
>>> COMPACT
>>>>>
>>>>> STORAGE internals before 4.0 release.
>>>>>
>>>>> It is already under review by Dinesh Joshi and Alex Petrov. Not a
>>>>> blocker but already under review.
>>>>>
>>>>> Below are my responses to the questions brought up.
>>>>>
>>>>>
>>>>> 1) Would you block the release over this
>>>>>
>>>>> ticket? - probably not
>>>>>
>>>>> 2) Would you prioritize this ticket over testing? - already
>>>>> implemented but if there are some big changes needed after the
>>> review,
>>>>> I doubt it we will want to prioritize over the testing
>>>>>
>>>>> 3) Does fixing
>>>>> this ticket make 4.0 a more stable release? - I will just cite
>> Alex
>>>>> Petrov who reported this Jira and I think the rest of us would
>>> agree
>>>>> with him here.
>>>>>
>>>>> "I would say it's quite important to clean up compact storage
>>>>> internals in 4.0 before the release. It should have no visible
>>>>> side-effects, but it'd be very good to have as it simplifies
>>> multiple
>>>>> code paths."
>>>>>
>>>>>
>>>>> Ekaterina Dimitrova
>>>>> e. ekaterina.dimitrova@datastax.com
>>>>> w. www.datastax.com
>>>>>
>>>>
>>>>
>>>> --
>>>> alex p
>>>>
>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
>>> For additional commands, e-mail: dev-help@cassandra.apache.org
>>>
>>>
>>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> For additional commands, e-mail: dev-help@cassandra.apache.org
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> For additional commands, e-mail: dev-help@cassandra.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
For additional commands, e-mail: dev-help@cassandra.apache.org
Re: [DISCUSS] CASSANDRA-13994
Posted by Benedict Elliott Smith <be...@apache.org>.
I'm being told this still isn't clear, so let me try in a bullet-point timeline:
* 4.0 Beta
* 4.0 Verification Work
* [Merge Window]
* 4.0 GA
* 4.0 Minor Releases
* ...
* 5.0 Dev
* ...
* 5.0 Verification Work
* GA 5.0
I think that anything that is prohibited from "[Merge Window]" because it invalidates "4.0 Verification Work" must also be prohibited until "5.0 Dev" because the next equivalent work that can now validate it occurs only at "5.0 Verification Work"
On 27/05/2020, 19:05, "Benedict Elliott Smith" <be...@apache.org> wrote:
I'm not sure if I communicated my point very well. I mean to say that if the reason we are prohibiting a patch to land post-beta is because it invalidates work we only perform pre-ga, then it probably should not be permitted to land post-ga either, since it must also invalidate the same work?
That is to say, if we're comfortable with work landing post-ga because we believe it to be safe to release without our pre-major-release verification, we should be comfortable with it landing at any time pre-ga too. Anything else seems inconsistent to me, and we should examine what assumptions we're making that permit this inconsistency to arise.
On 27/05/2020, 18:49, "Joshua McKenzie" <jm...@apache.org> wrote:
>
> because it invalidates our pre-release verification, then it should not
> land
until we next perform pre-release verification
At least for me there's a little softness around our collective alignment
on when pre-release verification takes place. If it's between alpha-1 and
ga we don't want changes that would invalidate those changes to land during
that time frame. Different for beta-1 to ga. We also risk invalidating
testing if we do any of that testing before wherever that cutoff is, and a
lack of clarity on that cutoff further muddies those waters.
My very loosely held perspective is that beta-1 to ga is the window in
which we apply the "don't do things that will invalidate verification", and
we plan to do that verification during the beta phase. I *think* this is
consistent w/the current framing of the lifecycle doc. That being said, I
don't have strong religion on this so if we collectively want to call it
"don't majorly disrupt from alpha-1 to ga", we can formalize that in the
docs and go ahead and triage current open scope for 4.0 and move things out.
On Wed, May 27, 2020 at 12:59 PM Ekaterina Dimitrova <
ekaterina.dimitrova@datastax.com> wrote:
> Thank you all for your input.
> I think an important topic is again to revise the lifecycle and ensure we
> really have the vision on what is left until beta. I will start a separate
> thread on the flaky tests situation soon.
>
> For this particular ticket I see a couple of things:
> - There are a lot of deletions of already not used code
> - I implemented it still in alpha as per our agreement that this will give
> us enough time for testing. Probably Dinesh as a reviewer can give some
> valuable feedback/opinion on the patch.
> - It definitely touches around important places but the important thing is
> to see how exactly it touches, I think
> - Considering it for alpha before the major testing in beta sounds
> reasonable to me but I guess it also depends on people availability to
> review it in detail and the exact test plans afterwards
>
> On Wed, 27 May 2020 at 7:14, Benedict Elliott Smith <be...@apache.org>
> wrote:
>
> > I think our "pre-beta" criteria should also be our "not in a major"
> > criteria.
> >
> > If work is prohibited because it invalidates our pre-release
> verification,
> > then it should not land until we next perform pre-release verification,
> > which only currently happens once per major.
> >
> > This could mean either landing less in a major, or permitting more in
> beta
> > etc.
> >
> > On 26/05/2020, 19:24, "Joshua McKenzie" <jm...@apache.org> wrote:
> >
> > I think an interesting question that informs when to stop accepting
> > specific changes in a release is when we expect any extensive
> > pre-release
> > testing to take place.
> >
> > If we go by our release lifecycle, gutting deprecated code seems
> > compatible
> > w/Alpha but I wouldn't endorse merging it into Beta:
> >
> > https://cwiki.apache.org/confluence/display/CASSANDRA/Release+Lifecycle.
> > Since almost all of the 40_quality_testing epic stuff is also beta
> > phase
> > and hasn't really taken off yet, it also seems like there will be
> > extensive
> > testing after this phase transition.
> >
> > All that being said, I'd advocate for marking FixVer 4.x to indicate
> > optionality and disallow merge of tickets like this after we're done
> > w/alpha phase in keeping w/our lifecycle doc in general.
> >
> > Does that make sense? Should we consider revisiting and revising the
> > lifecycle doc re: larger deprecation / changes and cycle stages?
> >
> >
> >
> > On Tue, May 26, 2020 at 12:53 PM Oleksandr Petrov <
> > oleksandr.petrov@gmail.com> wrote:
> >
> > > > 1) Would you block the release over this ticket?
> > >
> > > I would definitely not block the release on this ticket.
> > >
> > > > 2) Would you prioritize this ticket over testing?
> > >
> > > Same here, I would prioritise testing.
> > >
> > > > 3) Does fixing this ticket make 4.0 a more stable release?
> > >
> > > I wanted to give some context: I wrote that in August 2018. While I
> > still
> > > believe it is important to get rid of this code, I'm disinclined to
> > merge
> > > it into 4.0.
> > >
> > > Given that the patch is rather big (421 additions and 1,480
> > deletions) and
> > > touches many important places, including parser, I would be
> extremely
> > > cautious to merge it that late in release cycle. It would be great
> > to also
> > > hear arguments that would justify the risk.
> > >
> > > Thank you for starting this discussion,
> > > -- Alex
> > >
> > >
> > >
> > > On Tue, May 26, 2020 at 5:20 PM Ekaterina Dimitrova <
> > > ekaterina.dimitrova@datastax.com> wrote:
> > >
> > > > Dear all,
> > > >
> > > > Following the ticket review sent on 12th May I wanted to bring up
> > > > https://issues.apache.org/jira/browse/CASSANDRA-13994: Remove
> > COMPACT
> > > >
> > > > STORAGE internals before 4.0 release.
> > > >
> > > > It is already under review by Dinesh Joshi and Alex Petrov. Not a
> > > > blocker but already under review.
> > > >
> > > > Below are my responses to the questions brought up.
> > > >
> > > >
> > > > 1) Would you block the release over this
> > > >
> > > > ticket? - probably not
> > > >
> > > > 2) Would you prioritize this ticket over testing? - already
> > > > implemented but if there are some big changes needed after the
> > review,
> > > > I doubt it we will want to prioritize over the testing
> > > >
> > > > 3) Does fixing
> > > > this ticket make 4.0 a more stable release? - I will just cite
> Alex
> > > > Petrov who reported this Jira and I think the rest of us would
> > agree
> > > > with him here.
> > > >
> > > > "I would say it's quite important to clean up compact storage
> > > > internals in 4.0 before the release. It should have no visible
> > > > side-effects, but it'd be very good to have as it simplifies
> > multiple
> > > > code paths."
> > > >
> > > >
> > > > Ekaterina Dimitrova
> > > > e. ekaterina.dimitrova@datastax.com
> > > > w. www.datastax.com
> > > >
> > >
> > >
> > > --
> > > alex p
> > >
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> > For additional commands, e-mail: dev-help@cassandra.apache.org
> >
> >
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
For additional commands, e-mail: dev-help@cassandra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
For additional commands, e-mail: dev-help@cassandra.apache.org
Re: [DISCUSS] CASSANDRA-13994
Posted by Joshua McKenzie <jm...@apache.org>.
In this hypothetical, certainly not post-ga, and I'd argue we shouldn't
allow it post-beta1 and we need a clear demarcation of "this type of work
is ok to merge before X, it's not ok after X. Validation testing *will not
occur* before X, and will start after X".
It's a bit rigid, but it's the only way to have a clear inflection point
where you know subsequent work won't be invalidated. Otherwise we end up in
"I'm pretty sure the validation for disruptive thing X hasn't occurred so
I'm going to merge it now" hell.
(does what I'm typing here make sense given the context of what you said?
Had a little trouble parsing, I think because there's fuzziness around
"alpha 1 release" vs. "alpha phase" on how we label things in the project.
Maybe)
On Wed, May 27, 2020 at 2:05 PM Benedict Elliott Smith <be...@apache.org>
wrote:
> I'm not sure if I communicated my point very well. I mean to say that if
> the reason we are prohibiting a patch to land post-beta is because it
> invalidates work we only perform pre-ga, then it probably should not be
> permitted to land post-ga either, since it must also invalidate the same
> work?
>
> That is to say, if we're comfortable with work landing post-ga because we
> believe it to be safe to release without our pre-major-release
> verification, we should be comfortable with it landing at any time pre-ga
> too. Anything else seems inconsistent to me, and we should examine what
> assumptions we're making that permit this inconsistency to arise.
>
>
> On 27/05/2020, 18:49, "Joshua McKenzie" <jm...@apache.org> wrote:
>
> >
> > because it invalidates our pre-release verification, then it should
> not
> > land
>
> until we next perform pre-release verification
>
> At least for me there's a little softness around our collective
> alignment
> on when pre-release verification takes place. If it's between alpha-1
> and
> ga we don't want changes that would invalidate those changes to land
> during
> that time frame. Different for beta-1 to ga. We also risk invalidating
> testing if we do any of that testing before wherever that cutoff is,
> and a
> lack of clarity on that cutoff further muddies those waters.
>
> My very loosely held perspective is that beta-1 to ga is the window in
> which we apply the "don't do things that will invalidate
> verification", and
> we plan to do that verification during the beta phase. I *think* this
> is
> consistent w/the current framing of the lifecycle doc. That being
> said, I
> don't have strong religion on this so if we collectively want to call
> it
> "don't majorly disrupt from alpha-1 to ga", we can formalize that in
> the
> docs and go ahead and triage current open scope for 4.0 and move
> things out.
>
>
>
> On Wed, May 27, 2020 at 12:59 PM Ekaterina Dimitrova <
> ekaterina.dimitrova@datastax.com> wrote:
>
> > Thank you all for your input.
> > I think an important topic is again to revise the lifecycle and
> ensure we
> > really have the vision on what is left until beta. I will start a
> separate
> > thread on the flaky tests situation soon.
> >
> > For this particular ticket I see a couple of things:
> > - There are a lot of deletions of already not used code
> > - I implemented it still in alpha as per our agreement that this
> will give
> > us enough time for testing. Probably Dinesh as a reviewer can give
> some
> > valuable feedback/opinion on the patch.
> > - It definitely touches around important places but the important
> thing is
> > to see how exactly it touches, I think
> > - Considering it for alpha before the major testing in beta sounds
> > reasonable to me but I guess it also depends on people availability
> to
> > review it in detail and the exact test plans afterwards
> >
> > On Wed, 27 May 2020 at 7:14, Benedict Elliott Smith <
> benedict@apache.org>
> > wrote:
> >
> > > I think our "pre-beta" criteria should also be our "not in a major"
> > > criteria.
> > >
> > > If work is prohibited because it invalidates our pre-release
> > verification,
> > > then it should not land until we next perform pre-release
> verification,
> > > which only currently happens once per major.
> > >
> > > This could mean either landing less in a major, or permitting more
> in
> > beta
> > > etc.
> > >
> > > On 26/05/2020, 19:24, "Joshua McKenzie" <jm...@apache.org>
> wrote:
> > >
> > > I think an interesting question that informs when to stop
> accepting
> > > specific changes in a release is when we expect any extensive
> > > pre-release
> > > testing to take place.
> > >
> > > If we go by our release lifecycle, gutting deprecated code
> seems
> > > compatible
> > > w/Alpha but I wouldn't endorse merging it into Beta:
> > >
> > >
> https://cwiki.apache.org/confluence/display/CASSANDRA/Release+Lifecycle.
> > > Since almost all of the 40_quality_testing epic stuff is also
> beta
> > > phase
> > > and hasn't really taken off yet, it also seems like there will
> be
> > > extensive
> > > testing after this phase transition.
> > >
> > > All that being said, I'd advocate for marking FixVer 4.x to
> indicate
> > > optionality and disallow merge of tickets like this after
> we're done
> > > w/alpha phase in keeping w/our lifecycle doc in general.
> > >
> > > Does that make sense? Should we consider revisiting and
> revising the
> > > lifecycle doc re: larger deprecation / changes and cycle
> stages?
> > >
> > >
> > >
> > > On Tue, May 26, 2020 at 12:53 PM Oleksandr Petrov <
> > > oleksandr.petrov@gmail.com> wrote:
> > >
> > > > > 1) Would you block the release over this ticket?
> > > >
> > > > I would definitely not block the release on this ticket.
> > > >
> > > > > 2) Would you prioritize this ticket over testing?
> > > >
> > > > Same here, I would prioritise testing.
> > > >
> > > > > 3) Does fixing this ticket make 4.0 a more stable release?
> > > >
> > > > I wanted to give some context: I wrote that in August 2018.
> While I
> > > still
> > > > believe it is important to get rid of this code, I'm
> disinclined to
> > > merge
> > > > it into 4.0.
> > > >
> > > > Given that the patch is rather big (421 additions and 1,480
> > > deletions) and
> > > > touches many important places, including parser, I would be
> > extremely
> > > > cautious to merge it that late in release cycle. It would be
> great
> > > to also
> > > > hear arguments that would justify the risk.
> > > >
> > > > Thank you for starting this discussion,
> > > > -- Alex
> > > >
> > > >
> > > >
> > > > On Tue, May 26, 2020 at 5:20 PM Ekaterina Dimitrova <
> > > > ekaterina.dimitrova@datastax.com> wrote:
> > > >
> > > > > Dear all,
> > > > >
> > > > > Following the ticket review sent on 12th May I wanted to
> bring up
> > > > > https://issues.apache.org/jira/browse/CASSANDRA-13994:
> Remove
> > > COMPACT
> > > > >
> > > > > STORAGE internals before 4.0 release.
> > > > >
> > > > > It is already under review by Dinesh Joshi and Alex
> Petrov. Not a
> > > > > blocker but already under review.
> > > > >
> > > > > Below are my responses to the questions brought up.
> > > > >
> > > > >
> > > > > 1) Would you block the release over this
> > > > >
> > > > > ticket? - probably not
> > > > >
> > > > > 2) Would you prioritize this ticket over testing? - already
> > > > > implemented but if there are some big changes needed after
> the
> > > review,
> > > > > I doubt it we will want to prioritize over the testing
> > > > >
> > > > > 3) Does fixing
> > > > > this ticket make 4.0 a more stable release? - I will just
> cite
> > Alex
> > > > > Petrov who reported this Jira and I think the rest of us
> would
> > > agree
> > > > > with him here.
> > > > >
> > > > > "I would say it's quite important to clean up compact
> storage
> > > > > internals in 4.0 before the release. It should have no
> visible
> > > > > side-effects, but it'd be very good to have as it
> simplifies
> > > multiple
> > > > > code paths."
> > > > >
> > > > >
> > > > > Ekaterina Dimitrova
> > > > > e. ekaterina.dimitrova@datastax.com
> > > > > w. www.datastax.com
> > > > >
> > > >
> > > >
> > > > --
> > > > alex p
> > > >
> > >
> > >
> > >
> > >
> ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> > > For additional commands, e-mail: dev-help@cassandra.apache.org
> > >
> > >
> >
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> For additional commands, e-mail: dev-help@cassandra.apache.org
>
>
Re: [DISCUSS] CASSANDRA-13994
Posted by Benedict Elliott Smith <be...@apache.org>.
I'm not sure if I communicated my point very well. I mean to say that if the reason we are prohibiting a patch to land post-beta is because it invalidates work we only perform pre-ga, then it probably should not be permitted to land post-ga either, since it must also invalidate the same work?
That is to say, if we're comfortable with work landing post-ga because we believe it to be safe to release without our pre-major-release verification, we should be comfortable with it landing at any time pre-ga too. Anything else seems inconsistent to me, and we should examine what assumptions we're making that permit this inconsistency to arise.
On 27/05/2020, 18:49, "Joshua McKenzie" <jm...@apache.org> wrote:
>
> because it invalidates our pre-release verification, then it should not
> land
until we next perform pre-release verification
At least for me there's a little softness around our collective alignment
on when pre-release verification takes place. If it's between alpha-1 and
ga we don't want changes that would invalidate those changes to land during
that time frame. Different for beta-1 to ga. We also risk invalidating
testing if we do any of that testing before wherever that cutoff is, and a
lack of clarity on that cutoff further muddies those waters.
My very loosely held perspective is that beta-1 to ga is the window in
which we apply the "don't do things that will invalidate verification", and
we plan to do that verification during the beta phase. I *think* this is
consistent w/the current framing of the lifecycle doc. That being said, I
don't have strong religion on this so if we collectively want to call it
"don't majorly disrupt from alpha-1 to ga", we can formalize that in the
docs and go ahead and triage current open scope for 4.0 and move things out.
On Wed, May 27, 2020 at 12:59 PM Ekaterina Dimitrova <
ekaterina.dimitrova@datastax.com> wrote:
> Thank you all for your input.
> I think an important topic is again to revise the lifecycle and ensure we
> really have the vision on what is left until beta. I will start a separate
> thread on the flaky tests situation soon.
>
> For this particular ticket I see a couple of things:
> - There are a lot of deletions of already not used code
> - I implemented it still in alpha as per our agreement that this will give
> us enough time for testing. Probably Dinesh as a reviewer can give some
> valuable feedback/opinion on the patch.
> - It definitely touches around important places but the important thing is
> to see how exactly it touches, I think
> - Considering it for alpha before the major testing in beta sounds
> reasonable to me but I guess it also depends on people availability to
> review it in detail and the exact test plans afterwards
>
> On Wed, 27 May 2020 at 7:14, Benedict Elliott Smith <be...@apache.org>
> wrote:
>
> > I think our "pre-beta" criteria should also be our "not in a major"
> > criteria.
> >
> > If work is prohibited because it invalidates our pre-release
> verification,
> > then it should not land until we next perform pre-release verification,
> > which only currently happens once per major.
> >
> > This could mean either landing less in a major, or permitting more in
> beta
> > etc.
> >
> > On 26/05/2020, 19:24, "Joshua McKenzie" <jm...@apache.org> wrote:
> >
> > I think an interesting question that informs when to stop accepting
> > specific changes in a release is when we expect any extensive
> > pre-release
> > testing to take place.
> >
> > If we go by our release lifecycle, gutting deprecated code seems
> > compatible
> > w/Alpha but I wouldn't endorse merging it into Beta:
> >
> > https://cwiki.apache.org/confluence/display/CASSANDRA/Release+Lifecycle.
> > Since almost all of the 40_quality_testing epic stuff is also beta
> > phase
> > and hasn't really taken off yet, it also seems like there will be
> > extensive
> > testing after this phase transition.
> >
> > All that being said, I'd advocate for marking FixVer 4.x to indicate
> > optionality and disallow merge of tickets like this after we're done
> > w/alpha phase in keeping w/our lifecycle doc in general.
> >
> > Does that make sense? Should we consider revisiting and revising the
> > lifecycle doc re: larger deprecation / changes and cycle stages?
> >
> >
> >
> > On Tue, May 26, 2020 at 12:53 PM Oleksandr Petrov <
> > oleksandr.petrov@gmail.com> wrote:
> >
> > > > 1) Would you block the release over this ticket?
> > >
> > > I would definitely not block the release on this ticket.
> > >
> > > > 2) Would you prioritize this ticket over testing?
> > >
> > > Same here, I would prioritise testing.
> > >
> > > > 3) Does fixing this ticket make 4.0 a more stable release?
> > >
> > > I wanted to give some context: I wrote that in August 2018. While I
> > still
> > > believe it is important to get rid of this code, I'm disinclined to
> > merge
> > > it into 4.0.
> > >
> > > Given that the patch is rather big (421 additions and 1,480
> > deletions) and
> > > touches many important places, including parser, I would be
> extremely
> > > cautious to merge it that late in release cycle. It would be great
> > to also
> > > hear arguments that would justify the risk.
> > >
> > > Thank you for starting this discussion,
> > > -- Alex
> > >
> > >
> > >
> > > On Tue, May 26, 2020 at 5:20 PM Ekaterina Dimitrova <
> > > ekaterina.dimitrova@datastax.com> wrote:
> > >
> > > > Dear all,
> > > >
> > > > Following the ticket review sent on 12th May I wanted to bring up
> > > > https://issues.apache.org/jira/browse/CASSANDRA-13994: Remove
> > COMPACT
> > > >
> > > > STORAGE internals before 4.0 release.
> > > >
> > > > It is already under review by Dinesh Joshi and Alex Petrov. Not a
> > > > blocker but already under review.
> > > >
> > > > Below are my responses to the questions brought up.
> > > >
> > > >
> > > > 1) Would you block the release over this
> > > >
> > > > ticket? - probably not
> > > >
> > > > 2) Would you prioritize this ticket over testing? - already
> > > > implemented but if there are some big changes needed after the
> > review,
> > > > I doubt it we will want to prioritize over the testing
> > > >
> > > > 3) Does fixing
> > > > this ticket make 4.0 a more stable release? - I will just cite
> Alex
> > > > Petrov who reported this Jira and I think the rest of us would
> > agree
> > > > with him here.
> > > >
> > > > "I would say it's quite important to clean up compact storage
> > > > internals in 4.0 before the release. It should have no visible
> > > > side-effects, but it'd be very good to have as it simplifies
> > multiple
> > > > code paths."
> > > >
> > > >
> > > > Ekaterina Dimitrova
> > > > e. ekaterina.dimitrova@datastax.com
> > > > w. www.datastax.com
> > > >
> > >
> > >
> > > --
> > > alex p
> > >
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> > For additional commands, e-mail: dev-help@cassandra.apache.org
> >
> >
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
For additional commands, e-mail: dev-help@cassandra.apache.org
Re: [DISCUSS] CASSANDRA-13994
Posted by Joshua McKenzie <jm...@apache.org>.
>
> because it invalidates our pre-release verification, then it should not
> land
until we next perform pre-release verification
At least for me there's a little softness around our collective alignment
on when pre-release verification takes place. If it's between alpha-1 and
ga we don't want changes that would invalidate those changes to land during
that time frame. Different for beta-1 to ga. We also risk invalidating
testing if we do any of that testing before wherever that cutoff is, and a
lack of clarity on that cutoff further muddies those waters.
My very loosely held perspective is that beta-1 to ga is the window in
which we apply the "don't do things that will invalidate verification", and
we plan to do that verification during the beta phase. I *think* this is
consistent w/the current framing of the lifecycle doc. That being said, I
don't have strong religion on this so if we collectively want to call it
"don't majorly disrupt from alpha-1 to ga", we can formalize that in the
docs and go ahead and triage current open scope for 4.0 and move things out.
On Wed, May 27, 2020 at 12:59 PM Ekaterina Dimitrova <
ekaterina.dimitrova@datastax.com> wrote:
> Thank you all for your input.
> I think an important topic is again to revise the lifecycle and ensure we
> really have the vision on what is left until beta. I will start a separate
> thread on the flaky tests situation soon.
>
> For this particular ticket I see a couple of things:
> - There are a lot of deletions of already not used code
> - I implemented it still in alpha as per our agreement that this will give
> us enough time for testing. Probably Dinesh as a reviewer can give some
> valuable feedback/opinion on the patch.
> - It definitely touches around important places but the important thing is
> to see how exactly it touches, I think
> - Considering it for alpha before the major testing in beta sounds
> reasonable to me but I guess it also depends on people availability to
> review it in detail and the exact test plans afterwards
>
> On Wed, 27 May 2020 at 7:14, Benedict Elliott Smith <be...@apache.org>
> wrote:
>
> > I think our "pre-beta" criteria should also be our "not in a major"
> > criteria.
> >
> > If work is prohibited because it invalidates our pre-release
> verification,
> > then it should not land until we next perform pre-release verification,
> > which only currently happens once per major.
> >
> > This could mean either landing less in a major, or permitting more in
> beta
> > etc.
> >
> > On 26/05/2020, 19:24, "Joshua McKenzie" <jm...@apache.org> wrote:
> >
> > I think an interesting question that informs when to stop accepting
> > specific changes in a release is when we expect any extensive
> > pre-release
> > testing to take place.
> >
> > If we go by our release lifecycle, gutting deprecated code seems
> > compatible
> > w/Alpha but I wouldn't endorse merging it into Beta:
> >
> > https://cwiki.apache.org/confluence/display/CASSANDRA/Release+Lifecycle.
> > Since almost all of the 40_quality_testing epic stuff is also beta
> > phase
> > and hasn't really taken off yet, it also seems like there will be
> > extensive
> > testing after this phase transition.
> >
> > All that being said, I'd advocate for marking FixVer 4.x to indicate
> > optionality and disallow merge of tickets like this after we're done
> > w/alpha phase in keeping w/our lifecycle doc in general.
> >
> > Does that make sense? Should we consider revisiting and revising the
> > lifecycle doc re: larger deprecation / changes and cycle stages?
> >
> >
> >
> > On Tue, May 26, 2020 at 12:53 PM Oleksandr Petrov <
> > oleksandr.petrov@gmail.com> wrote:
> >
> > > > 1) Would you block the release over this ticket?
> > >
> > > I would definitely not block the release on this ticket.
> > >
> > > > 2) Would you prioritize this ticket over testing?
> > >
> > > Same here, I would prioritise testing.
> > >
> > > > 3) Does fixing this ticket make 4.0 a more stable release?
> > >
> > > I wanted to give some context: I wrote that in August 2018. While I
> > still
> > > believe it is important to get rid of this code, I'm disinclined to
> > merge
> > > it into 4.0.
> > >
> > > Given that the patch is rather big (421 additions and 1,480
> > deletions) and
> > > touches many important places, including parser, I would be
> extremely
> > > cautious to merge it that late in release cycle. It would be great
> > to also
> > > hear arguments that would justify the risk.
> > >
> > > Thank you for starting this discussion,
> > > -- Alex
> > >
> > >
> > >
> > > On Tue, May 26, 2020 at 5:20 PM Ekaterina Dimitrova <
> > > ekaterina.dimitrova@datastax.com> wrote:
> > >
> > > > Dear all,
> > > >
> > > > Following the ticket review sent on 12th May I wanted to bring up
> > > > https://issues.apache.org/jira/browse/CASSANDRA-13994: Remove
> > COMPACT
> > > >
> > > > STORAGE internals before 4.0 release.
> > > >
> > > > It is already under review by Dinesh Joshi and Alex Petrov. Not a
> > > > blocker but already under review.
> > > >
> > > > Below are my responses to the questions brought up.
> > > >
> > > >
> > > > 1) Would you block the release over this
> > > >
> > > > ticket? - probably not
> > > >
> > > > 2) Would you prioritize this ticket over testing? - already
> > > > implemented but if there are some big changes needed after the
> > review,
> > > > I doubt it we will want to prioritize over the testing
> > > >
> > > > 3) Does fixing
> > > > this ticket make 4.0 a more stable release? - I will just cite
> Alex
> > > > Petrov who reported this Jira and I think the rest of us would
> > agree
> > > > with him here.
> > > >
> > > > "I would say it's quite important to clean up compact storage
> > > > internals in 4.0 before the release. It should have no visible
> > > > side-effects, but it'd be very good to have as it simplifies
> > multiple
> > > > code paths."
> > > >
> > > >
> > > > Ekaterina Dimitrova
> > > > e. ekaterina.dimitrova@datastax.com
> > > > w. www.datastax.com
> > > >
> > >
> > >
> > > --
> > > alex p
> > >
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> > For additional commands, e-mail: dev-help@cassandra.apache.org
> >
> >
>
Fwd: [DISCUSS] CASSANDRA-13994
Posted by Ekaterina Dimitrova <ek...@datastax.com>.
Thank you all for your input.
I think an important topic is again to revise the lifecycle and ensure we
really have the vision on what is left until beta. I will start a separate
thread on the flaky tests situation soon.
For this particular ticket I see a couple of things:
- There are a lot of deletions of already not used code
- I implemented it still in alpha as per our agreement that this will give
us enough time for testing. Probably Dinesh as a reviewer can give some
valuable feedback/opinion on the patch.
- It definitely touches around important places but the important thing is
to see how exactly it touches, I think
- Considering it for alpha before the major testing in beta sounds
reasonable to me but I guess it also depends on people availability to
review it in detail and the exact test plans afterwards
On Wed, 27 May 2020 at 7:14, Benedict Elliott Smith <be...@apache.org>
wrote:
> I think our "pre-beta" criteria should also be our "not in a major"
> criteria.
>
> If work is prohibited because it invalidates our pre-release verification,
> then it should not land until we next perform pre-release verification,
> which only currently happens once per major.
>
> This could mean either landing less in a major, or permitting more in beta
> etc.
>
> On 26/05/2020, 19:24, "Joshua McKenzie" <jm...@apache.org> wrote:
>
> I think an interesting question that informs when to stop accepting
> specific changes in a release is when we expect any extensive
> pre-release
> testing to take place.
>
> If we go by our release lifecycle, gutting deprecated code seems
> compatible
> w/Alpha but I wouldn't endorse merging it into Beta:
>
> https://cwiki.apache.org/confluence/display/CASSANDRA/Release+Lifecycle.
> Since almost all of the 40_quality_testing epic stuff is also beta
> phase
> and hasn't really taken off yet, it also seems like there will be
> extensive
> testing after this phase transition.
>
> All that being said, I'd advocate for marking FixVer 4.x to indicate
> optionality and disallow merge of tickets like this after we're done
> w/alpha phase in keeping w/our lifecycle doc in general.
>
> Does that make sense? Should we consider revisiting and revising the
> lifecycle doc re: larger deprecation / changes and cycle stages?
>
>
>
> On Tue, May 26, 2020 at 12:53 PM Oleksandr Petrov <
> oleksandr.petrov@gmail.com> wrote:
>
> > > 1) Would you block the release over this ticket?
> >
> > I would definitely not block the release on this ticket.
> >
> > > 2) Would you prioritize this ticket over testing?
> >
> > Same here, I would prioritise testing.
> >
> > > 3) Does fixing this ticket make 4.0 a more stable release?
> >
> > I wanted to give some context: I wrote that in August 2018. While I
> still
> > believe it is important to get rid of this code, I'm disinclined to
> merge
> > it into 4.0.
> >
> > Given that the patch is rather big (421 additions and 1,480
> deletions) and
> > touches many important places, including parser, I would be extremely
> > cautious to merge it that late in release cycle. It would be great
> to also
> > hear arguments that would justify the risk.
> >
> > Thank you for starting this discussion,
> > -- Alex
> >
> >
> >
> > On Tue, May 26, 2020 at 5:20 PM Ekaterina Dimitrova <
> > ekaterina.dimitrova@datastax.com> wrote:
> >
> > > Dear all,
> > >
> > > Following the ticket review sent on 12th May I wanted to bring up
> > > https://issues.apache.org/jira/browse/CASSANDRA-13994: Remove
> COMPACT
> > >
> > > STORAGE internals before 4.0 release.
> > >
> > > It is already under review by Dinesh Joshi and Alex Petrov. Not a
> > > blocker but already under review.
> > >
> > > Below are my responses to the questions brought up.
> > >
> > >
> > > 1) Would you block the release over this
> > >
> > > ticket? - probably not
> > >
> > > 2) Would you prioritize this ticket over testing? - already
> > > implemented but if there are some big changes needed after the
> review,
> > > I doubt it we will want to prioritize over the testing
> > >
> > > 3) Does fixing
> > > this ticket make 4.0 a more stable release? - I will just cite Alex
> > > Petrov who reported this Jira and I think the rest of us would
> agree
> > > with him here.
> > >
> > > "I would say it's quite important to clean up compact storage
> > > internals in 4.0 before the release. It should have no visible
> > > side-effects, but it'd be very good to have as it simplifies
> multiple
> > > code paths."
> > >
> > >
> > > Ekaterina Dimitrova
> > > e. ekaterina.dimitrova@datastax.com
> > > w. www.datastax.com
> > >
> >
> >
> > --
> > alex p
> >
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> For additional commands, e-mail: dev-help@cassandra.apache.org
>
>
Re: [DISCUSS] CASSANDRA-13994
Posted by Benedict Elliott Smith <be...@apache.org>.
I think our "pre-beta" criteria should also be our "not in a major" criteria.
If work is prohibited because it invalidates our pre-release verification, then it should not land until we next perform pre-release verification, which only currently happens once per major.
This could mean either landing less in a major, or permitting more in beta etc.
On 26/05/2020, 19:24, "Joshua McKenzie" <jm...@apache.org> wrote:
I think an interesting question that informs when to stop accepting
specific changes in a release is when we expect any extensive pre-release
testing to take place.
If we go by our release lifecycle, gutting deprecated code seems compatible
w/Alpha but I wouldn't endorse merging it into Beta:
https://cwiki.apache.org/confluence/display/CASSANDRA/Release+Lifecycle.
Since almost all of the 40_quality_testing epic stuff is also beta phase
and hasn't really taken off yet, it also seems like there will be extensive
testing after this phase transition.
All that being said, I'd advocate for marking FixVer 4.x to indicate
optionality and disallow merge of tickets like this after we're done
w/alpha phase in keeping w/our lifecycle doc in general.
Does that make sense? Should we consider revisiting and revising the
lifecycle doc re: larger deprecation / changes and cycle stages?
On Tue, May 26, 2020 at 12:53 PM Oleksandr Petrov <
oleksandr.petrov@gmail.com> wrote:
> > 1) Would you block the release over this ticket?
>
> I would definitely not block the release on this ticket.
>
> > 2) Would you prioritize this ticket over testing?
>
> Same here, I would prioritise testing.
>
> > 3) Does fixing this ticket make 4.0 a more stable release?
>
> I wanted to give some context: I wrote that in August 2018. While I still
> believe it is important to get rid of this code, I'm disinclined to merge
> it into 4.0.
>
> Given that the patch is rather big (421 additions and 1,480 deletions) and
> touches many important places, including parser, I would be extremely
> cautious to merge it that late in release cycle. It would be great to also
> hear arguments that would justify the risk.
>
> Thank you for starting this discussion,
> -- Alex
>
>
>
> On Tue, May 26, 2020 at 5:20 PM Ekaterina Dimitrova <
> ekaterina.dimitrova@datastax.com> wrote:
>
> > Dear all,
> >
> > Following the ticket review sent on 12th May I wanted to bring up
> > https://issues.apache.org/jira/browse/CASSANDRA-13994: Remove COMPACT
> >
> > STORAGE internals before 4.0 release.
> >
> > It is already under review by Dinesh Joshi and Alex Petrov. Not a
> > blocker but already under review.
> >
> > Below are my responses to the questions brought up.
> >
> >
> > 1) Would you block the release over this
> >
> > ticket? - probably not
> >
> > 2) Would you prioritize this ticket over testing? - already
> > implemented but if there are some big changes needed after the review,
> > I doubt it we will want to prioritize over the testing
> >
> > 3) Does fixing
> > this ticket make 4.0 a more stable release? - I will just cite Alex
> > Petrov who reported this Jira and I think the rest of us would agree
> > with him here.
> >
> > "I would say it's quite important to clean up compact storage
> > internals in 4.0 before the release. It should have no visible
> > side-effects, but it'd be very good to have as it simplifies multiple
> > code paths."
> >
> >
> > Ekaterina Dimitrova
> > e. ekaterina.dimitrova@datastax.com
> > w. www.datastax.com
> >
>
>
> --
> alex p
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
For additional commands, e-mail: dev-help@cassandra.apache.org
Re: [DISCUSS] CASSANDRA-13994
Posted by Joshua McKenzie <jm...@apache.org>.
I think an interesting question that informs when to stop accepting
specific changes in a release is when we expect any extensive pre-release
testing to take place.
If we go by our release lifecycle, gutting deprecated code seems compatible
w/Alpha but I wouldn't endorse merging it into Beta:
https://cwiki.apache.org/confluence/display/CASSANDRA/Release+Lifecycle.
Since almost all of the 40_quality_testing epic stuff is also beta phase
and hasn't really taken off yet, it also seems like there will be extensive
testing after this phase transition.
All that being said, I'd advocate for marking FixVer 4.x to indicate
optionality and disallow merge of tickets like this after we're done
w/alpha phase in keeping w/our lifecycle doc in general.
Does that make sense? Should we consider revisiting and revising the
lifecycle doc re: larger deprecation / changes and cycle stages?
On Tue, May 26, 2020 at 12:53 PM Oleksandr Petrov <
oleksandr.petrov@gmail.com> wrote:
> > 1) Would you block the release over this ticket?
>
> I would definitely not block the release on this ticket.
>
> > 2) Would you prioritize this ticket over testing?
>
> Same here, I would prioritise testing.
>
> > 3) Does fixing this ticket make 4.0 a more stable release?
>
> I wanted to give some context: I wrote that in August 2018. While I still
> believe it is important to get rid of this code, I'm disinclined to merge
> it into 4.0.
>
> Given that the patch is rather big (421 additions and 1,480 deletions) and
> touches many important places, including parser, I would be extremely
> cautious to merge it that late in release cycle. It would be great to also
> hear arguments that would justify the risk.
>
> Thank you for starting this discussion,
> -- Alex
>
>
>
> On Tue, May 26, 2020 at 5:20 PM Ekaterina Dimitrova <
> ekaterina.dimitrova@datastax.com> wrote:
>
> > Dear all,
> >
> > Following the ticket review sent on 12th May I wanted to bring up
> > https://issues.apache.org/jira/browse/CASSANDRA-13994: Remove COMPACT
> >
> > STORAGE internals before 4.0 release.
> >
> > It is already under review by Dinesh Joshi and Alex Petrov. Not a
> > blocker but already under review.
> >
> > Below are my responses to the questions brought up.
> >
> >
> > 1) Would you block the release over this
> >
> > ticket? - probably not
> >
> > 2) Would you prioritize this ticket over testing? - already
> > implemented but if there are some big changes needed after the review,
> > I doubt it we will want to prioritize over the testing
> >
> > 3) Does fixing
> > this ticket make 4.0 a more stable release? - I will just cite Alex
> > Petrov who reported this Jira and I think the rest of us would agree
> > with him here.
> >
> > "I would say it's quite important to clean up compact storage
> > internals in 4.0 before the release. It should have no visible
> > side-effects, but it'd be very good to have as it simplifies multiple
> > code paths."
> >
> >
> > Ekaterina Dimitrova
> > e. ekaterina.dimitrova@datastax.com
> > w. www.datastax.com
> >
>
>
> --
> alex p
>
Re: [DISCUSS] CASSANDRA-13994
Posted by Oleksandr Petrov <ol...@gmail.com>.
> 1) Would you block the release over this ticket?
I would definitely not block the release on this ticket.
> 2) Would you prioritize this ticket over testing?
Same here, I would prioritise testing.
> 3) Does fixing this ticket make 4.0 a more stable release?
I wanted to give some context: I wrote that in August 2018. While I still
believe it is important to get rid of this code, I'm disinclined to merge
it into 4.0.
Given that the patch is rather big (421 additions and 1,480 deletions) and
touches many important places, including parser, I would be extremely
cautious to merge it that late in release cycle. It would be great to also
hear arguments that would justify the risk.
Thank you for starting this discussion,
-- Alex
On Tue, May 26, 2020 at 5:20 PM Ekaterina Dimitrova <
ekaterina.dimitrova@datastax.com> wrote:
> Dear all,
>
> Following the ticket review sent on 12th May I wanted to bring up
> https://issues.apache.org/jira/browse/CASSANDRA-13994: Remove COMPACT
>
> STORAGE internals before 4.0 release.
>
> It is already under review by Dinesh Joshi and Alex Petrov. Not a
> blocker but already under review.
>
> Below are my responses to the questions brought up.
>
>
> 1) Would you block the release over this
>
> ticket? - probably not
>
> 2) Would you prioritize this ticket over testing? - already
> implemented but if there are some big changes needed after the review,
> I doubt it we will want to prioritize over the testing
>
> 3) Does fixing
> this ticket make 4.0 a more stable release? - I will just cite Alex
> Petrov who reported this Jira and I think the rest of us would agree
> with him here.
>
> "I would say it's quite important to clean up compact storage
> internals in 4.0 before the release. It should have no visible
> side-effects, but it'd be very good to have as it simplifies multiple
> code paths."
>
>
> Ekaterina Dimitrova
> e. ekaterina.dimitrova@datastax.com
> w. www.datastax.com
>
--
alex p