Cassandra project status update 2022-08-29

[Josh McKenzie <jm...@apache.org>]

Going to be OOO the rest of this week so I figured I'd hit up a bit under the 2 week mark.

4.1:
Pretty big spike in test failures on the latest run. The Butler dashboard here: https://butler.cassandra.apache.org/#/, and the details here: https://butler.cassandra.apache.org/#/ci/upstream/compare/Cassandra-4.1/cassandra-4.1 are somewhat interesting.

Popping open the first 4 test failures for things that currently don't have tickets shows some of our usual suspects of "node already up" conflicts, timeouts, etc.
https://ci-cassandra.apache.org/job/Cassandra-4.1/149/testReport/dtest.jmx_test/TestJMX/test_compactionstats/
https://ci-cassandra.apache.org/job/Cassandra-4.1/149/testReport/dtest-novnode.bootstrap_test/TestBootstrap/test_bootstrap_with_reset_bootstrap_state/
https://ci-cassandra.apache.org/job/Cassandra-4.1/149/testReport/dtest-upgrade.upgrade_tests.upgrade_through_versions_test/TestUpgrade_indev_3_11_x_To_indev_4_1_x/test_parallel_upgrade/
https://ci-cassandra.apache.org/job/Cassandra-4.1/149/testReport/org.apache.cassandra.distributed.test/PaxosRepairTest/paxosRepairVersionGate/

My money's on test run 149 having some specific runtime env failures that we won't see re-occur. Given we were down to 4 failures on run 148, I say we sit tight and see how the next run goes.

In the butler dashboard, we see CASSANDRA-17461 (CASTest failure) that Benedict is looking into, and a CalledProcessError on dtest-novnode.auth_test.TestAuthRoles.test_role_caching_authenticated_user which has some flakiness to it; 10% in the last 30 runs in CI: https://ci-cassandra.apache.org/job/Cassandra-4.1/148/testReport/dtest-novnode.auth_test/TestAuthRoles/test_role_caching_authenticated_user/

If anyone has spare cycles to create a JIRA for that failure and take a look that'd be a huge help.

We discussed on a previous thread moving towards a "when a green run, go beta, when 3 green and no other tickets open, go GA" paradigm for the 4.1 release. When we get to a green run let's pick that discussion back up; seemed like we had a lazy consensus on the topic.


[New Contributors Getting Started]
As I asked on the last update: what are you interested in? To search JIRA for a topic of interest, replace "ReplaceTextHere" with the topic on the following JIRA search: https://issues.apache.org/jira/issues/?jql=project%20%3D%20cassandra%20AND%20resolution%20!%3D%20unresolved%20AND%20assignee%20is%20EMPTY%20AND%20summary%20~%20%27ReplaceTextHere%27%20ORDER%20BY%20priority%20ASC

If you're not sure, are interested in all of it, or otherwise could use a little more guidance, we have a set of curated tickets that are unassigned we call "LHF", or low-hanging-fruit, you can access via the following link: https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=484&quickFilter=2454&quickFilter=2160&quickFilter=2162

If you're looking for something a bit heftier, removing the "Starter Tickets" filter gives you 44 to choose from in 4.0.x (our next patch release, so limited to bugfixes): https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=484&quickFilter=2454&quickFilter=2160.

To get situated, here's an explanation of various types of contribution: https://cassandra.apache.org/_/community.html#how-to-contribute
An overview of the C* architecture: https://cassandra.apache.org/doc/latest/cassandra/architecture/overview.html
And here's our getting started contributing guide: https://cassandra.apache.org/_/development/index.html
We hang out in #cassandra-dev on https://the-asf.slack.com and there's a @cassandra_mentors alias you can use to reach a bunch of us that have volunteered to help newcomers get situated.


[Dev list Digest]
https://lists.apache.org/list?dev@cassandra.apache.org:lte=2w:

Given the slightly shorter time on calendar since our last update, we've been a *bit* quieter on the list.

Cassandra 4.0.6 was released - see here for the included changes in CHANGES.txt: https://gitbox.apache.org/repos/asf?p=cassandra.git;a=blob_plain;f=CHANGES.txt;hb=refs/tags/cassandra-4.0.6 and here for NEWS entries: https://gitbox.apache.org/repos/asf?p=cassandra.git;a=blob_plain;f=NEWS.txt;hb=refs/tags/cassandra-4.0.6

Ekaterina reached out about resuming work on JDK17 support and looking for anyone that wanted to collaborate on https://issues.apache.org/jira/browse/CASSANDRA-16895. JDK8 is getting a little long in the tooth (more on that later) so support for a new JDK would be a critical step in the direction of deprecating older platforms.

Paulo Motta forwarded the link to the ASF community survey which can be found here: https://edi-asf.limesurvey.net/912832?lang=en

Andres de la Pena opened up discussion around a dynamic data masking feature in CEP-20 on the following email thread: https://lists.apache.org/thread/qsmxsymozymy6dy9tp5xw9gn5fhz9nt4 - as the project continues to mature and be used in more organizations with compliance requirements, these kind of features are incredibly valuable to our end users. Thanks Andres!

A new episode of the Apache Cassandra Corner podcast will hit 2022/08/31 if nobody has any objections before then; it can be previewed in the thread from Aaron Ploetz here: https://lists.apache.org/thread/tygxrxt6v6ym3kzorbv24dx6c5h2zy7x

We have a couple of blog posts that have gone up for review and have had some back and forth on those; it's a pretty low friction way to help improve some of the "easier to consume" content in our ecosystem so if you have a knack for writing, or an interest in it, please chime in!

And as alluded to above, Blake's reached out above the removal of JDK8 support in trunk (next major) here: https://lists.apache.org/thread/8cdkpj7c6po1mn140k3mlwdx73gr74wq. Curious to see what requirements everyone is facing out there in the wild and what resources we have as a project in terms of the number of JDK's we support going forward.

[CI Trends]
https://butler.cassandra.apache.org/#/

Here's our trends on our branches for the last two weeks:

3.0: 11 -> 7
3.11: 20 -> 10
4.0: 5 -> 5
4.1: 7 -> 13
trunk: 8 -> 8

Continuing to tighten up excepting the anomaly on 4.1


[Release progress]
https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=484&quickFilter=2175

In the interest of time I'm going to defer some of the digging to each of you if you're curious and point out highlights here:

4.1 alpha + beta + rc: 11 issues total
Scrubber fixes, bulkloader fixes, Tidying up some hanging repairs, repair coordinators getting stuck in infinite loops, an index out of bounds exception in Paging State version, and four test fixes. Good 1.5 weeks for the beta!

4.X / Next: 17 issues
The big ones that stand out worth knowing about as developers on the project:
- Abe removed our dependency on Maven Ant Tasks in CASSANDRA-17750
- Ekaterina updated the versions of ASM, ByteBuddy, and Mockito deps in CASSANDRA-17835. Should be totally transparent to end users. Famous last words. ;)
- And a lot of bugfix / improvements + test fixes

And that's a wrap for this round.

~Josh