Cassandra project status update 2022-11-30

[Josh McKenzie <jm...@apache.org>]

The Cassandra 4.1-rc1 is out - give it a whirl and see how things go: https://downloads.apache.org/cassandra/4.1-rc1/

Top level, I want to call attention to the CFP for the Cassandra Summit. Patrick McFadin and Scott Hirleman reached out about proposing a talk; we're at 23 submissions with a target of over 100 to get sufficient volume to fill the two days of tracks. Deadline is December 11th, so please take a bit to put together a CFP if you're planning on attending the summit and have something you'd be interested in sharing with the community: https://lists.apache.org/thread/sbn6q6pop537l69vvrj0vz9j6xt6w24w

In terms of work in flight and where we need some help, we have a few tickets that need committer attention: https://issues.apache.org/jira/issues/?jql=project%20%3D%20CASSANDRA%20and%20resolution%20%3D%20unresolved%20and%20status%20%3D%20%22Needs%20Committer%22

CASSANDRA-17997: Improve git branch handling for CircleCI generate.sh
CASSANDRA-17861: Update Python test framework from nose to pytest in CCM
CASSANDRA-14930: decommission may cause timeout because messaging backlog is cleared

If you're a committer with some spare minutes please take a look at one of the above and see if you can help unstick them.

In terms of 4.1 status, we're in RC so ideally nothing will get merged and we'll go to GA in the near future. Great work everyone!

And in other tickets that could use some attention, we have a number of tickets (19 on 4.0.x and 36 on 4.x) that could use reviewers - either they're patch available with no reviewer or in progress without one: https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=484&selectedIssue=CASSANDRA-17251&quickFilter=2259


[New Contributors Getting Started]
We have a curated list of tickets on the core Cassandra codebase we've flagged as being good starter tickets - we currently have 12 of them unassigned on our current patch release version and the link can be found here: https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=484&quickFilter=2162&quickFilter=2454&quickFilter=2160

Another good option if you're looking to engage with the ecosystem, we have the official Cassandra Sidecar JIRA and open issues can be found here: https://issues.apache.org/jira/issues/?jql=project%20%3D%20CASSANDRASC%20AND%20resolution%20%3D%20Unresolved%20ORDER%20BY%20assignee%20DESC%2C%20priority%20DESC%2C%20updated%20DESC. You can expect this codebase to be considerably smaller and easier to get started on than Cassandra proper, so depending on your appetite and/or expertise this might be a good starting point as well.

The project can be cloned from the github repo here: https://github.com/apache/cassandra-sidecar

If you want to just peruse the backlog of open and unassigned tickets on our most recent releases, you can find these tickets here: https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=484&quickFilter=2160. 49 issues unassigned in 4.0.x (bugfix or improvement only) and 313 open unassigned in 4.x, so there's a lot to choose from.

We hang out in #cassandra-dev on https://the-asf.slack.com and there's a @cassandra_mentors alias you can use to reach a bunch of us that have volunteered to help newcomers get situated. If you need an invite to the slack channel feel free to reply to just me on this email and I'll get you set up.

Here's reference explaining the various types of contribution: https://cassandra.apache.org/_/community.html#how-to-contribute
An overview of the C* architecture: https://cassandra.apache.org/doc/latest/cassandra/architecture/overview.html
The getting started contributing guide: https://cassandra.apache.org/_/development/index.html


[Dev list Digest]
https://lists.apache.org/list?dev@cassandra.apache.org:lte=23d:
23 days instead of the previous 26; the Thanksgiving US holiday week kind of set things off by a week for me. :)

Maxim Muzafarov reached out about a fairly broad collection of hygiene, linting, and build-related topics here: https://lists.apache.org/thread/11j0hrv2bkx60xk7zvlgqgjwo982qv6h. There's been a lot of good discussion on this thread - check it out!

Benjamin Lerer proposed the addition of the Big-Math library to the project here: https://lists.apache.org/thread/k3q4f2fdmr5j4vjx1drqct4075sv38xt. Small/early consensus appears to be acceptance w/acknowledgement that we may have to take on maintenance of it down the road someday if it gets abandoned, but the license, quality, and current state of the library looks solid.

Branimir opened up CEP-25, Big Trie-indexed SSTable format, for discussion: https://lists.apache.org/thread/3dpdg6dgm3rqxj96cyhn58b50g415dyh.

We had some interesting back and forth about default GC, how we validate them, and when during our release cycle we validate them. While G1 didn't make the cut for 4.1 due to timing, it looks like a topic we're going to revisit for our next feature release: https://lists.apache.org/thread/j3gwc09ffxg2tyylgs2frhvrj4q85h93

Andres de la Pena brought up the topic of how we name CQL functions: https://lists.apache.org/thread/k9ml1k4fg6o7mfby1nr3y0mnq9r90dym

And last but not least, David Capwell brought up the addition of SpotBugs to our build: https://lists.apache.org/thread/1ro1mvkpvt4vr24nw7dbpdlxo82mq3hz. There's been a lot of chatter about our build system floating around both on slack and on the dev list, so expect to see more coming out on that topic in the relatively near future.


[ASF CI Trends]
https://butler.cassandra.apache.org/#/

Here's our trends on our branches for the last 26 days:

3.0: 10 -> 16
3.11: 11 -> 17
4.0: 2 -> 5
4.1: 9 -> 4
trunk: 21 -> 3

We're vetting 4.1 on circle, however ASF CI is looking pretty good (for 4.0+) at this point. If you check the butler page you can see that 3.11, 4.0, 4.1, and trunk are all trending down from a bump around Nov 11th.


[Release progress]
https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=484&quickFilter=2278

4.1 rc / ga: 3 issues
- Avoid schema mismatch problems on memtable API (CASSANDRA-18040)
- Enable dtest-offheap in CircleCI (CASSANDRA-17950)
- Test fix on the CommitLog (CASSANDRA-17928)

4.0.x: 0 issues unique to this release version

4.X / Next: 11 issues
- Add aggregation and scalar functions on collections (CASSANDRA-18060)
- Make CommitLogReplayer replay all tables in a keyspace rather than having to list all tables (CASSANDRA-18044)
- Add Math functions (CASSANDRA-17221)
- Make incremental backup configurable per table (CASSANDRA-15402)
- Change bin/cqlsh.py shebang to use PATH with env prefix (CASSANDRA-17832)
- Guardrail: allow_filtering_custom_error_message (CASSANDRA-17967)
- Fix CQL aggregation functions for collections, tuples, and UDTs (CASSANDRA-17811)
- nodetool/rebuild: Add flag to exclude nodes from local datacenter (CASSANDRA-17870)
- Replace cqlshlib/wcwidth.py with pypi module 'wcwidth' (CASSANDRA-17287)

Thanks everyone! It's been a long hard road to 4.1 (all relative; looking at you 4.0 :) ), but the work we're putting in now to stabilize our tests and environment is only going to pay off as we approach our next feature release next year.

~Josh