Cassandra project biweekly status update 2022-04-19

[Josh McKenzie <jm...@apache.org>]

Guess this is happening on Tuesdays now. ;)

[New contributor Getting Started]
We're excited to have you join us! We've setup a couple places for new contributors to get started: either failing tests or starter tickets we label "lhf" (low hanging fruit). Either category provides a smaller scoped onramp to start learning the codebase, our project specific processes, and the general open-source and Apache Way.

We use JIRA for our work tracking, and here are a couple saved Kanban boards to visualize the outstanding workloads in question:
Unassigned failing tests (68 currently unassigned): https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=496&quickFilter=2252
Unassigned starter tickets (25 unassigned): https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=484&quickFilter=2162&quickFilter=2160. The 4.0.x swim lane is for our latest stable GA release and the 4.x swim lane covers the upcoming 4.1 major release we'll be freezing in May.

Join us on slack at the apache server at https://the-asf.slack.com, channel #cassandra-dev, and feel free to reach out to the @cassandra_mentors alias to get in touch with volunteers who are interested in helping new folks get up to speed. Looking forward to seeing you there!


[Dev list Digest]
https://lists.apache.org/list?dev@cassandra.apache.org:lte=2w:

The Apache Foundation in conjunction with the PMC will be hosting a 2 day Cassandra track at ApacheCon North America; this will take place October 3-6 at the Sheraton Hotel in New Orleans. The CFP is open now and will run through May 23rd for talks on the Cassandra Track - please see Mick's email here for details: https://lists.apache.org/thread/2g3pvzt6gzwwojntwfj8dy6fy60jqc3s. It's been a challenging couple of years; I know I for one will be excited to connect with folks in person there. If you have something you did on the project, something you're thinking about doing, a fascinating use-case, anything you think would be interesting to your fellow Cassandra practitioners, please don't hesitate to submit a proposal for it; these events are only as good as we collectively make them and we have a history of having really great events, so let's not drop the ball now! :)

There will also be a performance engineering track at ApacheCon for the first time this year. With the scale a lot of us run Cassandra at and the data velocity, I'm sure a lot of folks would be really interested to hear about our experiences, lessons learned, and innovations in this space. See more here: https://www.linkedin.com/pulse/call-papers-performance-engineering-track-apachecon-paul-brebner/

We have another week going by with no volunteer for the Build Lead role. A reminder: the wiki for what the role entails can be found here: https://cwiki.apache.org/confluence/display/CASSANDRA/Build+Lead. The Build Lead role thus far has been instrumental in us keeping on top of our CI debt and at least holding steady with it; in the run up to the freeze and 4.1 release, it's going to become quite rapidly apparent if we aren't keeping up with introduction of new regressions. I'd hate to see us miss our date for 4.1 due to messy CI.

Chris is looking for reviewers on his "Multi-value data types and two gotchas to avoid" blog post here: https://lists.apache.org/thread/6f27kzt2rgc45odzwc15jybkj7n23qn3

The test artifact for 4.0.4 is available to kick the tires; see Mick's email here: https://lists.apache.org/thread/7kwg5sonxgg9owd2md3yy91tlbtkyhhq. This release has roughly 30 patches in it so there's a good bit of content queued up: https://issues.apache.org/jira/issues/?jql=project%20%3D%20cassandra%20and%20resolution%20!%3D%20unresolved%20and%20fixversion%20%3D%204.0.4%20order%20by%20priority%20desc

David brought up that we have some non-standard java classes exposed via JMX which can lead to issues if clients don't have the Cassandra jar's available. Probably something we want to avoid introducing more of in the future; see more here: https://lists.apache.org/thread/j7v75sxncm05dgc2wch84lkkh224lyz7


[CI Trends]
Butler dashboard: https://butler.cassandra.apache.org/#/
Diffing our 3 weeks of historical data again, we have:

trunk:  12 -> 18
4.0:    16 -> 0 (!)
3.11:   17 -> 16
3.0:    17 -> 6
Sum:    62 -> 40

Definitely trending in the direction we want in aggregate. Zooming in a bit on trunk as we approach the 4.1 freeze (https://butler.cassandra.apache.org/#/ci/upstream/compare/Cassandra-trunk/trunk), we can see that there's 4 consistent failures we can iron out between test_oversized_mutation and CasCriticalSectionTest, however the rest of the test report here implies pretty broad instability and flakiness across a large number of tests. What this means in practice is that number "18" is likely significantly higher (30+) if we're taking into account all failures within the past 5 runs for example. I'll open up a dedicated thread to discuss this when we hit the freeze; regardless, it's taking a lot of time and energy to hold us steady where we are which I appreciate from everyone. Hopefully we can put some practices in place and automation when we get the test failures to 0 to help incentivize keeping them there.


[Release progress]
https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=484&quickFilter=2175

4.0.4:
6 issues closed out in the past 2 weeks. Mostly test fixes and documentation; no big rocks to call out.

4.1.0:
We've closed out 18 issues again in 4.1.0 for this past two weeks. Quite a few different new guardrails went in (C-17498, 17504, 17509, 17522), and a new JMX endpoint to temporarily increase logging for blocking Read Repairs went in as well which should help operators debug certain scenarios (CASSANDRA-17471). The Global on/off flag for guardrails was removed; now it's on a per-guardrail basis only (CASSANDRA-17499). Some good house-cleaning has been going on with test failure fixes, removal of the unused cassandra-stress server (CASSANDRA-17535), and some great work on tidying up our python code and pylint issues (CASSANDRA-17546).

Looking good, and looking forward to the freeze and 4.1 hitting. Keep up the great work everyone!

~Josh