Cassandra project biweekly status update 2021-12-14

[Joshua McKenzie <jm...@apache.org>]

Happy Tuesday Morning and Afternoon, which is almost Monday!

Benjamin took the time to put together a roadmap (Awesome!) for the next
Cassandra release - check that out here:
https://cwiki.apache.org/confluence/display/CASSANDRA/Roadmap


[New contributors getting started]
We have a seasonally calibrated set of new tickets to get started on - the
Advent Calender!
https://issues.apache.org/jira/issues/?jql=project%20%3D%20CASSANDRA%20AND%20labels%20%3D%20AdventCalendar2021%20.
If you're new to the project, these are hand-curated tickets for you to
dive into, and you can reach out in the #cassandra-dev channel on
the-asf.slack.com asking for assistance from any of the 13 designated
cassandra mentors to help you out!

We have two other options for getting started if none of the above catch
your fancy: Failing tests, or starter tickets we label "lhf" (low hanging
fruit).

Query for failing tests:
https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=496&quickFilter=2252
Query for unassigned starter tickets:
https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=484&quickFilter=2162&quickFilter=2160

We're holding steady at 22 JIRA tickets that are unassigned test failures.

For unassigned lhf, we have 12 on the patch release (4.0.2) and 17 on the
minor release (4.1.0), some of which are likely overlapping with the
selected tickets from the Advent calendar above. Feel free to self-select
from any of the above lists or reach out on slack if you want some guidance
on where to get involved. No time like the present!


[Dev list discussions in the past 14 days]
https://lists.apache.org/list?dev@cassandra.apache.org:lte=2w:

A highlight from the dev list in the past couple weeks and to reiterate
here: Cassandra as of CASSANDRA-5883 uses logback for logging and is not
vulnerable to the recently disclosed log4j vulnerability. To everyone who
maintains other systems as well as Cassandra and is working through this
unprecedented vulnerability, thank you for everything you do in working to
keep everyone's information safe and secure.

Claude Warren has an open question about building secondary indexes; anyone
with expertise in that area with a few spare cycles, please reach out.
https://lists.apache.org/thread/bo83pxzfy4m9h45m242b38o20c4fdsyz

A discussion indicated consensus on backporting some changes to the 4.0
line for major improvements to behavior under contention when inserting on
collections. Benjamin is carrying the torch on reviewing and porting these
changes to 4.0.
https://lists.apache.org/thread/f3dl7rfc2kv9f5r9pxzyz6zojsss81b9. While the
changes are fairly extensive, this is in a well exercised area of the
codebase and is expected to be a fairly safe, and needed, change on the 4.0
line.

The last thing that comes to mind -  I'd love to get more feedback on the
topic of keeping a releasable trunk and our quality on the project. A lot
of valuable things have come out of the discussion thus far and I'm working
on a PR for those (script to simplify various circle CI operations and
config changes, changes to web site around testing documentation, changes
to wiki for build lead role), however the implications of our merge
strategy discussion on our CI integration proved to be a point without a
clear initial consensus. Curious to hear what other people think:
https://lists.apache.org/thread/8xt4tqb3w4j3jyxj859o3vs8f5xjgqg8


[Tickets Closed in the past 14 days]
https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=484&quickFilter=2175

For 4.0.2, test fixes and doc changes. Low drama, nothing major here.

On 4.1.0, the ticket count closed is misleading (as is so often the case).
Yifan introduced a new mechanism to CDC that'll allow for rolling the
hard-links based on hitting cap rather than pushing exceptions back to the
client if you hit saturation on CDC segments. Benedict merged in the CEP-10
Simulator changes so we can expect to see some interesting things showing
up in the not-so-distant future regarding correctness on our system from
the new testing paradigms this enables. Another big merge in the past two
weeks was Guardrails for CEP-3 that Andres got in. There's a lot of
movement on guardrail related tickets as well going on actively now;
exciting to see this make it into the project.

Last on this list - we've had repeated LEAK messages show up on resources,
often from histograms being fragile and prone to overflow, that should
hopefully be plugged by CASSANDRA-17174. The consequences of these LEAK
messages are far lower than our historical resource leakage before Stefania
and Benedict introduced these mechanisms as they're appropriately harvested
on GC, but there is the possibility that resource handles will be held for
extended durations in edge conditions if you see the LEAK DETECTED message
show up in your logs. It's worth noting: any time you grab handles to
resources that will eventually be cleaned up by a Transactional like
LifecycleTransaction or any other non-try-with-resources approach, be extra
cautious between the time you grab that handle and you register it with the
entity that'll clean it up. This is what we ran into here and remains a
risk for future modifications if you're not defensive.


[Tickets that need Reviewers]
Needs Reviewer:
https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=484&selectedIssue=CASSANDRA-16547&quickFilter=2259

We're down from 14 to 10 tickets needing review on 4.0.2, and down from 34
to 30 on 4.1.0. Trending in the right direction here!


[Test Failure Trendlines]
Butler: https://butler.cassandra.apache.org/#/
We're holding steady with quite a few failures:
* 3.0:    30
* 3.11:   40
* 4.0:    11
* trunk:  20

The state of our CI is what led me to open the thread about Releasable
Trunk and Quality; my hope is some of the ideas we came up with will at
least hold us steady at this number of failures and not further regress
without individual heroics.

JIRA:
https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=496&view=reporting&chart=cumulativeFlowDiagram&swimlane=1233&swimlane=1234&column=2195&column=2196&column=2197&days=90

No real appreciable change in number of open JIRA tickets for test
failures; this may change as we start spinning up the Build Lead role next
year.

Jenkins CI:
https://ci-cassandra.apache.org/job/Cassandra-trunk/lastCompletedBuild/testReport/history/
This is redundant w/butler report above; will drop this from future emails.

And that's it for this round. Wishing a happy and safe holidays to
everyone; I'm going to be taking some time off with the family over the
next couple of weeks so unless someone else wants to pick up a special
Christmas edition of this email, I'll see everyone in 2022 (... that feels
weird to type)!

/wave

~Josh