[DISCUSSION] Cassandra's code style and source code analysis

[Maxim Muzafarov <mm...@apache.org>]

Hello everyone,


First of all, thank you all for this awesome project which I have
often been inspired by. My name is Maxim Muzafarov I'm a Committer and
PMC of Apache Ignite hence you most likely don't know me as I come
from another part of the ASF. Perhaps, I did almost the same things
with B-Trees, off-heap memory management, rebalancing, checkpointing,
snapshotting, and IEPs (you are calling it CEPs) but on a slightly
different distributed database architecture.

Nevertheless,

I was chasing down for my first issue to get experience with Cassandra
and found a bunch of opened JIRAs related to the source code analysis
(static analysis as well as the code style). These issues still appear
in JIRA from time to time [1][2][3][4]. It seems to me there not
enough attention has been paid to this topic and all possible options
for this analysis and code style haven't been widely discussed before.
I'd like to summarize everything that I have found and offer my skills
and my experience for solving some of such issues we'll agree on.


= Motivation =

The goal is to make small contributions easier and safer to apply with
GitHub PRs for both a contributor and a committer by adding automation
code checks for each new Cassandra contribution. This also will help
to reduce the time required for reviewing and applying such PRs by an
experienced developer.

As you may know, the infrastructure team has disabled public sign-up
to ASF JIRA (the GitHub issues are recommended instead). Thus the
following things become more important if we are still interested in
attracting new contributions as it was discussed earlier [6].

I do not want to add extra steps to the existing workflow with code
review or make GitHub pull requests as default for patches as it also
was discussed already [7], just to improve the automation checks in it
and make checks more convenient.


= Proposed Solution =

== 1. Make the checkstyle config a single point of truth for the
source code style. ==

The checkstyle is already used and included in the Cassandra project
build lifecycle (ant command line, Jenkins, CircleCI). There is no
need to maintain code style configurations for different types of IDEs
(e.g. IntelliJ inspections configuration) since the checkstyle.xml
file can be directly imported to IDE used by a developer. This is fair
for Intellij Idea, NetBeans, and Eclipse.

So, I propose to focus on the checks themselves and checking pull
requests with automation scripts, rather than maintaining these
integrations. The benefits here are avoiding all issues with
maintaining configurations for different IDEs. Another good advantage
of this approach would be the ability to add new checkstyle rules
without touching IDE configuration - and such tickets will be LFH and
easy to commit.

The actions points here are:

- create an umbrella JIRA ticket for all checkstyle issues e.g. [8]
(or label checkstyle);
- add checkstyle to GitHub pull requests using GitHub actions (execute
ant command);
- include checkstyle to the build (already done);
- remove redundant code style configurations related to IDEs from the
source code e.g. [9];


== 2. Add additional tooling for code analysis to the build and GitHub
pull requests. ==

The source code static analysis and automated checks have been
discussed recently in the "SpotBugs to the build" topic [10]. I'd like
to add my 50 cents here.

Before discussing the pros and cons of each solution, let's focus on
the criteria that such solutions must meet. You can find the most
complete list of such tooling here [11].

From my point of view, the crucial criteria are:
- free for open-source (at least licenses should allow such usages);
- popularity in the ASF ecosystems;
- convenient integration and/or plugins for IDEs and GitHub;
- we must be able to integrate with CirleCI, and Jenkins as well as
launch from a command line;


=== Sonar ===

pros
+ this tool is free for open-source and recommended by the ASF
infrastructure team [12];
+ was already used for the Cassandra project some time ago at
sonarcloud.io [13];
+ it has GitHub pull requests analysis [14];

cons
- run locally requires additional configuration and TOKEN_ID due to
the analysis results stored in the ext database (infra will not
provide it for local usage);

=== SpotBugs (FindBugs) ===

pros
+ license is allowed to use it and run it as a library (should be legal for us);
+ it analyses the bytecode that differs from how the checkstyle works;
+ can be executed from the command line as well as integrated into the build;

cons
- requires compiled source code;

=== PMD ===

pros
+ BSD licenses more permissive than LGPL (SpotBugs);
+ analyses the source code like the checkstyle does;
+ have an extended rule sets for source code analyses;

cons
- the checkstyle is already used in the project, and should be enough for now;

=== IntelliJ IDEA ===

pros
+ free for open-source and can be used from the command line [15];
+ local checks are performed with the same code style configuration as
the IDE does during development;

cons
- inconvenient for different IDEs (Eclipse, NetBeans);


As a summary for this section:

There is no need to limit ourselves to the choice of such tools and
pick up only one from this list.

For instance, it seems it will be better to configure sonarcloud.io
reports on each push commit to the trunk. Sonar already has built-in
rules (no need to maintain them) and this is easy to add with the ASF
infra team install guides using GitHub actions or Jenkins. We will be
able to see immediately the source code trends as well as new good
candidates for LFH issues to fix.

In addition to Sonar reports, we can add SpotBugs (PMD is more like an
'improved' checkstyle, but for now it is more than enough and there is
no need to maintain another set of rules) to the build and GitHub pull
requests with limited configuration for the first time. Thus we will
be able to add new rules one by one.


The actions points here are:

- configure Sonar reports for the trunk (using GA or Jenkins);
- add SpotBugs to the build;
- add integration SpotBugs with GitHub pull requests;


== 3. Enable pushing backwards build results for both Jenkins and
CircleCI to GitHub pull requests. ==

The goal here is to have a "green checkbox" for those GitHub pull
requests that have corresponding Jenkins/CircleCI runs. According to
the following links, it is completely possible to have those.

https://github.com/jenkinsci/github-branch-source-plugin
https://circleci.com/docs/enable-checks/

Moreover, the GitHub Branch Source Plugin is already enabled for the
Cassandra project [16]. The same seems should work the same way for
CirleCI, but I have faced the infrastructure team comment [17] that
describes admin permissions are required for that, so the question is
still open here. I could dig a bit more once we've agreed on it.

The actions points here are:
- enable Jenkins integration for GitHub pull requests;
- enable CircleCI integration for GitHub pull requests;


== 4. Review IDE configs that are not used by the community. ==

Sorry for bumping up such a holy topic with IDEs that are being used
by developers, so please do not judge me strictly against it. My main
point here is "IDE should serve the source code" rather than "the
source code adapts to IDE", thus it will probably become easy to use
the project sources in an IDE once the migration from Ant to
Maven/Gradle occurs [5], but it is still possible to make things a bit
simpler today.

Some of the configuration IDE scripts still consume some time for
their maintenance and require to be tested on each new release occurs.
These files must be always up to date if a configuration file
structure changed in a new IDE version and this is inconvenient. I
don't propose removing such scripts, just a review of what IDEs are
exactly using.

For instance,

Cassandra + Eclipse doesn't seem to appear so often on StackOverflow.
It could mean only two things: everything is working fine or no one is
using it (hope the first one).
https://stackoverflow.com/questions/tagged/cassandra%20eclipse?sort=Newest

Cassandra + NetBeans appear even less frequently and the same questions occur.
https://stackoverflow.com/questions/tagged/cassandra%20netbeans?sort=Newest

The actions points here are:

- initiate a wide survey for user and dev lists, to get to know about
the usages;
- remove those configurations that are not used anymore;
- force migration from Ant to Gradle/Maven;



Summarizing everything proposed above I think it is possible to
simplify adding small contributions easier to the codebase as well as
save a bunch of committer's time.

So,
WDYT about the things described above?
Should I create a CEP for this?



[1] https://issues.apache.org/jira/browse/CASSANDRA-9209 (Add static
analysis to report any AutoCloseable objects)
[2] https://issues.apache.org/jira/browse/CASSANDRA-17400 (Fail build
or warn when closeable reference is not closed in tests)
[3] https://issues.apache.org/jira/browse/CASSANDRA-9919 (Cleanup
closeable iterator usage)
[4] https://issues.apache.org/jira/browse/CASSANDRA-17925 (Java source
code should have sorted imports as defined in the codestyle)
[5] https://issues.apache.org/jira/browse/CASSANDRA-17015 (Migrate
build from Ant to Maven/Gradle)
[6] https://lists.apache.org/thread/1t3hr1pylhpgc5c9d2382xtnw3wgkg06
(Attracting new contributors)
[7] https://lists.apache.org/thread/b3nowtrlpjwr46vftsb3ob38xqnrnxl1
(Switch to using GitHub pull requests?)
[8] https://issues.apache.org/jira/browse/CASSANDRA-17925 (Java source
code should have sorted imports as defined in the codestyle)
[9] https://github.com/apache/cassandra/tree/trunk/ide/idea/inspectionProfiles
[10] https://lists.apache.org/thread/1ro1mvkpvt4vr24nw7dbpdlxo82mq3hz
(Add SpotBugs to the build)
[11] https://en.wikipedia.org/wiki/List_of_tools_for_static_code_analysis
[12] https://cwiki.apache.org/confluence/display/INFRA/SonarCloud+for+ASF+projects
[13] https://sonarcloud.io/summary/overall?id=cassandra_180410
[14] https://docs.sonarqube.org/latest/analysis/pull-request/
[15] https://www.jetbrains.com/help/idea/command-line-code-inspector.html
[16] https://cwiki.apache.org/confluence/display/INFRA/ci-cassandra.apache.org
[17] https://issues.apache.org/jira/browse/INFRA-22367?focusedCommentId=17421230

Re: [DISCUSSION] Cassandra's code style and source code analysis

[Jon Haddad <ru...@apache.org>]

So much awesome here.  Big +1 to having checkstyle be the source of truth. 

On 2022/11/24 17:10:28 Maxim Muzafarov wrote:
> Hello everyone,
> 
> 
> First of all, thank you all for this awesome project which I have
> often been inspired by. My name is Maxim Muzafarov I'm a Committer and
> PMC of Apache Ignite hence you most likely don't know me as I come
> from another part of the ASF. Perhaps, I did almost the same things
> with B-Trees, off-heap memory management, rebalancing, checkpointing,
> snapshotting, and IEPs (you are calling it CEPs) but on a slightly
> different distributed database architecture.
> 
> Nevertheless,
> 
> I was chasing down for my first issue to get experience with Cassandra
> and found a bunch of opened JIRAs related to the source code analysis
> (static analysis as well as the code style). These issues still appear
> in JIRA from time to time [1][2][3][4]. It seems to me there not
> enough attention has been paid to this topic and all possible options
> for this analysis and code style haven't been widely discussed before.
> I'd like to summarize everything that I have found and offer my skills
> and my experience for solving some of such issues we'll agree on.
> 
> 
> = Motivation =
> 
> The goal is to make small contributions easier and safer to apply with
> GitHub PRs for both a contributor and a committer by adding automation
> code checks for each new Cassandra contribution. This also will help
> to reduce the time required for reviewing and applying such PRs by an
> experienced developer.
> 
> As you may know, the infrastructure team has disabled public sign-up
> to ASF JIRA (the GitHub issues are recommended instead). Thus the
> following things become more important if we are still interested in
> attracting new contributions as it was discussed earlier [6].
> 
> I do not want to add extra steps to the existing workflow with code
> review or make GitHub pull requests as default for patches as it also
> was discussed already [7], just to improve the automation checks in it
> and make checks more convenient.
> 
> 
> = Proposed Solution =
> 
> == 1. Make the checkstyle config a single point of truth for the
> source code style. ==
> 
> The checkstyle is already used and included in the Cassandra project
> build lifecycle (ant command line, Jenkins, CircleCI). There is no
> need to maintain code style configurations for different types of IDEs
> (e.g. IntelliJ inspections configuration) since the checkstyle.xml
> file can be directly imported to IDE used by a developer. This is fair
> for Intellij Idea, NetBeans, and Eclipse.
> 
> So, I propose to focus on the checks themselves and checking pull
> requests with automation scripts, rather than maintaining these
> integrations. The benefits here are avoiding all issues with
> maintaining configurations for different IDEs. Another good advantage
> of this approach would be the ability to add new checkstyle rules
> without touching IDE configuration - and such tickets will be LFH and
> easy to commit.
> 
> The actions points here are:
> 
> - create an umbrella JIRA ticket for all checkstyle issues e.g. [8]
> (or label checkstyle);
> - add checkstyle to GitHub pull requests using GitHub actions (execute
> ant command);
> - include checkstyle to the build (already done);
> - remove redundant code style configurations related to IDEs from the
> source code e.g. [9];
> 
> 
> == 2. Add additional tooling for code analysis to the build and GitHub
> pull requests. ==
> 
> The source code static analysis and automated checks have been
> discussed recently in the "SpotBugs to the build" topic [10]. I'd like
> to add my 50 cents here.
> 
> Before discussing the pros and cons of each solution, let's focus on
> the criteria that such solutions must meet. You can find the most
> complete list of such tooling here [11].
> 
> From my point of view, the crucial criteria are:
> - free for open-source (at least licenses should allow such usages);
> - popularity in the ASF ecosystems;
> - convenient integration and/or plugins for IDEs and GitHub;
> - we must be able to integrate with CirleCI, and Jenkins as well as
> launch from a command line;
> 
> 
> === Sonar ===
> 
> pros
> + this tool is free for open-source and recommended by the ASF
> infrastructure team [12];
> + was already used for the Cassandra project some time ago at
> sonarcloud.io [13];
> + it has GitHub pull requests analysis [14];
> 
> cons
> - run locally requires additional configuration and TOKEN_ID due to
> the analysis results stored in the ext database (infra will not
> provide it for local usage);
> 
> === SpotBugs (FindBugs) ===
> 
> pros
> + license is allowed to use it and run it as a library (should be legal for us);
> + it analyses the bytecode that differs from how the checkstyle works;
> + can be executed from the command line as well as integrated into the build;
> 
> cons
> - requires compiled source code;
> 
> === PMD ===
> 
> pros
> + BSD licenses more permissive than LGPL (SpotBugs);
> + analyses the source code like the checkstyle does;
> + have an extended rule sets for source code analyses;
> 
> cons
> - the checkstyle is already used in the project, and should be enough for now;
> 
> === IntelliJ IDEA ===
> 
> pros
> + free for open-source and can be used from the command line [15];
> + local checks are performed with the same code style configuration as
> the IDE does during development;
> 
> cons
> - inconvenient for different IDEs (Eclipse, NetBeans);
> 
> 
> As a summary for this section:
> 
> There is no need to limit ourselves to the choice of such tools and
> pick up only one from this list.
> 
> For instance, it seems it will be better to configure sonarcloud.io
> reports on each push commit to the trunk. Sonar already has built-in
> rules (no need to maintain them) and this is easy to add with the ASF
> infra team install guides using GitHub actions or Jenkins. We will be
> able to see immediately the source code trends as well as new good
> candidates for LFH issues to fix.
> 
> In addition to Sonar reports, we can add SpotBugs (PMD is more like an
> 'improved' checkstyle, but for now it is more than enough and there is
> no need to maintain another set of rules) to the build and GitHub pull
> requests with limited configuration for the first time. Thus we will
> be able to add new rules one by one.
> 
> 
> The actions points here are:
> 
> - configure Sonar reports for the trunk (using GA or Jenkins);
> - add SpotBugs to the build;
> - add integration SpotBugs with GitHub pull requests;
> 
> 
> == 3. Enable pushing backwards build results for both Jenkins and
> CircleCI to GitHub pull requests. ==
> 
> The goal here is to have a "green checkbox" for those GitHub pull
> requests that have corresponding Jenkins/CircleCI runs. According to
> the following links, it is completely possible to have those.
> 
> https://github.com/jenkinsci/github-branch-source-plugin
> https://circleci.com/docs/enable-checks/
> 
> Moreover, the GitHub Branch Source Plugin is already enabled for the
> Cassandra project [16]. The same seems should work the same way for
> CirleCI, but I have faced the infrastructure team comment [17] that
> describes admin permissions are required for that, so the question is
> still open here. I could dig a bit more once we've agreed on it.
> 
> The actions points here are:
> - enable Jenkins integration for GitHub pull requests;
> - enable CircleCI integration for GitHub pull requests;
> 
> 
> == 4. Review IDE configs that are not used by the community. ==
> 
> Sorry for bumping up such a holy topic with IDEs that are being used
> by developers, so please do not judge me strictly against it. My main
> point here is "IDE should serve the source code" rather than "the
> source code adapts to IDE", thus it will probably become easy to use
> the project sources in an IDE once the migration from Ant to
> Maven/Gradle occurs [5], but it is still possible to make things a bit
> simpler today.
> 
> Some of the configuration IDE scripts still consume some time for
> their maintenance and require to be tested on each new release occurs.
> These files must be always up to date if a configuration file
> structure changed in a new IDE version and this is inconvenient. I
> don't propose removing such scripts, just a review of what IDEs are
> exactly using.
> 
> For instance,
> 
> Cassandra + Eclipse doesn't seem to appear so often on StackOverflow.
> It could mean only two things: everything is working fine or no one is
> using it (hope the first one).
> https://stackoverflow.com/questions/tagged/cassandra%20eclipse?sort=Newest
> 
> Cassandra + NetBeans appear even less frequently and the same questions occur.
> https://stackoverflow.com/questions/tagged/cassandra%20netbeans?sort=Newest
> 
> The actions points here are:
> 
> - initiate a wide survey for user and dev lists, to get to know about
> the usages;
> - remove those configurations that are not used anymore;
> - force migration from Ant to Gradle/Maven;
> 
> 
> 
> Summarizing everything proposed above I think it is possible to
> simplify adding small contributions easier to the codebase as well as
> save a bunch of committer's time.
> 
> So,
> WDYT about the things described above?
> Should I create a CEP for this?
> 
> 
> 
> [1] https://issues.apache.org/jira/browse/CASSANDRA-9209 (Add static
> analysis to report any AutoCloseable objects)
> [2] https://issues.apache.org/jira/browse/CASSANDRA-17400 (Fail build
> or warn when closeable reference is not closed in tests)
> [3] https://issues.apache.org/jira/browse/CASSANDRA-9919 (Cleanup
> closeable iterator usage)
> [4] https://issues.apache.org/jira/browse/CASSANDRA-17925 (Java source
> code should have sorted imports as defined in the codestyle)
> [5] https://issues.apache.org/jira/browse/CASSANDRA-17015 (Migrate
> build from Ant to Maven/Gradle)
> [6] https://lists.apache.org/thread/1t3hr1pylhpgc5c9d2382xtnw3wgkg06
> (Attracting new contributors)
> [7] https://lists.apache.org/thread/b3nowtrlpjwr46vftsb3ob38xqnrnxl1
> (Switch to using GitHub pull requests?)
> [8] https://issues.apache.org/jira/browse/CASSANDRA-17925 (Java source
> code should have sorted imports as defined in the codestyle)
> [9] https://github.com/apache/cassandra/tree/trunk/ide/idea/inspectionProfiles
> [10] https://lists.apache.org/thread/1ro1mvkpvt4vr24nw7dbpdlxo82mq3hz
> (Add SpotBugs to the build)
> [11] https://en.wikipedia.org/wiki/List_of_tools_for_static_code_analysis
> [12] https://cwiki.apache.org/confluence/display/INFRA/SonarCloud+for+ASF+projects
> [13] https://sonarcloud.io/summary/overall?id=cassandra_180410
> [14] https://docs.sonarqube.org/latest/analysis/pull-request/
> [15] https://www.jetbrains.com/help/idea/command-line-code-inspector.html
> [16] https://cwiki.apache.org/confluence/display/INFRA/ci-cassandra.apache.org
> [17] https://issues.apache.org/jira/browse/INFRA-22367?focusedCommentId=17421230
> 

Re: [DISCUSSION] Cassandra's code style and source code analysis

[Benedict <be...@apache.org>]

> == 1. Make the checkstyle config a single point of truth for the
> source code style. ==

-1, the single point of truth is the wiki - everything else follows this. The code style is a community consensus, and as previously discussed the last time it came up, mostly not strict. If we intend to impose hard checks here that reject patches we will need to debate carefully which elements of the code style are to become hard rules.

> There is no
> need to maintain code style configurations for different types of IDEs
> (e.g. IntelliJ inspections configuration) since the checkstyle.xml
> file can be directly imported to IDE used by a developer. This is fair
>> 
>> for Intellij Idea, NetBeans, and Eclipse.

Does this update IntelliJ’s own linter, or does a plugin run check style over the codebase? I couldn’t find a native import, and the latter is probably a downgrade for most active developers.

>> == 2. Add additional tooling for code analysis to the build and GitHub
>> pull requests. ==


I’m generally in favour of carefully expanding our use of additional linters, but we should activate new lints very cautiously, as not all lints are equally good. I think there needs to be a lot of community consensus on the broad expansion of lints that can reject patches.

> On 24 Nov 2022, at 20:56, Mick Semb Wever <mc...@apache.org> wrote:
> 
>  Thank you for a solid write up Maxim. And welcome to Cassandra, it's
> very positive to see you here.
> 
> I whole-heartedly agree with nearly everything you write. Some input
> and questions inline.
> 
> 
>> 
>> As you may know, the infrastructure team has disabled public sign-up
>> to ASF JIRA (the GitHub issues are recommended instead).
>> 
> 
> 
> I suspect (based on chatter in general, but not on dev@ AFAIK) is to
> avoid GH issues and stick with jira. The sign-up hurdle we will
> document on the website: CASSANDRA-18064
> 
> 
> 
>> == 1. Make the checkstyle config a single point of truth for the
>> source code style. ==
>> 
>> The checkstyle is already used and included in the Cassandra project
>> build lifecycle (ant command line, Jenkins, CircleCI). There is no
>> need to maintain code style configurations for different types of IDEs
>> (e.g. IntelliJ inspections configuration) since the checkstyle.xml
>> file can be directly imported to IDE used by a developer. This is fair
>> for Intellij Idea, NetBeans, and Eclipse.
> 
> 
> Big +1
> 
> 
>> So, I propose to focus on the checks themselves and checking pull
>> requests with automation scripts, rather than maintaining these
>> integrations. The benefits here are avoiding all issues with
>> maintaining configurations for different IDEs. Another good advantage
>> of this approach would be the ability to add new checkstyle rules
>> without touching IDE configuration - and such tickets will be LFH and
>> easy to commit.
>> 
>> The actions points here are:
>> 
>> - create an umbrella JIRA ticket for all checkstyle issues e.g. [8]
>> (or label checkstyle);
>> - add checkstyle to GitHub pull requests using GitHub actions (execute
>> ant command);
> 
> 
> Instead of custom GHA scripting, please use our existing
> cassandra-artifact.sh (which should already include all such checks).
> 
> Something like https://github.com/apache/cassandra/compare/cassandra-3.11...thelastpickle:cassandra:mck/github-actions/3.11
> 
> 
> 
>> == 3. Enable pushing backwards build results for both Jenkins and
>> CircleCI to GitHub pull requests. ==
>> 
>> The goal here is to have a "green checkbox" for those GitHub pull
>> requests that have corresponding Jenkins/CircleCI runs. According to
>> the following links, it is completely possible to have those.
>> 
>> https://github.com/jenkinsci/github-branch-source-plugin
>> https://circleci.com/docs/enable-checks/
>> 
>> Moreover, the GitHub Branch Source Plugin is already enabled for the
>> Cassandra project [16]. The same seems should work the same way for
>> CirleCI, but I have faced the infrastructure team comment [17] that
>> describes admin permissions are required for that, so the question is
>> still open here. I could dig a bit more once we've agreed on it.
>> 
>> The actions points here are:
>> - enable Jenkins integration for GitHub pull requests;
>> - enable CircleCI integration for GitHub pull requests;
> 
> 
> Some folk use CircleCI, some use ci-cassandra. The green checkbox idea
> is great, so long as it's optional. We don't want PRs triggering the
> runs either (there are other mechanisms for triggering for now).
> 
> 
>> The actions points here are:
>> 
>> - initiate a wide survey for user and dev lists, to get to know about
>> the usages;
>> - remove those configurations that are not used anymore;
>> - force migration from Ant to Gradle/Maven;
> 
> 
> Let's leave this out for now. There's too many unknowns here. If
> there's an IDE configuration that's broken, no one has reported it for
> ages, and no one is around to fix it, then I say we should raise the
> discussion to remove it.
> 
> The Gradle/Maven migration is a hot one, contribute to that discussion
> but let's not tangle this work up with it, IMHO.
> 
> Totally agree that IDE project files should be as light weight as possible.
> 
> 
>> Summarizing everything proposed above I think it is possible to
>> simplify adding small contributions easier to the codebase as well as
>> save a bunch of committer's time.
>> 
>> So,
>> WDYT about the things described above?
>> Should I create a CEP for this?
> 
> 
> I see no need for a CEP here. An epic and tickets will work.
> Again, thanks for the input Maxim!

Re: [DISCUSSION] Cassandra's code style and source code analysis

[Benedict <be...@apache.org>]

I should say, if a majority of the core developers are secretly really keen to see one of the other build systems takeover, I won’t stand in the way. But my preference is not to take the productivity hit, and I know many of the core developers prefer to avoid active involvement on list, and I suspect most are at best ambivalent to this. So without evidence of strong support for the migration from those folk, I’ll continue to voice my concerns as one of the more vocal ones.

> On 25 Nov 2022, at 10:07, Benedict <be...@apache.org> wrote:
> 
> There’s always a handful of people asking for it, but notably few if any of the full time contributors doing the majority of the core development of Cassandra. It strikes me as something very appealing to others, but less so to those wanting to get on with development.
> 
> I never really see a good argument articulated for the migration, besides general hand waving that ant is old, and people like newer build systems. Ant is working fine, so there isn’t a strong technical reason to replace it, and there are good organisational reasons not to.
> 
> Why do you consider a migration inevitable?
> 
> 
> 
>> On 25 Nov 2022, at 09:58, Miklosovic, Stefan <St...@netapp.com> wrote:
>> 
>> Interesting take on Ant / no-Ant, Benedict. I am very curious how this unfolds. My long-term perception is that changing it to something else is more or less inevitable but if there is a broader consensus to not do that .... well.
>> 
>> ________________________________________
>> From: Benedict <be...@apache.org>
>> Sent: Friday, November 25, 2022 10:52
>> To: dev@cassandra.apache.org
>> Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis
>> 
>> NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.
>> 
>> 
>> 
>> 
>> I was in a bit of a rush last night. I should say that I’m of course +1 a general endeavour to clean this up, and to expand our use of linters, and I appreciate your volunteering to help out in this way Maxim.
>> 
>> However, responding to Stefan, I’m pretty -1 migrating from ant to another build system without really good reason. Migration has a real cost to productivity for all existing contributors, and the phantom of increasing new contributions has never paid off historically. I’m all for easing people into participation, but not at penalty to the existing contributor base.
>> 
>> If the only reason is to make it easier to open in a different IDE, we can perhaps have some basic build files outlining code structure for importing, that are compatible with our canonical ant build? We could perhaps even generate them.
>> 
>> 
>>>> On 25 Nov 2022, at 09:35, Miklosovic, Stefan <St...@netapp.com> wrote:
>>> 
>>> For the record, I was testing that same combo Claude mentioned and it did not work out of the box but it is definitely possible to set up successfully. I do not remember the details.
>>> 
>>> To replay to Maxim, it all seems good to me, roughly, but I humbly think it all boils down to Maven/Gradle refactoring and on top of that we can do all else.
>>> 
>>> For example, there is (1) where the solution, besides fixing the tests, is to introduce an Ant task which would check this on build. That being said, how is that going to look like when we change Ant for something else? That stuff suddenly becomes obsolete.
>>> 
>>> This case maybe applies to other problems we want to solve as well. I do not want to do something tailored for one build system just to rewrite it all or to spend significant amount of time on that again when we switch the build system.
>>> 
>>> For that reason I think changing Ant for something else should be top priority (as I understand that it the hot topic for community for very long time) and then everything else should follow. We should spend time on things mentioned only in case they do not collide with any build system at all.
>>> 
>>> (1) https://issues.apache.org/jira/browse/CASSANDRA-17964
>>> 
>>> Stefan
>>> 
>>> ________________________________________
>>> From: Claude Warren, Jr via dev <de...@cassandra.apache.org>
>>> Sent: Friday, November 25, 2022 10:16
>>> To: dev@cassandra.apache.org
>>> Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis
>>> 
>>> NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.
>>> 
>>> 
>>> 
>>> +1 for the concept as a whole.  I am certain I could find nits to pick if I looked deeply.
>>> 
>>> @mck -- I did have a problem with Cassandra + Eclipse + Java11 (Classpath).  I gave up and am spending time trying to learn IntelliJ.  I also mentioned it in one of the discussion areas.
>>> 
>>> Claude
>>> 
>>>> On Thu, Nov 24, 2022 at 8:55 PM Mick Semb Wever <mc...@apache.org>> wrote:
>>> Thank you for a solid write up Maxim. And welcome to Cassandra, it's
>>> very positive to see you here.
>>> 
>>> I whole-heartedly agree with nearly everything you write. Some input
>>> and questions inline.
>>> 
>>> 
>>>> 
>>>> As you may know, the infrastructure team has disabled public sign-up
>>>> to ASF JIRA (the GitHub issues are recommended instead).
>>>> 
>>> 
>>> 
>>> I suspect (based on chatter in general, but not on dev@ AFAIK) is to
>>> avoid GH issues and stick with jira. The sign-up hurdle we will
>>> document on the website: CASSANDRA-18064
>>> 
>>> 
>>> 
>>>> == 1. Make the checkstyle config a single point of truth for the
>>>> source code style. ==
>>>> 
>>>> The checkstyle is already used and included in the Cassandra project
>>>> build lifecycle (ant command line, Jenkins, CircleCI). There is no
>>>> need to maintain code style configurations for different types of IDEs
>>>> (e.g. IntelliJ inspections configuration) since the checkstyle.xml
>>>> file can be directly imported to IDE used by a developer. This is fair
>>>> for Intellij Idea, NetBeans, and Eclipse.
>>> 
>>> 
>>> Big +1
>>> 
>>> 
>>>> So, I propose to focus on the checks themselves and checking pull
>>>> requests with automation scripts, rather than maintaining these
>>>> integrations. The benefits here are avoiding all issues with
>>>> maintaining configurations for different IDEs. Another good advantage
>>>> of this approach would be the ability to add new checkstyle rules
>>>> without touching IDE configuration - and such tickets will be LFH and
>>>> easy to commit.
>>>> 
>>>> The actions points here are:
>>>> 
>>>> - create an umbrella JIRA ticket for all checkstyle issues e.g. [8]
>>>> (or label checkstyle);
>>>> - add checkstyle to GitHub pull requests using GitHub actions (execute
>>>> ant command);
>>> 
>>> 
>>> Instead of custom GHA scripting, please use our existing
>>> cassandra-artifact.sh (which should already include all such checks).
>>> 
>>> Something like https://github.com/apache/cassandra/compare/cassandra-3.11...thelastpickle:cassandra:mck/github-actions/3.11
>>> 
>>> 
>>> 
>>>> == 3. Enable pushing backwards build results for both Jenkins and
>>>> CircleCI to GitHub pull requests. ==
>>>> 
>>>> The goal here is to have a "green checkbox" for those GitHub pull
>>>> requests that have corresponding Jenkins/CircleCI runs. According to
>>>> the following links, it is completely possible to have those.
>>>> 
>>>> https://github.com/jenkinsci/github-branch-source-plugin
>>>> https://circleci.com/docs/enable-checks/
>>>> 
>>>> Moreover, the GitHub Branch Source Plugin is already enabled for the
>>>> Cassandra project [16]. The same seems should work the same way for
>>>> CirleCI, but I have faced the infrastructure team comment [17] that
>>>> describes admin permissions are required for that, so the question is
>>>> still open here. I could dig a bit more once we've agreed on it.
>>>> 
>>>> The actions points here are:
>>>> - enable Jenkins integration for GitHub pull requests;
>>>> - enable CircleCI integration for GitHub pull requests;
>>> 
>>> 
>>> Some folk use CircleCI, some use ci-cassandra. The green checkbox idea
>>> is great, so long as it's optional. We don't want PRs triggering the
>>> runs either (there are other mechanisms for triggering for now).
>>> 
>>> 
>>>> The actions points here are:
>>>> 
>>>> - initiate a wide survey for user and dev lists, to get to know about
>>>> the usages;
>>>> - remove those configurations that are not used anymore;
>>>> - force migration from Ant to Gradle/Maven;
>>> 
>>> 
>>> Let's leave this out for now. There's too many unknowns here. If
>>> there's an IDE configuration that's broken, no one has reported it for
>>> ages, and no one is around to fix it, then I say we should raise the
>>> discussion to remove it.
>>> 
>>> The Gradle/Maven migration is a hot one, contribute to that discussion
>>> but let's not tangle this work up with it, IMHO.
>>> 
>>> Totally agree that IDE project files should be as light weight as possible.
>>> 
>>> 
>>>> Summarizing everything proposed above I think it is possible to
>>>> simplify adding small contributions easier to the codebase as well as
>>>> save a bunch of committer's time.
>>>> 
>>>> So,
>>>> WDYT about the things described above?
>>>> Should I create a CEP for this?
>>> 
>>> 
>>> I see no need for a CEP here. An epic and tickets will work.
>>> Again, thanks for the input Maxim!

Re: [DISCUSSION] Cassandra's code style and source code analysis

[Benedict <be...@apache.org>]

On the topic of code style, the wiki not only covers many elements of style that are impossible to encode in a linter, but explicitly uses terminology on many lintable items that is not “hard.” I should know, I formulated it this way deliberately, and made this explicit in the discussion while shepherding it through a vote - only quite recently.

For both these reason I think it is impossible to make checkstyle the source of truth.

We can vote for a subset of rules we want everyone to follow without any exception, and translate these into a checkstyle.xml without a vote. But:

1) I personally have seen limited value to strict style enforcement on projects where I’ve interacted with it. Expressing code neatly involves a lot of authorship decisions that a linter simply cannot understand.
2) There’s already a broad array of styles in the project, and this is fine. We don’t have to check-in identical looking code, so long as it is all the same flavour. 
3) It would anyway be hugely disruptive to apply to PRs to existing code.

I think if you’re keen on this it would be best to start by selecting your preferred subset of hard rules.

> About the check style as the source of truth for the style guides, I am +1 to this as well… I feel that wiki is a bad place for this and we can use the check style file to generate the wiki text (no idea if there are tools for this…).  About the comment on “hard” requirements… my understanding has been that new changes must follow and I know several reviewers who act this way in the review process…. If we agree on a style, why do we want to keep allowing ignoring it?


> On 28 Nov 2022, at 20:18, David Capwell <dc...@apple.com> wrote:
> 
> I am strong +1 to new linters, I have been working on SpotBugs but not sent a patch due to sickness and holidays…
> 
> About the check style as the source of truth for the style guides, I am +1 to this as well… I feel that wiki is a bad place for this and we can use the check style file to generate the wiki text (no idea if there are tools for this…).  About the comment on “hard” requirements… my understanding has been that new changes must follow and I know several reviewers who act this way in the review process…. If we agree on a style, why do we want to keep allowing ignoring it?
> 
>> There’s always a handful of people asking for it, but notably few if any of the full time contributors doing the majority of the core development of Cassandra.
> 
> 
> I don’t agree with that statement, I know several committers/PMC who want to switch, but every 6 months this gets brought up and gets shot down… so we just give up trying… 
> 
>> I never really see a good argument articulated for the migration, besides general hand waving that ant is old, and people like newer build systems. Ant is working fine, so there isn’t a strong technical reason to replace it, and there are good organisational reasons not to.
> 
> I don’t know how Mick feels, but I tend to find that maintaining our current Ant build is a big waste of my time, and that every time I need to go to this layer its far more brittle than it should be…. There are pushes to refactor to try to get more order in the build (side conversation between Mick and I), but these tend to hide the complexity from most…. I strongly feel that our current build is duct taping things together, this isn’t a complaint of the maintainers but more how Ant works…
> 
> And there are a few things I feel newer build systems offer that actually make my life better:
> 
> 1) cache -  I hate running tests in a loop as this always has the same pattern: rebuild everything from scratch for 1m to run a single test that takes 30 milliseconds…. With ant we “could” force our tasks to cache, but then we have to go do that for everything… we are basically rewriting and reimplementing what other builds have
> 
> 2) consistency - CI runs tests using 1 task, but people like to run tests using a different task… they tend to have different setup and act slightly differently… I had to go improve this for jvm-dtest and see that simulator is in the same shape (if you run simulator tests using the normal CI task, they will fail; you need the bulk task that doesn’t have the same features as the CI version)
> 
> 3) reusable work, avoid copy/paste - when we add new jars/artifacts we need to copy/paste a set of tasks to implement a “sub-module” or “sub-project” (depending on build tool you prefer), not only is this very easy to get wrong, it discourages splitting work into these units (we talked about moving utils/concurrent into its own jar for Accord…)
> 
> 4) only having to know 1 build system - right now we are Ant + Maven; to maintain this project you need to understand both… you can get by not knowing this until you touch dependencies, and if you need tasks that also know about those dependencies… you need to see how this is taped together and grasp that logic…
> 
> Maybe I am bitter as it took a whole day to get SpotBugs working with Ant… and it is just the following in other builds:
> 
> Gradle: apply plugin: “com.github.spotbugs"
> Maven: 
> 
> <reporting>
>     <plugins>
>       <plugin>
>         <groupId>com.github.spotbugs</groupId>
>         <artifactId>spotbugs-maven-plugin</artifactId>
>       </plugin>
>     </plugins>
>   </reporting>
> 
> 
>> On Nov 28, 2022, at 10:46 AM, Maxim Muzafarov <mm...@apache.org> wrote:
>> 
>> Thank you all for the feedback and productive discussion.
>> 
>> 
>> I couldn't have formed my thoughts on the build tools for the product
>> better and provided such good examples than Scott did. Rephrasing what
>> I wrote in the first letter, seeing Maven/Gradle in the project
>> underfoot, a modern IDE will take care of all the necessary files and
>> configurations for themselves much better than we do with scripts. I
>> fully agree that there is no rush with such migration, and the
>> databases in such cases must be more conservative than progressive,
>> and not change anything without strong benefits and a broad consensus
>> on it. I still believe this consensus can be reached in future and
>> when (and if) the consensus will be reached, a clear migration plan
>> should be developed for several releases ahead as well. There's still
>> a lot of work to be done here that's why I mentioned it at the end of
>> my proposal, so as not to pay too much attention to this question at
>> this moment.
>> I've added a link to this thread to the JIRA issue [1], so we don't
>> lose the insights mentioned by members above.
>> 
>> 
>> I want to take away your concerns about lints expansion for now. I
>> thought first of all about making all the source code-checking tools
>> more convenient for use with a minimal set of already existing lints
>> rather than adding or forcing new rules. I really want to avoid here
>> cases with storing multiple configurations for a single tool e.g.
>> having different configurations for 'optional' or 'mandatory' checks
>> as well as different configurations for 'production' or 'tests'.
>> 
>> Thus, the ideal picture in my mind of all discussed above is :
>> 
>> We have:
>> - checkstyle
>> - SpotBugs
>> - Sonar
>> 
>> They work the same way for:
>> - Jenkins builds
>> - CirleCI builds
>> - GitHub pull requests
>> - build on the local machine
>> 
>> For all that, we have the code style webpage [2] (and wiki [3]) is
>> pretty well described, there is no need to expand checking tools with
>> new rules until we will get working these tools on the minimal set of
>> rules. For instance, we can pick up for the checkstyle 'Unused
>> imports', 'Import order', for the SpotBugs 'AutoCloseable',
>> 'Number.valueOf', for Sonar - only reports to monitor the source code
>> trends.
>> 
>> I agree that adding new lints require a broad consensus, so I'd like
>> to avoid such debatable questions for now. Moreover, even with the
>> lints already agreed upon, it is still risky to implement some of them
>> because they can contain a lot of boilerplate changes and may affect
>> more important fixes ready for merge.
>> 
>> 
>> So, as a first step, I can invest my time into the checkstyle tool and
>> make it work everywhere with the same configuration.
>> WDYT?
>> 
>> 
>> P.S.
>> 
>> For IntelliJ with the Checkstyle Plugin it's easy to import the
>> checkstyle.xml the following way:
>> Preferences -> Code Style -> Show Scheme Actions (wheel) -> Import
>> scheme -> Checkstyle configuration.
>> 
>> 
>> [1] https://issues.apache.org/jira/browse/CASSANDRA-17015
>> [2] https://cassandra.apache.org/_/development/code_style.html
>> [3] https://cwiki.apache.org/confluence/display/CASSANDRA2/CodeStyle
>> 
>>> On Sun, 27 Nov 2022 at 13:17, Josh McKenzie <jm...@apache.org> wrote:
>>> 
>>> My .02 on the build discussion is we should try and keep the guts of that in one place, be it the other email thread or on JIRA. Some insightful points made on this thread but would hate to see this thread derailed on a complex independent topic as well as see some of these points lost on the other discussion.
>>> 
>>> I think there needs to be a lot of community consensus on the broad expansion of lints that can reject patches.
>>> 
>>> +1. It may be worthwhile to configure 2 tiers of lints, optional and required, so we can move to a more gradual process of cleaning up lint violations for those that are interested in that type of work. I know in the past we've seen value in looking at the diff in linting violations even w/a 1k+ noisy violation environment.
>>> 
>>> 
>>> On Fri, Nov 25, 2022, at 12:41 PM, scott@paradoxica.net wrote:
>>> 
>>> For me, the strongest arguments in favor of adopting a modern build tool like Maven or Gradle are their ecosystems - both explicit (in terms of plugins), and implicit (in terms of nearly all build tooling supporting both of them, but not ant).
>>> 
>>> Investment in Ant - and in tooling that integrates with Ant - fell off years ago. This makes integrating build-phase aspects of Cassandra with other tooling a very frustrating task that users of most build tools get for free. Many tools built in the last several years don’t support it, or do so only as an afterthought.
>>> 
>>> Two recent examples that have caused pain for me, which I suspect are felt by many:
>>> 
>>> – Integration with internal build systems at many companies that develop Cassandra. Because ant has fallen into disuse, this integration is heavily manual instead of automatic and free. It usually requires forking the project’s build.xml, developing custom tooling around it, or creating a mock Gradle build that wraps ant lifecycle tasks (which also requires overriding ant tasks whose names clash).
>>> 
>>> – Security toolchain integration. Many users and developers of Cassandra also integrate with security tooling at their respective companies. Because Ant has fallen into disuse, most tooling commonly used by security organizations doesn’t support it. SBOMs are a good example, as their introduction postdates ant’s decline. Maven plugins exist to generate them in CycloneDX and SPDX format, but no such plugins exist for ant. Cassandra users and developers who need them must manually write shims to produce SBOMs that users of modern build tools get for free.
>>> 
>>> These might not be use cases anticipated by the project, but they represent work I suspect a large number of contributors to the project are required to perform to make Cassandra usable for them.
>>> 
>>> The ecosystem point means that the fix for this has to be external to the project if the project continues to use Ant: lots of one-off scripts and shims unsuitable for contribution with sole maintainers at their respective companies to provide functionality that users of modern build tools get for free. It also demands continuous, incremental work to adapt to changes in security and build tooling in use at many companies that don’t need to be made for projects using well-supported tools like Maven or Gradle.
>>> 
>>> – Scott
>>> 
>>>> On Nov 25, 2022, at 4:56 AM, Benedict <be...@apache.org> wrote:
>>>> 
>>>> I think modules are already fairly well supported - we in effect already have several? FQL, Simulator and others I think.
>>>> 
>>>> I can anyway say with absolute certainty that the main impediment to modularising is not build tooling, it’s the difficulty of the task, the cost to the project of undertaking it, and therefore its relative payoff versus other things that could be undertaken by the folk with the relevant expertise to do it.
>>>> 
>>>> I’ve also been around long enough to see enough efforts to broaden contributions fail to have an impact, that basing costly decisions on this kind of reasoning doesn’t resonate. The main impediments to contributions are the complexity of the codebase (and problem domain) and our limited capacity to respond promptly to high quality contributions. Until we fix those fundamental issues, this kind of rearranging of chairs seems more like a branding exercise to ourselves than to anyone else.
>>>> 
>>>> 
>>>>> On 25 Nov 2022, at 10:22, Miklosovic, Stefan <St...@netapp.com> wrote:
>>>>> 
>>>>> I agree with what you wrote. How I understand it is that migrating to Maven/Gradle makes the project more "attractive" for newcomers. If a project is built on "that old un-cool Ant", it might be a little bit off-putting and questionable if we are "stuck in the past on build systems and not progressing".
>>>>> 
>>>>> So in that sense I agree this is more "marketing" rather than technological question but on the other hand, does not Maven/Gradle allow us to modularize the project better? Maybe we would like to modularize but nobody is up to that because build system makes it impossible or at least quite inconvenient to do so. Do you really think there are not any significant benefits to switch even if it "just works" now?
>>>>> 
>>>>> ________________________________________
>>>>> From: Benedict <be...@apache.org>
>>>>> Sent: Friday, November 25, 2022 11:07
>>>>> To: dev@cassandra.apache.org
>>>>> Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis
>>>>> 
>>>>> NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> There’s always a handful of people asking for it, but notably few if any of the full time contributors doing the majority of the core development of Cassandra. It strikes me as something very appealing to others, but less so to those wanting to get on with development.
>>>>> 
>>>>> I never really see a good argument articulated for the migration, besides general hand waving that ant is old, and people like newer build systems. Ant is working fine, so there isn’t a strong technical reason to replace it, and there are good organisational reasons not to.
>>>>> 
>>>>> Why do you consider a migration inevitable?
>>>>> 
>>>>> 
>>>>> 
>>>>>> On 25 Nov 2022, at 09:58, Miklosovic, Stefan <St...@netapp.com> wrote:
>>>>>> 
>>>>>> Interesting take on Ant / no-Ant, Benedict. I am very curious how this unfolds. My long-term perception is that changing it to something else is more or less inevitable but if there is a broader consensus to not do that .... well.
>>>>>> 
>>>>>> ________________________________________
>>>>>> From: Benedict <be...@apache.org>
>>>>>> Sent: Friday, November 25, 2022 10:52
>>>>>> To: dev@cassandra.apache.org
>>>>>> Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis
>>>>>> 
>>>>>> NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> I was in a bit of a rush last night. I should say that I’m of course +1 a general endeavour to clean this up, and to expand our use of linters, and I appreciate your volunteering to help out in this way Maxim.
>>>>>> 
>>>>>> However, responding to Stefan, I’m pretty -1 migrating from ant to another build system without really good reason. Migration has a real cost to productivity for all existing contributors, and the phantom of increasing new contributions has never paid off historically. I’m all for easing people into participation, but not at penalty to the existing contributor base.
>>>>>> 
>>>>>> If the only reason is to make it easier to open in a different IDE, we can perhaps have some basic build files outlining code structure for importing, that are compatible with our canonical ant build? We could perhaps even generate them.
>>>>>> 
>>>>>> 
>>>>>>>> On 25 Nov 2022, at 09:35, Miklosovic, Stefan <St...@netapp.com> wrote:
>>>>>>> 
>>>>>>> For the record, I was testing that same combo Claude mentioned and it did not work out of the box but it is definitely possible to set up successfully. I do not remember the details.
>>>>>>> 
>>>>>>> To replay to Maxim, it all seems good to me, roughly, but I humbly think it all boils down to Maven/Gradle refactoring and on top of that we can do all else.
>>>>>>> 
>>>>>>> For example, there is (1) where the solution, besides fixing the tests, is to introduce an Ant task which would check this on build. That being said, how is that going to look like when we change Ant for something else? That stuff suddenly becomes obsolete.
>>>>>>> 
>>>>>>> This case maybe applies to other problems we want to solve as well. I do not want to do something tailored for one build system just to rewrite it all or to spend significant amount of time on that again when we switch the build system.
>>>>>>> 
>>>>>>> For that reason I think changing Ant for something else should be top priority (as I understand that it the hot topic for community for very long time) and then everything else should follow. We should spend time on things mentioned only in case they do not collide with any build system at all.
>>>>>>> 
>>>>>>> (1) https://issues.apache.org/jira/browse/CASSANDRA-17964
>>>>>>> 
>>>>>>> Stefan
>>>>>>> 
>>>>>>> ________________________________________
>>>>>>> From: Claude Warren, Jr via dev <de...@cassandra.apache.org>
>>>>>>> Sent: Friday, November 25, 2022 10:16
>>>>>>> To: dev@cassandra.apache.org
>>>>>>> Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis
>>>>>>> 
>>>>>>> NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> +1 for the concept as a whole.  I am certain I could find nits to pick if I looked deeply.
>>>>>>> 
>>>>>>> @mck -- I did have a problem with Cassandra + Eclipse + Java11 (Classpath).  I gave up and am spending time trying to learn IntelliJ.  I also mentioned it in one of the discussion areas.
>>>>>>> 
>>>>>>> Claude
>>>>>>> 
>>>>>>>> On Thu, Nov 24, 2022 at 8:55 PM Mick Semb Wever <mc...@apache.org>> wrote:
>>>>>>> Thank you for a solid write up Maxim. And welcome to Cassandra, it's
>>>>>>> very positive to see you here.
>>>>>>> 
>>>>>>> I whole-heartedly agree with nearly everything you write. Some input
>>>>>>> and questions inline.
>>>>>>> 
>>>>>>> 
>>>>>>>> 
>>>>>>>> As you may know, the infrastructure team has disabled public sign-up
>>>>>>>> to ASF JIRA (the GitHub issues are recommended instead).
>>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> I suspect (based on chatter in general, but not on dev@ AFAIK) is to
>>>>>>> avoid GH issues and stick with jira. The sign-up hurdle we will
>>>>>>> document on the website: CASSANDRA-18064
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>>> == 1. Make the checkstyle config a single point of truth for the
>>>>>>>> source code style. ==
>>>>>>>> 
>>>>>>>> The checkstyle is already used and included in the Cassandra project
>>>>>>>> build lifecycle (ant command line, Jenkins, CircleCI). There is no
>>>>>>>> need to maintain code style configurations for different types of IDEs
>>>>>>>> (e.g. IntelliJ inspections configuration) since the checkstyle.xml
>>>>>>>> file can be directly imported to IDE used by a developer. This is fair
>>>>>>>> for Intellij Idea, NetBeans, and Eclipse.
>>>>>>> 
>>>>>>> 
>>>>>>> Big +1
>>>>>>> 
>>>>>>> 
>>>>>>>> So, I propose to focus on the checks themselves and checking pull
>>>>>>>> requests with automation scripts, rather than maintaining these
>>>>>>>> integrations. The benefits here are avoiding all issues with
>>>>>>>> maintaining configurations for different IDEs. Another good advantage
>>>>>>>> of this approach would be the ability to add new checkstyle rules
>>>>>>>> without touching IDE configuration - and such tickets will be LFH and
>>>>>>>> easy to commit.
>>>>>>>> 
>>>>>>>> The actions points here are:
>>>>>>>> 
>>>>>>>> - create an umbrella JIRA ticket for all checkstyle issues e.g. [8]
>>>>>>>> (or label checkstyle);
>>>>>>>> - add checkstyle to GitHub pull requests using GitHub actions (execute
>>>>>>>> ant command);
>>>>>>> 
>>>>>>> 
>>>>>>> Instead of custom GHA scripting, please use our existing
>>>>>>> cassandra-artifact.sh (which should already include all such checks).
>>>>>>> 
>>>>>>> Something like https://github.com/apache/cassandra/compare/cassandra-3.11...thelastpickle:cassandra:mck/github-actions/3.11
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>>> == 3. Enable pushing backwards build results for both Jenkins and
>>>>>>>> CircleCI to GitHub pull requests. ==
>>>>>>>> 
>>>>>>>> The goal here is to have a "green checkbox" for those GitHub pull
>>>>>>>> requests that have corresponding Jenkins/CircleCI runs. According to
>>>>>>>> the following links, it is completely possible to have those.
>>>>>>>> 
>>>>>>>> https://github.com/jenkinsci/github-branch-source-plugin
>>>>>>>> https://circleci.com/docs/enable-checks/
>>>>>>>> 
>>>>>>>> Moreover, the GitHub Branch Source Plugin is already enabled for the
>>>>>>>> Cassandra project [16]. The same seems should work the same way for
>>>>>>>> CirleCI, but I have faced the infrastructure team comment [17] that
>>>>>>>> describes admin permissions are required for that, so the question is
>>>>>>>> still open here. I could dig a bit more once we've agreed on it.
>>>>>>>> 
>>>>>>>> The actions points here are:
>>>>>>>> - enable Jenkins integration for GitHub pull requests;
>>>>>>>> - enable CircleCI integration for GitHub pull requests;
>>>>>>> 
>>>>>>> 
>>>>>>> Some folk use CircleCI, some use ci-cassandra. The green checkbox idea
>>>>>>> is great, so long as it's optional. We don't want PRs triggering the
>>>>>>> runs either (there are other mechanisms for triggering for now).
>>>>>>> 
>>>>>>> 
>>>>>>>> The actions points here are:
>>>>>>>> 
>>>>>>>> - initiate a wide survey for user and dev lists, to get to know about
>>>>>>>> the usages;
>>>>>>>> - remove those configurations that are not used anymore;
>>>>>>>> - force migration from Ant to Gradle/Maven;
>>>>>>> 
>>>>>>> 
>>>>>>> Let's leave this out for now. There's too many unknowns here. If
>>>>>>> there's an IDE configuration that's broken, no one has reported it for
>>>>>>> ages, and no one is around to fix it, then I say we should raise the
>>>>>>> discussion to remove it.
>>>>>>> 
>>>>>>> The Gradle/Maven migration is a hot one, contribute to that discussion
>>>>>>> but let's not tangle this work up with it, IMHO.
>>>>>>> 
>>>>>>> Totally agree that IDE project files should be as light weight as possible.
>>>>>>> 
>>>>>>> 
>>>>>>>> Summarizing everything proposed above I think it is possible to
>>>>>>>> simplify adding small contributions easier to the codebase as well as
>>>>>>>> save a bunch of committer's time.
>>>>>>>> 
>>>>>>>> So,
>>>>>>>> WDYT about the things described above?
>>>>>>>> Should I create a CEP for this?
>>>>>>> 
>>>>>>> 
>>>>>>> I see no need for a CEP here. An epic and tickets will work.
>>>>>>> Again, thanks for the input Maxim!
>>>>> 
>>> 
>>> 
>>> 
> 

Re: [DISCUSSION] Cassandra's code style and source code analysis

[Łukasz Dywicki <lu...@code-house.org>]

I can put few sentences in this context, because my involvement into 
Cassandra mailing list started with looking at build system. It was 8 or 
maybe 9 years ago.

Simple fact that Cassandra core team do understand their chain of Ant 
task execution doesn't automatically make all other contributors and 
interested parties understand it. More over, another person outside of 
present core team with prior knowledge of Ant, will require some amount 
of time to dive into task definitions and how they are being invoked. 
There is a reason why usage of Ant fade away over time, its because 
maintenance of builds such Cassandra takes with time more and more effort.
Maven is built around convention over configuration, most of external 
contributors who worked with it before will come with enough of 
knowledge to understand what your build is doing and how. A lot of hate 
people feel to Maven is caused by its limitations which they try to 
overcome for some reason. Gradle is closer in its flexibility to Ant and 
gives much more power, but requires a bit of though to not complicate 
things too much.
Switching to standardized build requires a work, but usually it improves 
architecture and separation of concerns. System you build might have a 
complex logic because of business, standards or technology requirements, 
but your build is not part of that complexity. If you assume it is then 
you just double amount of troubles as your technology will depend on 
over-engineered pipeline. One of major pain points I found back in 2013 
or 2014 was that Cassandra had a lot of cross dependencies between 
packages making it impossible to simplify build in first place. Today I 
am even scared to look at it now. Whether these cross dependencies are 
needed or not - I was not able to answer myself, yet most of properly 
designed systems tend to build directed dependencies free of cycles.

 From looking at history of root build.xml I see 75 contributors who 
touched it. It is not a small number, I would say that for project witch 
such small amount of external dependencies it is a lot.
It is still a fair question if you will benefit from using other build 
tool. Yet I could put opposite question - where you would be, if you 
would make a change 7 years ago? How many hours of tweaking of Ant you 
would save?

With regard to linters I saw it used in openHAB, I know Apache Maven is 
looking to adopt an unified code style as well. In both cases ecosystem 
of these projects is quite large and amount of people doing 
contributions is larger than core team. Yet, even if cassandra is 
smaller in terms of people contributing patches, it has serious 
enterprise use cases. Spotless, the-last-npe and other build time 
plugins can assure that you never get a PR or patch with amended 
whitespaces or code formatting. Unnecessary parts will be fixed by tool 
so patch will contain only needed change. Whether you will enforce it by 
CI or by hand, that's other question. For sure staying with Ant doesn't

Best regards,
Łukasz


On 28.11.2022 22:19, Benedict wrote:
> Scott makes some valid points about legitimate benefits. I personally 
> doubt the high upfront cost of migration will take less than a decade to 
> pay back in time saved managing shims. But, it’s a tangible 
> justification. Conveniently the bulk of my contributions are also at 
> Scott’s prerogative, so if he’s fine with me (and others) wasting their 
> time battling Gradle or Maven, or losing time to the migration, then I 
> think my complaint is functionally neutered.
> 
> I think, though, that those pushing such a disruptive change into others 
> had better work very darn hard to deliver a smooth experience.
> 
>> I tend to find that maintaining our current Ant build is a big waste 
>> of my time, and that every time I need to go to this layer its far 
>> more brittle than it should be
> 
> Whereas I find Ant an absolute pleasure, and Gradle a nightmare, and 
> already regret using it for Accord. I wasted more than a day just trying 
> to get some test artefacts in one module exposed in another, and 
> eventually gave up. I have made dozens of forays into our Ant build and 
> *never* abandoned my goal because I couldn’t accomplish it. I think 
> people are really glossing over all the pain other build systems bring - 
> even without our complex build requirements.
> 
> If you’re going to make the project adopt your preferred build system, 
> you become responsible for the experience of everyone using it. Make 
> sure you’re ready for that.
> 
> 
>> I think those pushing such a disruptive change on others had better 
>> work very darn hard to deliver a smooth experience.
> 
> 
>> On 28 Nov 2022, at 20:18, David Capwell <dc...@apple.com> wrote:
>>
>> I am strong +1 to new linters, I have been working on SpotBugs but 
>> not sent a patch due to sickness and holidays…
>>
>> About the check style as the source of truth for the style guides, I 
>> am +1 to this as well… I feel that wiki is a bad place for this and we 
>> can use the check style file to generate the wiki text (no idea if 
>> there are tools for this…).  About the comment on “hard” requirements… 
>> my understanding has been that new changes must follow and I know 
>> several reviewers who act this way in the review process…. If we agree 
>> on a style, why do we want to keep allowing ignoring it?
>>
>>> There’s always a handful of people asking for it, but notably few if 
>>> any of the full time contributors doing the majority of the core 
>>> development of Cassandra.
>>
>> I don’t agree with that statement, I know several committers/PMC who 
>> want to switch, but every 6 months this gets brought up and gets shot 
>> down… so we just give up trying…
>>
>>> I never really see a good argument articulated for the migration, 
>>> besides general hand waving that ant is old, and people like newer 
>>> build systems. Ant is working fine, so there isn’t a strong technical 
>>> reason to replace it, and there are good organisational reasons not to.
>>
>> I don’t know how Mick feels, but I tend to find that maintaining our 
>> current Ant build is a big waste of my time, and that every time I 
>> need to go to this layer its far more brittle than it should be…. 
>> There are pushes to refactor to try to get more order in the build 
>> (side conversation between Mick and I), but these tend to hide the 
>> complexity from most…. I strongly feel that our current build is duct 
>> taping things together, this isn’t a complaint of the maintainers but 
>> more how Ant works…
>>
>> And there are a few things I feel newer build systems offer that 
>> actually make my life better:
>>
>> 1) cache -  I hate running tests in a loop as this always has the same 
>> pattern: rebuild everything from scratch for 1m to run a single test 
>> that takes 30 milliseconds…. With ant we “could” force our tasks to 
>> cache, but then we have to go do that for everything… we are basically 
>> rewriting and reimplementing what other builds have
>>
>> 2) consistency - CI runs tests using 1 task, but people like to run 
>> tests using a different task… they tend to have different setup and 
>> act slightly differently… I had to go improve this for jvm-dtest and 
>> see that simulator is in the same shape (if you run simulator tests 
>> using the normal CI task, they will fail; you need the bulk task that 
>> doesn’t have the same features as the CI version)
>>
>> 3) reusable work, avoid copy/paste - when we add new jars/artifacts we 
>> need to copy/paste a set of tasks to implement a “sub-module” 
>> or “sub-project” (depending on build tool you prefer), not only is 
>> this very easy to get wrong, it discourages splitting work into these 
>> units (we talked about moving utils/concurrent into its own jar for 
>> Accord…)
>>
>> 4) only having to know 1 build system - right now we are Ant + Maven; 
>> to maintain this project you need to understand both… you can get by 
>> not knowing this until you touch dependencies, and if you need tasks 
>> that also know about those dependencies… you need to see how this is 
>> taped together and grasp that logic…
>>
>> Maybe I am bitter as it took a whole day to get SpotBugs working with 
>> Ant… and it is just the following in other builds:
>>
>> Gradle: apply plugin: “com.github.spotbugs"
>> Maven:
>>
>> <reporting><plugins><plugin><groupId>com.github.spotbugs</groupId><artifactId>spotbugs-maven-plugin</artifactId></plugin></plugins></reporting>
>>
>>
>>> On Nov 28, 2022, at 10:46 AM, Maxim Muzafarov <mmuzaf@apache.org 
>>> <ma...@apache.org>> wrote:
>>>
>>> Thank you all for the feedback and productive discussion.
>>>
>>>
>>> I couldn't have formed my thoughts on the build tools for the product
>>> better and provided such good examples than Scott did. Rephrasing what
>>> I wrote in the first letter, seeing Maven/Gradle in the project
>>> underfoot, a modern IDE will take care of all the necessary files and
>>> configurations for themselves much better than we do with scripts. I
>>> fully agree that there is no rush with such migration, and the
>>> databases in such cases must be more conservative than progressive,
>>> and not change anything without strong benefits and a broad consensus
>>> on it. I still believe this consensus can be reached in future and
>>> when (and if) the consensus will be reached, a clear migration plan
>>> should be developed for several releases ahead as well. There's still
>>> a lot of work to be done here that's why I mentioned it at the end of
>>> my proposal, so as not to pay too much attention to this question at
>>> this moment.
>>> I've added a link to this thread to the JIRA issue [1], so we don't
>>> lose the insights mentioned by members above.
>>>
>>>
>>> I want to take away your concerns about lints expansion for now. I
>>> thought first of all about making all the source code-checking tools
>>> more convenient for use with a minimal set of already existing lints
>>> rather than adding or forcing new rules. I really want to avoid here
>>> cases with storing multiple configurations for a single tool e.g.
>>> having different configurations for 'optional' or 'mandatory' checks
>>> as well as different configurations for 'production' or 'tests'.
>>>
>>> Thus, the ideal picture in my mind of all discussed above is :
>>>
>>> We have:
>>> - checkstyle
>>> - SpotBugs
>>> - Sonar
>>>
>>> They work the same way for:
>>> - Jenkins builds
>>> - CirleCI builds
>>> - GitHub pull requests
>>> - build on the local machine
>>>
>>> For all that, we have the code style webpage [2] (and wiki [3]) is
>>> pretty well described, there is no need to expand checking tools with
>>> new rules until we will get working these tools on the minimal set of
>>> rules. For instance, we can pick up for the checkstyle 'Unused
>>> imports', 'Import order', for the SpotBugs 'AutoCloseable',
>>> 'Number.valueOf', for Sonar - only reports to monitor the source code
>>> trends.
>>>
>>> I agree that adding new lints require a broad consensus, so I'd like
>>> to avoid such debatable questions for now. Moreover, even with the
>>> lints already agreed upon, it is still risky to implement some of them
>>> because they can contain a lot of boilerplate changes and may affect
>>> more important fixes ready for merge.
>>>
>>>
>>> So, as a first step, I can invest my time into the checkstyle tool and
>>> make it work everywhere with the same configuration.
>>> WDYT?
>>>
>>>
>>> P.S.
>>>
>>> For IntelliJ with the Checkstyle Plugin it's easy to import the
>>> checkstyle.xml the following way:
>>> Preferences -> Code Style -> Show Scheme Actions (wheel) -> Import
>>> scheme -> Checkstyle configuration.
>>>
>>>
>>> [1] https://issues.apache.org/jira/browse/CASSANDRA-17015 
>>> <https://issues.apache.org/jira/browse/CASSANDRA-17015>
>>> [2] https://cassandra.apache.org/_/development/code_style.html 
>>> <https://cassandra.apache.org/_/development/code_style.html>
>>> [3] https://cwiki.apache.org/confluence/display/CASSANDRA2/CodeStyle 
>>> <https://cwiki.apache.org/confluence/display/CASSANDRA2/CodeStyle>
>>>
>>> On Sun, 27 Nov 2022 at 13:17, Josh McKenzie <jmckenzie@apache.org 
>>> <ma...@apache.org>> wrote:
>>>>
>>>> My .02 on the build discussion is we should try and keep the guts of 
>>>> that in one place, be it the other email thread or on JIRA. Some 
>>>> insightful points made on this thread but would hate to see this 
>>>> thread derailed on a complex independent topic as well as see some 
>>>> of these points lost on the other discussion.
>>>>
>>>> I think there needs to be a lot of community consensus on the broad 
>>>> expansion of lints that can reject patches.
>>>>
>>>> +1. It may be worthwhile to configure 2 tiers of lints, optional and 
>>>> required, so we can move to a more gradual process of cleaning up 
>>>> lint violations for those that are interested in that type of work. 
>>>> I know in the past we've seen value in looking at the diff in 
>>>> linting violations even w/a 1k+ noisy violation environment.
>>>>
>>>>
>>>> On Fri, Nov 25, 2022, at 12:41 PM, scott@paradoxica.net 
>>>> <ma...@paradoxica.net> wrote:
>>>>
>>>> For me, the strongest arguments in favor of adopting a modern build 
>>>> tool like Maven or Gradle are their ecosystems - both explicit (in 
>>>> terms of plugins), and implicit (in terms of nearly all build 
>>>> tooling supporting both of them, but not ant).
>>>>
>>>> Investment in Ant - and in tooling that integrates with Ant - fell 
>>>> off years ago. This makes integrating build-phase aspects of 
>>>> Cassandra with other tooling a very frustrating task that users of 
>>>> most build tools get for free. Many tools built in the last several 
>>>> years don’t support it, or do so only as an afterthought.
>>>>
>>>> Two recent examples that have caused pain for me, which I suspect 
>>>> are felt by many:
>>>>
>>>> – Integration with internal build systems at many companies that 
>>>> develop Cassandra. Because ant has fallen into disuse, this 
>>>> integration is heavily manual instead of automatic and free. It 
>>>> usually requires forking the project’s build.xml, developing custom 
>>>> tooling around it, or creating a mock Gradle build that wraps ant 
>>>> lifecycle tasks (which also requires overriding ant tasks whose 
>>>> names clash).
>>>>
>>>> – Security toolchain integration. Many users and developers of 
>>>> Cassandra also integrate with security tooling at their respective 
>>>> companies. Because Ant has fallen into disuse, most tooling commonly 
>>>> used by security organizations doesn’t support it. SBOMs are a good 
>>>> example, as their introduction postdates ant’s decline. Maven 
>>>> plugins exist to generate them in CycloneDX and SPDX format, but no 
>>>> such plugins exist for ant. Cassandra users and developers who need 
>>>> them must manually write shims to produce SBOMs that users of modern 
>>>> build tools get for free.
>>>>
>>>> These might not be use cases anticipated by the project, but they 
>>>> represent work I suspect a large number of contributors to the 
>>>> project are required to perform to make Cassandra usable for them.
>>>>
>>>> The ecosystem point means that the fix for this has to be external 
>>>> to the project if the project continues to use Ant: lots of one-off 
>>>> scripts and shims unsuitable for contribution with sole maintainers 
>>>> at their respective companies to provide functionality that users of 
>>>> modern build tools get for free. It also demands continuous, 
>>>> incremental work to adapt to changes in security and build tooling 
>>>> in use at many companies that don’t need to be made for projects 
>>>> using well-supported tools like Maven or Gradle.
>>>>
>>>> – Scott
>>>>
>>>>> On Nov 25, 2022, at 4:56 AM, Benedict <benedict@apache.org 
>>>>> <ma...@apache.org>> wrote:
>>>>>
>>>>> I think modules are already fairly well supported - we in effect 
>>>>> already have several? FQL, Simulator and others I think.
>>>>>
>>>>> I can anyway say with absolute certainty that the main impediment 
>>>>> to modularising is not build tooling, it’s the difficulty of the 
>>>>> task, the cost to the project of undertaking it, and therefore its 
>>>>> relative payoff versus other things that could be undertaken by the 
>>>>> folk with the relevant expertise to do it.
>>>>>
>>>>> I’ve also been around long enough to see enough efforts to broaden 
>>>>> contributions fail to have an impact, that basing costly decisions 
>>>>> on this kind of reasoning doesn’t resonate. The main impediments to 
>>>>> contributions are the complexity of the codebase (and problem 
>>>>> domain) and our limited capacity to respond promptly to high 
>>>>> quality contributions. Until we fix those fundamental issues, this 
>>>>> kind of rearranging of chairs seems more like a branding exercise 
>>>>> to ourselves than to anyone else.
>>>>>
>>>>>
>>>>>> On 25 Nov 2022, at 10:22, Miklosovic, Stefan 
>>>>>> <Stefan.Miklosovic@netapp.com 
>>>>>> <ma...@netapp.com>> wrote:
>>>>>>
>>>>>> I agree with what you wrote. How I understand it is that 
>>>>>> migrating to Maven/Gradle makes the project more "attractive" for 
>>>>>> newcomers. If a project is built on "that old un-cool Ant", it 
>>>>>> might be a little bit off-putting and questionable if we are 
>>>>>> "stuck in the past on build systems and not progressing".
>>>>>>
>>>>>> So in that sense I agree this is more "marketing" rather than 
>>>>>> technological question but on the other hand, does not 
>>>>>> Maven/Gradle allow us to modularize the project better? Maybe we 
>>>>>> would like to modularize but nobody is up to that because build 
>>>>>> system makes it impossible or at least quite inconvenient to do 
>>>>>> so. Do you really think there are not any significant benefits to 
>>>>>> switch even if it "just works" now?
>>>>>>
>>>>>> ________________________________________
>>>>>> From: Benedict <benedict@apache.org <ma...@apache.org>>
>>>>>> Sent: Friday, November 25, 2022 11:07
>>>>>> To: dev@cassandra.apache.org <ma...@cassandra.apache.org>
>>>>>> Subject: Re: [DISCUSSION] Cassandra's code style and source code 
>>>>>> analysis
>>>>>>
>>>>>> NetApp Security WARNING: This is an external email. Do not click 
>>>>>> links or open attachments unless you recognize the sender and know 
>>>>>> the content is safe.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> There’s always a handful of people asking for it, but notably few 
>>>>>> if any of the full time contributors doing the majority of the 
>>>>>> core development of Cassandra. It strikes me as something very 
>>>>>> appealing to others, but less so to those wanting to get on with 
>>>>>> development.
>>>>>>
>>>>>> I never really see a good argument articulated for the migration, 
>>>>>> besides general hand waving that ant is old, and people like newer 
>>>>>> build systems. Ant is working fine, so there isn’t a strong 
>>>>>> technical reason to replace it, and there are good organisational 
>>>>>> reasons not to.
>>>>>>
>>>>>> Why do you consider a migration inevitable?
>>>>>>
>>>>>>
>>>>>>
>>>>>>> On 25 Nov 2022, at 09:58, Miklosovic, Stefan 
>>>>>>> <Stefan.Miklosovic@netapp.com 
>>>>>>> <ma...@netapp.com>> wrote:
>>>>>>>
>>>>>>> Interesting take on Ant / no-Ant, Benedict. I am very curious 
>>>>>>> how this unfolds. My long-term perception is that changing it to 
>>>>>>> something else is more or less inevitable but if there is a 
>>>>>>> broader consensus to not do that .... well.
>>>>>>>
>>>>>>> ________________________________________
>>>>>>> From: Benedict <benedict@apache.org <ma...@apache.org>>
>>>>>>> Sent: Friday, November 25, 2022 10:52
>>>>>>> To: dev@cassandra.apache.org <ma...@cassandra.apache.org>
>>>>>>> Subject: Re: [DISCUSSION] Cassandra's code style and source code 
>>>>>>> analysis
>>>>>>>
>>>>>>> NetApp Security WARNING: This is an external email. Do not click 
>>>>>>> links or open attachments unless you recognize the sender and 
>>>>>>> know the content is safe.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> I was in a bit of a rush last night. I should say that I’m of 
>>>>>>> course +1 a general endeavour to clean this up, and to expand our 
>>>>>>> use of linters, and I appreciate your volunteering to help out in 
>>>>>>> this way Maxim.
>>>>>>>
>>>>>>> However, responding to Stefan, I’m pretty -1 migrating from ant 
>>>>>>> to another build system without really good reason. Migration has 
>>>>>>> a real cost to productivity for all existing contributors, and 
>>>>>>> the phantom of increasing new contributions has never paid off 
>>>>>>> historically. I’m all for easing people into participation, but 
>>>>>>> not at penalty to the existing contributor base.
>>>>>>>
>>>>>>> If the only reason is to make it easier to open in a different 
>>>>>>> IDE, we can perhaps have some basic build files outlining code 
>>>>>>> structure for importing, that are compatible with our canonical 
>>>>>>> ant build? We could perhaps even generate them.
>>>>>>>
>>>>>>>
>>>>>>>>> On 25 Nov 2022, at 09:35, Miklosovic, Stefan 
>>>>>>>>> <Stefan.Miklosovic@netapp.com 
>>>>>>>>> <ma...@netapp.com>> wrote:
>>>>>>>>
>>>>>>>> For the record, I was testing that same combo Claude mentioned 
>>>>>>>> and it did not work out of the box but it is definitely possible 
>>>>>>>> to set up successfully. I do not remember the details.
>>>>>>>>
>>>>>>>> To replay to Maxim, it all seems good to me, roughly, but I 
>>>>>>>> humbly think it all boils down to Maven/Gradle refactoring and 
>>>>>>>> on top of that we can do all else.
>>>>>>>>
>>>>>>>> For example, there is (1) where the solution, besides fixing the 
>>>>>>>> tests, is to introduce an Ant task which would check this on 
>>>>>>>> build. That being said, how is that going to look like when we 
>>>>>>>> change Ant for something else? That stuff suddenly becomes obsolete.
>>>>>>>>
>>>>>>>> This case maybe applies to other problems we want to solve as 
>>>>>>>> well. I do not want to do something tailored for one build 
>>>>>>>> system just to rewrite it all or to spend significant amount of 
>>>>>>>> time on that again when we switch the build system.
>>>>>>>>
>>>>>>>> For that reason I think changing Ant for something else should 
>>>>>>>> be top priority (as I understand that it the hot topic for 
>>>>>>>> community for very long time) and then everything else should 
>>>>>>>> follow. We should spend time on things mentioned only in case 
>>>>>>>> they do not collide with any build system at all.
>>>>>>>>
>>>>>>>> (1) https://issues.apache.org/jira/browse/CASSANDRA-17964 
>>>>>>>> <https://issues.apache.org/jira/browse/CASSANDRA-17964>
>>>>>>>>
>>>>>>>> Stefan
>>>>>>>>
>>>>>>>> ________________________________________
>>>>>>>> From: Claude Warren, Jr via dev <dev@cassandra.apache.org 
>>>>>>>> <ma...@cassandra.apache.org>>
>>>>>>>> Sent: Friday, November 25, 2022 10:16
>>>>>>>> To: dev@cassandra.apache.org <ma...@cassandra.apache.org>
>>>>>>>> Subject: Re: [DISCUSSION] Cassandra's code style and source code 
>>>>>>>> analysis
>>>>>>>>
>>>>>>>> NetApp Security WARNING: This is an external email. Do not click 
>>>>>>>> links or open attachments unless you recognize the sender and 
>>>>>>>> know the content is safe.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> +1 for the concept as a whole.  I am certain I could find nits 
>>>>>>>> to pick if I looked deeply.
>>>>>>>>
>>>>>>>> @mck -- I did have a problem with Cassandra + Eclipse + Java11 
>>>>>>>> (Classpath).  I gave up and am spending time trying to learn 
>>>>>>>> IntelliJ.  I also mentioned it in one of the discussion areas.
>>>>>>>>
>>>>>>>> Claude
>>>>>>>>
>>>>>>>>> On Thu, Nov 24, 2022 at 8:55 PM Mick Semb Wever <mck@apache.org 
>>>>>>>>> <ma...@apache.org><mailto:mck@apache.org 
>>>>>>>>> <ma...@apache.org>>> wrote:
>>>>>>>> Thank you for a solid write up Maxim. And welcome to Cassandra, it's
>>>>>>>> very positive to see you here.
>>>>>>>>
>>>>>>>> I whole-heartedly agree with nearly everything you write. Some input
>>>>>>>> and questions inline.
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>> As you may know, the infrastructure team has disabled public 
>>>>>>>>> sign-up
>>>>>>>>> to ASF JIRA (the GitHub issues are recommended instead).
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> I suspect (based on chatter in general, but not on dev@ AFAIK) is to
>>>>>>>> avoid GH issues and stick with jira. The sign-up hurdle we will
>>>>>>>> document on the website: CASSANDRA-18064
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> == 1. Make the checkstyle config a single point of truth for the
>>>>>>>>> source code style. ==
>>>>>>>>>
>>>>>>>>> The checkstyle is already used and included in the Cassandra 
>>>>>>>>> project
>>>>>>>>> build lifecycle (ant command line, Jenkins, CircleCI). There is no
>>>>>>>>> need to maintain code style configurations for different types 
>>>>>>>>> of IDEs
>>>>>>>>> (e.g. IntelliJ inspections configuration) since the checkstyle.xml
>>>>>>>>> file can be directly imported to IDE used by a developer. This 
>>>>>>>>> is fair
>>>>>>>>> for Intellij Idea, NetBeans, and Eclipse.
>>>>>>>>
>>>>>>>>
>>>>>>>> Big +1
>>>>>>>>
>>>>>>>>
>>>>>>>>> So, I propose to focus on the checks themselves and checking pull
>>>>>>>>> requests with automation scripts, rather than maintaining these
>>>>>>>>> integrations. The benefits here are avoiding all issues with
>>>>>>>>> maintaining configurations for different IDEs. Another good 
>>>>>>>>> advantage
>>>>>>>>> of this approach would be the ability to add new checkstyle rules
>>>>>>>>> without touching IDE configuration - and such tickets will be 
>>>>>>>>> LFH and
>>>>>>>>> easy to commit.
>>>>>>>>>
>>>>>>>>> The actions points here are:
>>>>>>>>>
>>>>>>>>> - create an umbrella JIRA ticket for all checkstyle issues e.g. [8]
>>>>>>>>> (or label checkstyle);
>>>>>>>>> - add checkstyle to GitHub pull requests using GitHub actions 
>>>>>>>>> (execute
>>>>>>>>> ant command);
>>>>>>>>
>>>>>>>>
>>>>>>>> Instead of custom GHA scripting, please use our existing
>>>>>>>> cassandra-artifact.sh (which should already include all such 
>>>>>>>> checks).
>>>>>>>>
>>>>>>>> Something like 
>>>>>>>> https://github.com/apache/cassandra/compare/cassandra-3.11...thelastpickle:cassandra:mck/github-actions/3.11 <https://github.com/apache/cassandra/compare/cassandra-3.11...thelastpickle:cassandra:mck/github-actions/3.11>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> == 3. Enable pushing backwards build results for both Jenkins and
>>>>>>>>> CircleCI to GitHub pull requests. ==
>>>>>>>>>
>>>>>>>>> The goal here is to have a "green checkbox" for those GitHub pull
>>>>>>>>> requests that have corresponding Jenkins/CircleCI runs. 
>>>>>>>>> According to
>>>>>>>>> the following links, it is completely possible to have those.
>>>>>>>>>
>>>>>>>>> https://github.com/jenkinsci/github-branch-source-plugin 
>>>>>>>>> <https://github.com/jenkinsci/github-branch-source-plugin>
>>>>>>>>> https://circleci.com/docs/enable-checks/
>>>>>>>>>
>>>>>>>>> Moreover, the GitHub Branch Source Plugin is already enabled 
>>>>>>>>> for the
>>>>>>>>> Cassandra project [16]. The same seems should work the same way for
>>>>>>>>> CirleCI, but I have faced the infrastructure team comment [17] that
>>>>>>>>> describes admin permissions are required for that, so the 
>>>>>>>>> question is
>>>>>>>>> still open here. I could dig a bit more once we've agreed on it.
>>>>>>>>>
>>>>>>>>> The actions points here are:
>>>>>>>>> - enable Jenkins integration for GitHub pull requests;
>>>>>>>>> - enable CircleCI integration for GitHub pull requests;
>>>>>>>>
>>>>>>>>
>>>>>>>> Some folk use CircleCI, some use ci-cassandra. The green 
>>>>>>>> checkbox idea
>>>>>>>> is great, so long as it's optional. We don't want PRs triggering the
>>>>>>>> runs either (there are other mechanisms for triggering for now).
>>>>>>>>
>>>>>>>>
>>>>>>>>> The actions points here are:
>>>>>>>>>
>>>>>>>>> - initiate a wide survey for user and dev lists, to get to know 
>>>>>>>>> about
>>>>>>>>> the usages;
>>>>>>>>> - remove those configurations that are not used anymore;
>>>>>>>>> - force migration from Ant to Gradle/Maven;
>>>>>>>>
>>>>>>>>
>>>>>>>> Let's leave this out for now. There's too many unknowns here. If
>>>>>>>> there's an IDE configuration that's broken, no one has reported 
>>>>>>>> it for
>>>>>>>> ages, and no one is around to fix it, then I say we should raise the
>>>>>>>> discussion to remove it.
>>>>>>>>
>>>>>>>> The Gradle/Maven migration is a hot one, contribute to that 
>>>>>>>> discussion
>>>>>>>> but let's not tangle this work up with it, IMHO.
>>>>>>>>
>>>>>>>> Totally agree that IDE project files should be as light weight 
>>>>>>>> as possible.
>>>>>>>>
>>>>>>>>
>>>>>>>>> Summarizing everything proposed above I think it is possible to
>>>>>>>>> simplify adding small contributions easier to the codebase as 
>>>>>>>>> well as
>>>>>>>>> save a bunch of committer's time.
>>>>>>>>>
>>>>>>>>> So,
>>>>>>>>> WDYT about the things described above?
>>>>>>>>> Should I create a CEP for this?
>>>>>>>>
>>>>>>>>
>>>>>>>> I see no need for a CEP here. An epic and tickets will work.
>>>>>>>> Again, thanks for the input Maxim!
>>>>>>
>>>>
>>>>
>>>>
>>

Re: [DISCUSSION] Cassandra's code style and source code analysis

[Benedict <be...@apache.org>]

Scott makes some valid points about legitimate benefits. I personally doubt the high upfront cost of migration will take less than a decade to pay back in time saved managing shims. But, it’s a tangible justification. Conveniently the bulk of my contributions are also at Scott’s prerogative, so if he’s fine with me (and others) wasting their time battling Gradle or Maven, or losing time to the migration, then I think my complaint is functionally neutered.

I think, though, that those pushing such a disruptive change into others had better work very darn hard to deliver a smooth experience.

> I tend to find that maintaining our current Ant build is a big waste of my time, and that every time I need to go to this layer its far more brittle than it should be

Whereas I find Ant an absolute pleasure, and Gradle a nightmare, and already regret using it for Accord. I wasted more than a day just trying to get some test artefacts in one module exposed in another, and eventually gave up. I have made dozens of forays into our Ant build and *never* abandoned my goal because I couldn’t accomplish it. I think people are really glossing over all the pain other build systems bring - even without our complex build requirements. 

If you’re going to make the project adopt your preferred build system, you become responsible for the experience of everyone using it. Make sure you’re ready for that.


> I think those pushing such a disruptive change on others had better work very darn hard to deliver a smooth experience.



> On 28 Nov 2022, at 20:18, David Capwell <dc...@apple.com> wrote:
> 
> I am strong +1 to new linters, I have been working on SpotBugs but not sent a patch due to sickness and holidays…
> 
> About the check style as the source of truth for the style guides, I am +1 to this as well… I feel that wiki is a bad place for this and we can use the check style file to generate the wiki text (no idea if there are tools for this…).  About the comment on “hard” requirements… my understanding has been that new changes must follow and I know several reviewers who act this way in the review process…. If we agree on a style, why do we want to keep allowing ignoring it?
> 
>> There’s always a handful of people asking for it, but notably few if any of the full time contributors doing the majority of the core development of Cassandra.
> 
> 
> I don’t agree with that statement, I know several committers/PMC who want to switch, but every 6 months this gets brought up and gets shot down… so we just give up trying… 
> 
>> I never really see a good argument articulated for the migration, besides general hand waving that ant is old, and people like newer build systems. Ant is working fine, so there isn’t a strong technical reason to replace it, and there are good organisational reasons not to.
> 
> I don’t know how Mick feels, but I tend to find that maintaining our current Ant build is a big waste of my time, and that every time I need to go to this layer its far more brittle than it should be…. There are pushes to refactor to try to get more order in the build (side conversation between Mick and I), but these tend to hide the complexity from most…. I strongly feel that our current build is duct taping things together, this isn’t a complaint of the maintainers but more how Ant works…
> 
> And there are a few things I feel newer build systems offer that actually make my life better:
> 
> 1) cache -  I hate running tests in a loop as this always has the same pattern: rebuild everything from scratch for 1m to run a single test that takes 30 milliseconds…. With ant we “could” force our tasks to cache, but then we have to go do that for everything… we are basically rewriting and reimplementing what other builds have
> 
> 2) consistency - CI runs tests using 1 task, but people like to run tests using a different task… they tend to have different setup and act slightly differently… I had to go improve this for jvm-dtest and see that simulator is in the same shape (if you run simulator tests using the normal CI task, they will fail; you need the bulk task that doesn’t have the same features as the CI version)
> 
> 3) reusable work, avoid copy/paste - when we add new jars/artifacts we need to copy/paste a set of tasks to implement a “sub-module” or “sub-project” (depending on build tool you prefer), not only is this very easy to get wrong, it discourages splitting work into these units (we talked about moving utils/concurrent into its own jar for Accord…)
> 
> 4) only having to know 1 build system - right now we are Ant + Maven; to maintain this project you need to understand both… you can get by not knowing this until you touch dependencies, and if you need tasks that also know about those dependencies… you need to see how this is taped together and grasp that logic…
> 
> Maybe I am bitter as it took a whole day to get SpotBugs working with Ant… and it is just the following in other builds:
> 
> Gradle: apply plugin: “com.github.spotbugs"
> Maven: 
> 
> <reporting>
>     <plugins>
>       <plugin>
>         <groupId>com.github.spotbugs</groupId>
>         <artifactId>spotbugs-maven-plugin</artifactId>
>       </plugin>
>     </plugins>
>   </reporting>
> 
> 
>> On Nov 28, 2022, at 10:46 AM, Maxim Muzafarov <mm...@apache.org> wrote:
>> 
>> Thank you all for the feedback and productive discussion.
>> 
>> 
>> I couldn't have formed my thoughts on the build tools for the product
>> better and provided such good examples than Scott did. Rephrasing what
>> I wrote in the first letter, seeing Maven/Gradle in the project
>> underfoot, a modern IDE will take care of all the necessary files and
>> configurations for themselves much better than we do with scripts. I
>> fully agree that there is no rush with such migration, and the
>> databases in such cases must be more conservative than progressive,
>> and not change anything without strong benefits and a broad consensus
>> on it. I still believe this consensus can be reached in future and
>> when (and if) the consensus will be reached, a clear migration plan
>> should be developed for several releases ahead as well. There's still
>> a lot of work to be done here that's why I mentioned it at the end of
>> my proposal, so as not to pay too much attention to this question at
>> this moment.
>> I've added a link to this thread to the JIRA issue [1], so we don't
>> lose the insights mentioned by members above.
>> 
>> 
>> I want to take away your concerns about lints expansion for now. I
>> thought first of all about making all the source code-checking tools
>> more convenient for use with a minimal set of already existing lints
>> rather than adding or forcing new rules. I really want to avoid here
>> cases with storing multiple configurations for a single tool e.g.
>> having different configurations for 'optional' or 'mandatory' checks
>> as well as different configurations for 'production' or 'tests'.
>> 
>> Thus, the ideal picture in my mind of all discussed above is :
>> 
>> We have:
>> - checkstyle
>> - SpotBugs
>> - Sonar
>> 
>> They work the same way for:
>> - Jenkins builds
>> - CirleCI builds
>> - GitHub pull requests
>> - build on the local machine
>> 
>> For all that, we have the code style webpage [2] (and wiki [3]) is
>> pretty well described, there is no need to expand checking tools with
>> new rules until we will get working these tools on the minimal set of
>> rules. For instance, we can pick up for the checkstyle 'Unused
>> imports', 'Import order', for the SpotBugs 'AutoCloseable',
>> 'Number.valueOf', for Sonar - only reports to monitor the source code
>> trends.
>> 
>> I agree that adding new lints require a broad consensus, so I'd like
>> to avoid such debatable questions for now. Moreover, even with the
>> lints already agreed upon, it is still risky to implement some of them
>> because they can contain a lot of boilerplate changes and may affect
>> more important fixes ready for merge.
>> 
>> 
>> So, as a first step, I can invest my time into the checkstyle tool and
>> make it work everywhere with the same configuration.
>> WDYT?
>> 
>> 
>> P.S.
>> 
>> For IntelliJ with the Checkstyle Plugin it's easy to import the
>> checkstyle.xml the following way:
>> Preferences -> Code Style -> Show Scheme Actions (wheel) -> Import
>> scheme -> Checkstyle configuration.
>> 
>> 
>> [1] https://issues.apache.org/jira/browse/CASSANDRA-17015
>> [2] https://cassandra.apache.org/_/development/code_style.html
>> [3] https://cwiki.apache.org/confluence/display/CASSANDRA2/CodeStyle
>> 
>>> On Sun, 27 Nov 2022 at 13:17, Josh McKenzie <jm...@apache.org> wrote:
>>> 
>>> My .02 on the build discussion is we should try and keep the guts of that in one place, be it the other email thread or on JIRA. Some insightful points made on this thread but would hate to see this thread derailed on a complex independent topic as well as see some of these points lost on the other discussion.
>>> 
>>> I think there needs to be a lot of community consensus on the broad expansion of lints that can reject patches.
>>> 
>>> +1. It may be worthwhile to configure 2 tiers of lints, optional and required, so we can move to a more gradual process of cleaning up lint violations for those that are interested in that type of work. I know in the past we've seen value in looking at the diff in linting violations even w/a 1k+ noisy violation environment.
>>> 
>>> 
>>> On Fri, Nov 25, 2022, at 12:41 PM, scott@paradoxica.net wrote:
>>> 
>>> For me, the strongest arguments in favor of adopting a modern build tool like Maven or Gradle are their ecosystems - both explicit (in terms of plugins), and implicit (in terms of nearly all build tooling supporting both of them, but not ant).
>>> 
>>> Investment in Ant - and in tooling that integrates with Ant - fell off years ago. This makes integrating build-phase aspects of Cassandra with other tooling a very frustrating task that users of most build tools get for free. Many tools built in the last several years don’t support it, or do so only as an afterthought.
>>> 
>>> Two recent examples that have caused pain for me, which I suspect are felt by many:
>>> 
>>> – Integration with internal build systems at many companies that develop Cassandra. Because ant has fallen into disuse, this integration is heavily manual instead of automatic and free. It usually requires forking the project’s build.xml, developing custom tooling around it, or creating a mock Gradle build that wraps ant lifecycle tasks (which also requires overriding ant tasks whose names clash).
>>> 
>>> – Security toolchain integration. Many users and developers of Cassandra also integrate with security tooling at their respective companies. Because Ant has fallen into disuse, most tooling commonly used by security organizations doesn’t support it. SBOMs are a good example, as their introduction postdates ant’s decline. Maven plugins exist to generate them in CycloneDX and SPDX format, but no such plugins exist for ant. Cassandra users and developers who need them must manually write shims to produce SBOMs that users of modern build tools get for free.
>>> 
>>> These might not be use cases anticipated by the project, but they represent work I suspect a large number of contributors to the project are required to perform to make Cassandra usable for them.
>>> 
>>> The ecosystem point means that the fix for this has to be external to the project if the project continues to use Ant: lots of one-off scripts and shims unsuitable for contribution with sole maintainers at their respective companies to provide functionality that users of modern build tools get for free. It also demands continuous, incremental work to adapt to changes in security and build tooling in use at many companies that don’t need to be made for projects using well-supported tools like Maven or Gradle.
>>> 
>>> – Scott
>>> 
>>>> On Nov 25, 2022, at 4:56 AM, Benedict <be...@apache.org> wrote:
>>>> 
>>>> I think modules are already fairly well supported - we in effect already have several? FQL, Simulator and others I think.
>>>> 
>>>> I can anyway say with absolute certainty that the main impediment to modularising is not build tooling, it’s the difficulty of the task, the cost to the project of undertaking it, and therefore its relative payoff versus other things that could be undertaken by the folk with the relevant expertise to do it.
>>>> 
>>>> I’ve also been around long enough to see enough efforts to broaden contributions fail to have an impact, that basing costly decisions on this kind of reasoning doesn’t resonate. The main impediments to contributions are the complexity of the codebase (and problem domain) and our limited capacity to respond promptly to high quality contributions. Until we fix those fundamental issues, this kind of rearranging of chairs seems more like a branding exercise to ourselves than to anyone else.
>>>> 
>>>> 
>>>>> On 25 Nov 2022, at 10:22, Miklosovic, Stefan <St...@netapp.com> wrote:
>>>>> 
>>>>> I agree with what you wrote. How I understand it is that migrating to Maven/Gradle makes the project more "attractive" for newcomers. If a project is built on "that old un-cool Ant", it might be a little bit off-putting and questionable if we are "stuck in the past on build systems and not progressing".
>>>>> 
>>>>> So in that sense I agree this is more "marketing" rather than technological question but on the other hand, does not Maven/Gradle allow us to modularize the project better? Maybe we would like to modularize but nobody is up to that because build system makes it impossible or at least quite inconvenient to do so. Do you really think there are not any significant benefits to switch even if it "just works" now?
>>>>> 
>>>>> ________________________________________
>>>>> From: Benedict <be...@apache.org>
>>>>> Sent: Friday, November 25, 2022 11:07
>>>>> To: dev@cassandra.apache.org
>>>>> Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis
>>>>> 
>>>>> NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> There’s always a handful of people asking for it, but notably few if any of the full time contributors doing the majority of the core development of Cassandra. It strikes me as something very appealing to others, but less so to those wanting to get on with development.
>>>>> 
>>>>> I never really see a good argument articulated for the migration, besides general hand waving that ant is old, and people like newer build systems. Ant is working fine, so there isn’t a strong technical reason to replace it, and there are good organisational reasons not to.
>>>>> 
>>>>> Why do you consider a migration inevitable?
>>>>> 
>>>>> 
>>>>> 
>>>>>> On 25 Nov 2022, at 09:58, Miklosovic, Stefan <St...@netapp.com> wrote:
>>>>>> 
>>>>>> Interesting take on Ant / no-Ant, Benedict. I am very curious how this unfolds. My long-term perception is that changing it to something else is more or less inevitable but if there is a broader consensus to not do that .... well.
>>>>>> 
>>>>>> ________________________________________
>>>>>> From: Benedict <be...@apache.org>
>>>>>> Sent: Friday, November 25, 2022 10:52
>>>>>> To: dev@cassandra.apache.org
>>>>>> Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis
>>>>>> 
>>>>>> NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> I was in a bit of a rush last night. I should say that I’m of course +1 a general endeavour to clean this up, and to expand our use of linters, and I appreciate your volunteering to help out in this way Maxim.
>>>>>> 
>>>>>> However, responding to Stefan, I’m pretty -1 migrating from ant to another build system without really good reason. Migration has a real cost to productivity for all existing contributors, and the phantom of increasing new contributions has never paid off historically. I’m all for easing people into participation, but not at penalty to the existing contributor base.
>>>>>> 
>>>>>> If the only reason is to make it easier to open in a different IDE, we can perhaps have some basic build files outlining code structure for importing, that are compatible with our canonical ant build? We could perhaps even generate them.
>>>>>> 
>>>>>> 
>>>>>>>> On 25 Nov 2022, at 09:35, Miklosovic, Stefan <St...@netapp.com> wrote:
>>>>>>> 
>>>>>>> For the record, I was testing that same combo Claude mentioned and it did not work out of the box but it is definitely possible to set up successfully. I do not remember the details.
>>>>>>> 
>>>>>>> To replay to Maxim, it all seems good to me, roughly, but I humbly think it all boils down to Maven/Gradle refactoring and on top of that we can do all else.
>>>>>>> 
>>>>>>> For example, there is (1) where the solution, besides fixing the tests, is to introduce an Ant task which would check this on build. That being said, how is that going to look like when we change Ant for something else? That stuff suddenly becomes obsolete.
>>>>>>> 
>>>>>>> This case maybe applies to other problems we want to solve as well. I do not want to do something tailored for one build system just to rewrite it all or to spend significant amount of time on that again when we switch the build system.
>>>>>>> 
>>>>>>> For that reason I think changing Ant for something else should be top priority (as I understand that it the hot topic for community for very long time) and then everything else should follow. We should spend time on things mentioned only in case they do not collide with any build system at all.
>>>>>>> 
>>>>>>> (1) https://issues.apache.org/jira/browse/CASSANDRA-17964
>>>>>>> 
>>>>>>> Stefan
>>>>>>> 
>>>>>>> ________________________________________
>>>>>>> From: Claude Warren, Jr via dev <de...@cassandra.apache.org>
>>>>>>> Sent: Friday, November 25, 2022 10:16
>>>>>>> To: dev@cassandra.apache.org
>>>>>>> Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis
>>>>>>> 
>>>>>>> NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> +1 for the concept as a whole.  I am certain I could find nits to pick if I looked deeply.
>>>>>>> 
>>>>>>> @mck -- I did have a problem with Cassandra + Eclipse + Java11 (Classpath).  I gave up and am spending time trying to learn IntelliJ.  I also mentioned it in one of the discussion areas.
>>>>>>> 
>>>>>>> Claude
>>>>>>> 
>>>>>>>> On Thu, Nov 24, 2022 at 8:55 PM Mick Semb Wever <mc...@apache.org>> wrote:
>>>>>>> Thank you for a solid write up Maxim. And welcome to Cassandra, it's
>>>>>>> very positive to see you here.
>>>>>>> 
>>>>>>> I whole-heartedly agree with nearly everything you write. Some input
>>>>>>> and questions inline.
>>>>>>> 
>>>>>>> 
>>>>>>>> 
>>>>>>>> As you may know, the infrastructure team has disabled public sign-up
>>>>>>>> to ASF JIRA (the GitHub issues are recommended instead).
>>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> I suspect (based on chatter in general, but not on dev@ AFAIK) is to
>>>>>>> avoid GH issues and stick with jira. The sign-up hurdle we will
>>>>>>> document on the website: CASSANDRA-18064
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>>> == 1. Make the checkstyle config a single point of truth for the
>>>>>>>> source code style. ==
>>>>>>>> 
>>>>>>>> The checkstyle is already used and included in the Cassandra project
>>>>>>>> build lifecycle (ant command line, Jenkins, CircleCI). There is no
>>>>>>>> need to maintain code style configurations for different types of IDEs
>>>>>>>> (e.g. IntelliJ inspections configuration) since the checkstyle.xml
>>>>>>>> file can be directly imported to IDE used by a developer. This is fair
>>>>>>>> for Intellij Idea, NetBeans, and Eclipse.
>>>>>>> 
>>>>>>> 
>>>>>>> Big +1
>>>>>>> 
>>>>>>> 
>>>>>>>> So, I propose to focus on the checks themselves and checking pull
>>>>>>>> requests with automation scripts, rather than maintaining these
>>>>>>>> integrations. The benefits here are avoiding all issues with
>>>>>>>> maintaining configurations for different IDEs. Another good advantage
>>>>>>>> of this approach would be the ability to add new checkstyle rules
>>>>>>>> without touching IDE configuration - and such tickets will be LFH and
>>>>>>>> easy to commit.
>>>>>>>> 
>>>>>>>> The actions points here are:
>>>>>>>> 
>>>>>>>> - create an umbrella JIRA ticket for all checkstyle issues e.g. [8]
>>>>>>>> (or label checkstyle);
>>>>>>>> - add checkstyle to GitHub pull requests using GitHub actions (execute
>>>>>>>> ant command);
>>>>>>> 
>>>>>>> 
>>>>>>> Instead of custom GHA scripting, please use our existing
>>>>>>> cassandra-artifact.sh (which should already include all such checks).
>>>>>>> 
>>>>>>> Something like https://github.com/apache/cassandra/compare/cassandra-3.11...thelastpickle:cassandra:mck/github-actions/3.11
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>>> == 3. Enable pushing backwards build results for both Jenkins and
>>>>>>>> CircleCI to GitHub pull requests. ==
>>>>>>>> 
>>>>>>>> The goal here is to have a "green checkbox" for those GitHub pull
>>>>>>>> requests that have corresponding Jenkins/CircleCI runs. According to
>>>>>>>> the following links, it is completely possible to have those.
>>>>>>>> 
>>>>>>>> https://github.com/jenkinsci/github-branch-source-plugin
>>>>>>>> https://circleci.com/docs/enable-checks/
>>>>>>>> 
>>>>>>>> Moreover, the GitHub Branch Source Plugin is already enabled for the
>>>>>>>> Cassandra project [16]. The same seems should work the same way for
>>>>>>>> CirleCI, but I have faced the infrastructure team comment [17] that
>>>>>>>> describes admin permissions are required for that, so the question is
>>>>>>>> still open here. I could dig a bit more once we've agreed on it.
>>>>>>>> 
>>>>>>>> The actions points here are:
>>>>>>>> - enable Jenkins integration for GitHub pull requests;
>>>>>>>> - enable CircleCI integration for GitHub pull requests;
>>>>>>> 
>>>>>>> 
>>>>>>> Some folk use CircleCI, some use ci-cassandra. The green checkbox idea
>>>>>>> is great, so long as it's optional. We don't want PRs triggering the
>>>>>>> runs either (there are other mechanisms for triggering for now).
>>>>>>> 
>>>>>>> 
>>>>>>>> The actions points here are:
>>>>>>>> 
>>>>>>>> - initiate a wide survey for user and dev lists, to get to know about
>>>>>>>> the usages;
>>>>>>>> - remove those configurations that are not used anymore;
>>>>>>>> - force migration from Ant to Gradle/Maven;
>>>>>>> 
>>>>>>> 
>>>>>>> Let's leave this out for now. There's too many unknowns here. If
>>>>>>> there's an IDE configuration that's broken, no one has reported it for
>>>>>>> ages, and no one is around to fix it, then I say we should raise the
>>>>>>> discussion to remove it.
>>>>>>> 
>>>>>>> The Gradle/Maven migration is a hot one, contribute to that discussion
>>>>>>> but let's not tangle this work up with it, IMHO.
>>>>>>> 
>>>>>>> Totally agree that IDE project files should be as light weight as possible.
>>>>>>> 
>>>>>>> 
>>>>>>>> Summarizing everything proposed above I think it is possible to
>>>>>>>> simplify adding small contributions easier to the codebase as well as
>>>>>>>> save a bunch of committer's time.
>>>>>>>> 
>>>>>>>> So,
>>>>>>>> WDYT about the things described above?
>>>>>>>> Should I create a CEP for this?
>>>>>>> 
>>>>>>> 
>>>>>>> I see no need for a CEP here. An epic and tickets will work.
>>>>>>> Again, thanks for the input Maxim!
>>>>> 
>>> 
>>> 
>>> 
> 

Re: [DISCUSSION] Cassandra's code style and source code analysis

[David Capwell <dc...@apple.com>]

I am strong +1 to new linters, I have been working on SpotBugs but not sent a patch due to sickness and holidays…

About the check style as the source of truth for the style guides, I am +1 to this as well… I feel that wiki is a bad place for this and we can use the check style file to generate the wiki text (no idea if there are tools for this…).  About the comment on “hard” requirements… my understanding has been that new changes must follow and I know several reviewers who act this way in the review process…. If we agree on a style, why do we want to keep allowing ignoring it?

> There’s always a handful of people asking for it, but notably few if any of the full time contributors doing the majority of the core development of Cassandra.


I don’t agree with that statement, I know several committers/PMC who want to switch, but every 6 months this gets brought up and gets shot down… so we just give up trying… 

> I never really see a good argument articulated for the migration, besides general hand waving that ant is old, and people like newer build systems. Ant is working fine, so there isn’t a strong technical reason to replace it, and there are good organisational reasons not to.

I don’t know how Mick feels, but I tend to find that maintaining our current Ant build is a big waste of my time, and that every time I need to go to this layer its far more brittle than it should be…. There are pushes to refactor to try to get more order in the build (side conversation between Mick and I), but these tend to hide the complexity from most…. I strongly feel that our current build is duct taping things together, this isn’t a complaint of the maintainers but more how Ant works…

And there are a few things I feel newer build systems offer that actually make my life better:

1) cache -  I hate running tests in a loop as this always has the same pattern: rebuild everything from scratch for 1m to run a single test that takes 30 milliseconds…. With ant we “could” force our tasks to cache, but then we have to go do that for everything… we are basically rewriting and reimplementing what other builds have

2) consistency - CI runs tests using 1 task, but people like to run tests using a different task… they tend to have different setup and act slightly differently… I had to go improve this for jvm-dtest and see that simulator is in the same shape (if you run simulator tests using the normal CI task, they will fail; you need the bulk task that doesn’t have the same features as the CI version)

3) reusable work, avoid copy/paste - when we add new jars/artifacts we need to copy/paste a set of tasks to implement a “sub-module” or “sub-project” (depending on build tool you prefer), not only is this very easy to get wrong, it discourages splitting work into these units (we talked about moving utils/concurrent into its own jar for Accord…)

4) only having to know 1 build system - right now we are Ant + Maven; to maintain this project you need to understand both… you can get by not knowing this until you touch dependencies, and if you need tasks that also know about those dependencies… you need to see how this is taped together and grasp that logic…

Maybe I am bitter as it took a whole day to get SpotBugs working with Ant… and it is just the following in other builds:

Gradle: apply plugin: “com.github.spotbugs"
Maven: 

<reporting>
    <plugins>
      <plugin>
        <groupId>com.github.spotbugs</groupId>
        <artifactId>spotbugs-maven-plugin</artifactId>
      </plugin>
    </plugins>
  </reporting>


> On Nov 28, 2022, at 10:46 AM, Maxim Muzafarov <mm...@apache.org> wrote:
> 
> Thank you all for the feedback and productive discussion.
> 
> 
> I couldn't have formed my thoughts on the build tools for the product
> better and provided such good examples than Scott did. Rephrasing what
> I wrote in the first letter, seeing Maven/Gradle in the project
> underfoot, a modern IDE will take care of all the necessary files and
> configurations for themselves much better than we do with scripts. I
> fully agree that there is no rush with such migration, and the
> databases in such cases must be more conservative than progressive,
> and not change anything without strong benefits and a broad consensus
> on it. I still believe this consensus can be reached in future and
> when (and if) the consensus will be reached, a clear migration plan
> should be developed for several releases ahead as well. There's still
> a lot of work to be done here that's why I mentioned it at the end of
> my proposal, so as not to pay too much attention to this question at
> this moment.
> I've added a link to this thread to the JIRA issue [1], so we don't
> lose the insights mentioned by members above.
> 
> 
> I want to take away your concerns about lints expansion for now. I
> thought first of all about making all the source code-checking tools
> more convenient for use with a minimal set of already existing lints
> rather than adding or forcing new rules. I really want to avoid here
> cases with storing multiple configurations for a single tool e.g.
> having different configurations for 'optional' or 'mandatory' checks
> as well as different configurations for 'production' or 'tests'.
> 
> Thus, the ideal picture in my mind of all discussed above is :
> 
> We have:
> - checkstyle
> - SpotBugs
> - Sonar
> 
> They work the same way for:
> - Jenkins builds
> - CirleCI builds
> - GitHub pull requests
> - build on the local machine
> 
> For all that, we have the code style webpage [2] (and wiki [3]) is
> pretty well described, there is no need to expand checking tools with
> new rules until we will get working these tools on the minimal set of
> rules. For instance, we can pick up for the checkstyle 'Unused
> imports', 'Import order', for the SpotBugs 'AutoCloseable',
> 'Number.valueOf', for Sonar - only reports to monitor the source code
> trends.
> 
> I agree that adding new lints require a broad consensus, so I'd like
> to avoid such debatable questions for now. Moreover, even with the
> lints already agreed upon, it is still risky to implement some of them
> because they can contain a lot of boilerplate changes and may affect
> more important fixes ready for merge.
> 
> 
> So, as a first step, I can invest my time into the checkstyle tool and
> make it work everywhere with the same configuration.
> WDYT?
> 
> 
> P.S.
> 
> For IntelliJ with the Checkstyle Plugin it's easy to import the
> checkstyle.xml the following way:
> Preferences -> Code Style -> Show Scheme Actions (wheel) -> Import
> scheme -> Checkstyle configuration.
> 
> 
> [1] https://issues.apache.org/jira/browse/CASSANDRA-17015
> [2] https://cassandra.apache.org/_/development/code_style.html
> [3] https://cwiki.apache.org/confluence/display/CASSANDRA2/CodeStyle
> 
> On Sun, 27 Nov 2022 at 13:17, Josh McKenzie <jm...@apache.org> wrote:
>> 
>> My .02 on the build discussion is we should try and keep the guts of that in one place, be it the other email thread or on JIRA. Some insightful points made on this thread but would hate to see this thread derailed on a complex independent topic as well as see some of these points lost on the other discussion.
>> 
>> I think there needs to be a lot of community consensus on the broad expansion of lints that can reject patches.
>> 
>> +1. It may be worthwhile to configure 2 tiers of lints, optional and required, so we can move to a more gradual process of cleaning up lint violations for those that are interested in that type of work. I know in the past we've seen value in looking at the diff in linting violations even w/a 1k+ noisy violation environment.
>> 
>> 
>> On Fri, Nov 25, 2022, at 12:41 PM, scott@paradoxica.net wrote:
>> 
>> For me, the strongest arguments in favor of adopting a modern build tool like Maven or Gradle are their ecosystems - both explicit (in terms of plugins), and implicit (in terms of nearly all build tooling supporting both of them, but not ant).
>> 
>> Investment in Ant - and in tooling that integrates with Ant - fell off years ago. This makes integrating build-phase aspects of Cassandra with other tooling a very frustrating task that users of most build tools get for free. Many tools built in the last several years don’t support it, or do so only as an afterthought.
>> 
>> Two recent examples that have caused pain for me, which I suspect are felt by many:
>> 
>> – Integration with internal build systems at many companies that develop Cassandra. Because ant has fallen into disuse, this integration is heavily manual instead of automatic and free. It usually requires forking the project’s build.xml, developing custom tooling around it, or creating a mock Gradle build that wraps ant lifecycle tasks (which also requires overriding ant tasks whose names clash).
>> 
>> – Security toolchain integration. Many users and developers of Cassandra also integrate with security tooling at their respective companies. Because Ant has fallen into disuse, most tooling commonly used by security organizations doesn’t support it. SBOMs are a good example, as their introduction postdates ant’s decline. Maven plugins exist to generate them in CycloneDX and SPDX format, but no such plugins exist for ant. Cassandra users and developers who need them must manually write shims to produce SBOMs that users of modern build tools get for free.
>> 
>> These might not be use cases anticipated by the project, but they represent work I suspect a large number of contributors to the project are required to perform to make Cassandra usable for them.
>> 
>> The ecosystem point means that the fix for this has to be external to the project if the project continues to use Ant: lots of one-off scripts and shims unsuitable for contribution with sole maintainers at their respective companies to provide functionality that users of modern build tools get for free. It also demands continuous, incremental work to adapt to changes in security and build tooling in use at many companies that don’t need to be made for projects using well-supported tools like Maven or Gradle.
>> 
>> – Scott
>> 
>>> On Nov 25, 2022, at 4:56 AM, Benedict <be...@apache.org> wrote:
>>> 
>>> I think modules are already fairly well supported - we in effect already have several? FQL, Simulator and others I think.
>>> 
>>> I can anyway say with absolute certainty that the main impediment to modularising is not build tooling, it’s the difficulty of the task, the cost to the project of undertaking it, and therefore its relative payoff versus other things that could be undertaken by the folk with the relevant expertise to do it.
>>> 
>>> I’ve also been around long enough to see enough efforts to broaden contributions fail to have an impact, that basing costly decisions on this kind of reasoning doesn’t resonate. The main impediments to contributions are the complexity of the codebase (and problem domain) and our limited capacity to respond promptly to high quality contributions. Until we fix those fundamental issues, this kind of rearranging of chairs seems more like a branding exercise to ourselves than to anyone else.
>>> 
>>> 
>>>> On 25 Nov 2022, at 10:22, Miklosovic, Stefan <St...@netapp.com> wrote:
>>>> 
>>>> I agree with what you wrote. How I understand it is that migrating to Maven/Gradle makes the project more "attractive" for newcomers. If a project is built on "that old un-cool Ant", it might be a little bit off-putting and questionable if we are "stuck in the past on build systems and not progressing".
>>>> 
>>>> So in that sense I agree this is more "marketing" rather than technological question but on the other hand, does not Maven/Gradle allow us to modularize the project better? Maybe we would like to modularize but nobody is up to that because build system makes it impossible or at least quite inconvenient to do so. Do you really think there are not any significant benefits to switch even if it "just works" now?
>>>> 
>>>> ________________________________________
>>>> From: Benedict <be...@apache.org>
>>>> Sent: Friday, November 25, 2022 11:07
>>>> To: dev@cassandra.apache.org
>>>> Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis
>>>> 
>>>> NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.
>>>> 
>>>> 
>>>> 
>>>> 
>>>> There’s always a handful of people asking for it, but notably few if any of the full time contributors doing the majority of the core development of Cassandra. It strikes me as something very appealing to others, but less so to those wanting to get on with development.
>>>> 
>>>> I never really see a good argument articulated for the migration, besides general hand waving that ant is old, and people like newer build systems. Ant is working fine, so there isn’t a strong technical reason to replace it, and there are good organisational reasons not to.
>>>> 
>>>> Why do you consider a migration inevitable?
>>>> 
>>>> 
>>>> 
>>>>> On 25 Nov 2022, at 09:58, Miklosovic, Stefan <St...@netapp.com> wrote:
>>>>> 
>>>>> Interesting take on Ant / no-Ant, Benedict. I am very curious how this unfolds. My long-term perception is that changing it to something else is more or less inevitable but if there is a broader consensus to not do that .... well.
>>>>> 
>>>>> ________________________________________
>>>>> From: Benedict <be...@apache.org>
>>>>> Sent: Friday, November 25, 2022 10:52
>>>>> To: dev@cassandra.apache.org
>>>>> Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis
>>>>> 
>>>>> NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> I was in a bit of a rush last night. I should say that I’m of course +1 a general endeavour to clean this up, and to expand our use of linters, and I appreciate your volunteering to help out in this way Maxim.
>>>>> 
>>>>> However, responding to Stefan, I’m pretty -1 migrating from ant to another build system without really good reason. Migration has a real cost to productivity for all existing contributors, and the phantom of increasing new contributions has never paid off historically. I’m all for easing people into participation, but not at penalty to the existing contributor base.
>>>>> 
>>>>> If the only reason is to make it easier to open in a different IDE, we can perhaps have some basic build files outlining code structure for importing, that are compatible with our canonical ant build? We could perhaps even generate them.
>>>>> 
>>>>> 
>>>>>>> On 25 Nov 2022, at 09:35, Miklosovic, Stefan <St...@netapp.com> wrote:
>>>>>> 
>>>>>> For the record, I was testing that same combo Claude mentioned and it did not work out of the box but it is definitely possible to set up successfully. I do not remember the details.
>>>>>> 
>>>>>> To replay to Maxim, it all seems good to me, roughly, but I humbly think it all boils down to Maven/Gradle refactoring and on top of that we can do all else.
>>>>>> 
>>>>>> For example, there is (1) where the solution, besides fixing the tests, is to introduce an Ant task which would check this on build. That being said, how is that going to look like when we change Ant for something else? That stuff suddenly becomes obsolete.
>>>>>> 
>>>>>> This case maybe applies to other problems we want to solve as well. I do not want to do something tailored for one build system just to rewrite it all or to spend significant amount of time on that again when we switch the build system.
>>>>>> 
>>>>>> For that reason I think changing Ant for something else should be top priority (as I understand that it the hot topic for community for very long time) and then everything else should follow. We should spend time on things mentioned only in case they do not collide with any build system at all.
>>>>>> 
>>>>>> (1) https://issues.apache.org/jira/browse/CASSANDRA-17964
>>>>>> 
>>>>>> Stefan
>>>>>> 
>>>>>> ________________________________________
>>>>>> From: Claude Warren, Jr via dev <de...@cassandra.apache.org>
>>>>>> Sent: Friday, November 25, 2022 10:16
>>>>>> To: dev@cassandra.apache.org
>>>>>> Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis
>>>>>> 
>>>>>> NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> +1 for the concept as a whole.  I am certain I could find nits to pick if I looked deeply.
>>>>>> 
>>>>>> @mck -- I did have a problem with Cassandra + Eclipse + Java11 (Classpath).  I gave up and am spending time trying to learn IntelliJ.  I also mentioned it in one of the discussion areas.
>>>>>> 
>>>>>> Claude
>>>>>> 
>>>>>>> On Thu, Nov 24, 2022 at 8:55 PM Mick Semb Wever <mc...@apache.org>> wrote:
>>>>>> Thank you for a solid write up Maxim. And welcome to Cassandra, it's
>>>>>> very positive to see you here.
>>>>>> 
>>>>>> I whole-heartedly agree with nearly everything you write. Some input
>>>>>> and questions inline.
>>>>>> 
>>>>>> 
>>>>>>> 
>>>>>>> As you may know, the infrastructure team has disabled public sign-up
>>>>>>> to ASF JIRA (the GitHub issues are recommended instead).
>>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> I suspect (based on chatter in general, but not on dev@ AFAIK) is to
>>>>>> avoid GH issues and stick with jira. The sign-up hurdle we will
>>>>>> document on the website: CASSANDRA-18064
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>>> == 1. Make the checkstyle config a single point of truth for the
>>>>>>> source code style. ==
>>>>>>> 
>>>>>>> The checkstyle is already used and included in the Cassandra project
>>>>>>> build lifecycle (ant command line, Jenkins, CircleCI). There is no
>>>>>>> need to maintain code style configurations for different types of IDEs
>>>>>>> (e.g. IntelliJ inspections configuration) since the checkstyle.xml
>>>>>>> file can be directly imported to IDE used by a developer. This is fair
>>>>>>> for Intellij Idea, NetBeans, and Eclipse.
>>>>>> 
>>>>>> 
>>>>>> Big +1
>>>>>> 
>>>>>> 
>>>>>>> So, I propose to focus on the checks themselves and checking pull
>>>>>>> requests with automation scripts, rather than maintaining these
>>>>>>> integrations. The benefits here are avoiding all issues with
>>>>>>> maintaining configurations for different IDEs. Another good advantage
>>>>>>> of this approach would be the ability to add new checkstyle rules
>>>>>>> without touching IDE configuration - and such tickets will be LFH and
>>>>>>> easy to commit.
>>>>>>> 
>>>>>>> The actions points here are:
>>>>>>> 
>>>>>>> - create an umbrella JIRA ticket for all checkstyle issues e.g. [8]
>>>>>>> (or label checkstyle);
>>>>>>> - add checkstyle to GitHub pull requests using GitHub actions (execute
>>>>>>> ant command);
>>>>>> 
>>>>>> 
>>>>>> Instead of custom GHA scripting, please use our existing
>>>>>> cassandra-artifact.sh (which should already include all such checks).
>>>>>> 
>>>>>> Something like https://github.com/apache/cassandra/compare/cassandra-3.11...thelastpickle:cassandra:mck/github-actions/3.11
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>>> == 3. Enable pushing backwards build results for both Jenkins and
>>>>>>> CircleCI to GitHub pull requests. ==
>>>>>>> 
>>>>>>> The goal here is to have a "green checkbox" for those GitHub pull
>>>>>>> requests that have corresponding Jenkins/CircleCI runs. According to
>>>>>>> the following links, it is completely possible to have those.
>>>>>>> 
>>>>>>> https://github.com/jenkinsci/github-branch-source-plugin
>>>>>>> https://circleci.com/docs/enable-checks/
>>>>>>> 
>>>>>>> Moreover, the GitHub Branch Source Plugin is already enabled for the
>>>>>>> Cassandra project [16]. The same seems should work the same way for
>>>>>>> CirleCI, but I have faced the infrastructure team comment [17] that
>>>>>>> describes admin permissions are required for that, so the question is
>>>>>>> still open here. I could dig a bit more once we've agreed on it.
>>>>>>> 
>>>>>>> The actions points here are:
>>>>>>> - enable Jenkins integration for GitHub pull requests;
>>>>>>> - enable CircleCI integration for GitHub pull requests;
>>>>>> 
>>>>>> 
>>>>>> Some folk use CircleCI, some use ci-cassandra. The green checkbox idea
>>>>>> is great, so long as it's optional. We don't want PRs triggering the
>>>>>> runs either (there are other mechanisms for triggering for now).
>>>>>> 
>>>>>> 
>>>>>>> The actions points here are:
>>>>>>> 
>>>>>>> - initiate a wide survey for user and dev lists, to get to know about
>>>>>>> the usages;
>>>>>>> - remove those configurations that are not used anymore;
>>>>>>> - force migration from Ant to Gradle/Maven;
>>>>>> 
>>>>>> 
>>>>>> Let's leave this out for now. There's too many unknowns here. If
>>>>>> there's an IDE configuration that's broken, no one has reported it for
>>>>>> ages, and no one is around to fix it, then I say we should raise the
>>>>>> discussion to remove it.
>>>>>> 
>>>>>> The Gradle/Maven migration is a hot one, contribute to that discussion
>>>>>> but let's not tangle this work up with it, IMHO.
>>>>>> 
>>>>>> Totally agree that IDE project files should be as light weight as possible.
>>>>>> 
>>>>>> 
>>>>>>> Summarizing everything proposed above I think it is possible to
>>>>>>> simplify adding small contributions easier to the codebase as well as
>>>>>>> save a bunch of committer's time.
>>>>>>> 
>>>>>>> So,
>>>>>>> WDYT about the things described above?
>>>>>>> Should I create a CEP for this?
>>>>>> 
>>>>>> 
>>>>>> I see no need for a CEP here. An epic and tickets will work.
>>>>>> Again, thanks for the input Maxim!
>>>> 
>> 
>> 
>> 

Re: [DISCUSSION] Cassandra's code style and source code analysis

[Maxim Muzafarov <mm...@apache.org>]

Thank you all for the feedback and productive discussion.


I couldn't have formed my thoughts on the build tools for the product
better and provided such good examples than Scott did. Rephrasing what
I wrote in the first letter, seeing Maven/Gradle in the project
underfoot, a modern IDE will take care of all the necessary files and
configurations for themselves much better than we do with scripts. I
fully agree that there is no rush with such migration, and the
databases in such cases must be more conservative than progressive,
and not change anything without strong benefits and a broad consensus
on it. I still believe this consensus can be reached in future and
when (and if) the consensus will be reached, a clear migration plan
should be developed for several releases ahead as well. There's still
a lot of work to be done here that's why I mentioned it at the end of
my proposal, so as not to pay too much attention to this question at
this moment.
I've added a link to this thread to the JIRA issue [1], so we don't
lose the insights mentioned by members above.


I want to take away your concerns about lints expansion for now. I
thought first of all about making all the source code-checking tools
more convenient for use with a minimal set of already existing lints
rather than adding or forcing new rules. I really want to avoid here
cases with storing multiple configurations for a single tool e.g.
having different configurations for 'optional' or 'mandatory' checks
as well as different configurations for 'production' or 'tests'.

Thus, the ideal picture in my mind of all discussed above is :

We have:
- checkstyle
- SpotBugs
- Sonar

They work the same way for:
- Jenkins builds
- CirleCI builds
- GitHub pull requests
- build on the local machine

For all that, we have the code style webpage [2] (and wiki [3]) is
pretty well described, there is no need to expand checking tools with
new rules until we will get working these tools on the minimal set of
rules. For instance, we can pick up for the checkstyle 'Unused
imports', 'Import order', for the SpotBugs 'AutoCloseable',
'Number.valueOf', for Sonar - only reports to monitor the source code
trends.

I agree that adding new lints require a broad consensus, so I'd like
to avoid such debatable questions for now. Moreover, even with the
lints already agreed upon, it is still risky to implement some of them
because they can contain a lot of boilerplate changes and may affect
more important fixes ready for merge.


So, as a first step, I can invest my time into the checkstyle tool and
make it work everywhere with the same configuration.
WDYT?


P.S.

For IntelliJ with the Checkstyle Plugin it's easy to import the
checkstyle.xml the following way:
Preferences -> Code Style -> Show Scheme Actions (wheel) -> Import
scheme -> Checkstyle configuration.


[1] https://issues.apache.org/jira/browse/CASSANDRA-17015
[2] https://cassandra.apache.org/_/development/code_style.html
[3] https://cwiki.apache.org/confluence/display/CASSANDRA2/CodeStyle

On Sun, 27 Nov 2022 at 13:17, Josh McKenzie <jm...@apache.org> wrote:
>
> My .02 on the build discussion is we should try and keep the guts of that in one place, be it the other email thread or on JIRA. Some insightful points made on this thread but would hate to see this thread derailed on a complex independent topic as well as see some of these points lost on the other discussion.
>
> I think there needs to be a lot of community consensus on the broad expansion of lints that can reject patches.
>
> +1. It may be worthwhile to configure 2 tiers of lints, optional and required, so we can move to a more gradual process of cleaning up lint violations for those that are interested in that type of work. I know in the past we've seen value in looking at the diff in linting violations even w/a 1k+ noisy violation environment.
>
>
> On Fri, Nov 25, 2022, at 12:41 PM, scott@paradoxica.net wrote:
>
> For me, the strongest arguments in favor of adopting a modern build tool like Maven or Gradle are their ecosystems - both explicit (in terms of plugins), and implicit (in terms of nearly all build tooling supporting both of them, but not ant).
>
> Investment in Ant - and in tooling that integrates with Ant - fell off years ago. This makes integrating build-phase aspects of Cassandra with other tooling a very frustrating task that users of most build tools get for free. Many tools built in the last several years don’t support it, or do so only as an afterthought.
>
> Two recent examples that have caused pain for me, which I suspect are felt by many:
>
> – Integration with internal build systems at many companies that develop Cassandra. Because ant has fallen into disuse, this integration is heavily manual instead of automatic and free. It usually requires forking the project’s build.xml, developing custom tooling around it, or creating a mock Gradle build that wraps ant lifecycle tasks (which also requires overriding ant tasks whose names clash).
>
> – Security toolchain integration. Many users and developers of Cassandra also integrate with security tooling at their respective companies. Because Ant has fallen into disuse, most tooling commonly used by security organizations doesn’t support it. SBOMs are a good example, as their introduction postdates ant’s decline. Maven plugins exist to generate them in CycloneDX and SPDX format, but no such plugins exist for ant. Cassandra users and developers who need them must manually write shims to produce SBOMs that users of modern build tools get for free.
>
> These might not be use cases anticipated by the project, but they represent work I suspect a large number of contributors to the project are required to perform to make Cassandra usable for them.
>
> The ecosystem point means that the fix for this has to be external to the project if the project continues to use Ant: lots of one-off scripts and shims unsuitable for contribution with sole maintainers at their respective companies to provide functionality that users of modern build tools get for free. It also demands continuous, incremental work to adapt to changes in security and build tooling in use at many companies that don’t need to be made for projects using well-supported tools like Maven or Gradle.
>
> – Scott
>
> > On Nov 25, 2022, at 4:56 AM, Benedict <be...@apache.org> wrote:
> >
> > I think modules are already fairly well supported - we in effect already have several? FQL, Simulator and others I think.
> >
> > I can anyway say with absolute certainty that the main impediment to modularising is not build tooling, it’s the difficulty of the task, the cost to the project of undertaking it, and therefore its relative payoff versus other things that could be undertaken by the folk with the relevant expertise to do it.
> >
> > I’ve also been around long enough to see enough efforts to broaden contributions fail to have an impact, that basing costly decisions on this kind of reasoning doesn’t resonate. The main impediments to contributions are the complexity of the codebase (and problem domain) and our limited capacity to respond promptly to high quality contributions. Until we fix those fundamental issues, this kind of rearranging of chairs seems more like a branding exercise to ourselves than to anyone else.
> >
> >
> >> On 25 Nov 2022, at 10:22, Miklosovic, Stefan <St...@netapp.com> wrote:
> >>
> >> I agree with what you wrote. How I understand it is that migrating to Maven/Gradle makes the project more "attractive" for newcomers. If a project is built on "that old un-cool Ant", it might be a little bit off-putting and questionable if we are "stuck in the past on build systems and not progressing".
> >>
> >> So in that sense I agree this is more "marketing" rather than technological question but on the other hand, does not Maven/Gradle allow us to modularize the project better? Maybe we would like to modularize but nobody is up to that because build system makes it impossible or at least quite inconvenient to do so. Do you really think there are not any significant benefits to switch even if it "just works" now?
> >>
> >> ________________________________________
> >> From: Benedict <be...@apache.org>
> >> Sent: Friday, November 25, 2022 11:07
> >> To: dev@cassandra.apache.org
> >> Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis
> >>
> >> NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.
> >>
> >>
> >>
> >>
> >> There’s always a handful of people asking for it, but notably few if any of the full time contributors doing the majority of the core development of Cassandra. It strikes me as something very appealing to others, but less so to those wanting to get on with development.
> >>
> >> I never really see a good argument articulated for the migration, besides general hand waving that ant is old, and people like newer build systems. Ant is working fine, so there isn’t a strong technical reason to replace it, and there are good organisational reasons not to.
> >>
> >> Why do you consider a migration inevitable?
> >>
> >>
> >>
> >>> On 25 Nov 2022, at 09:58, Miklosovic, Stefan <St...@netapp.com> wrote:
> >>>
> >>> Interesting take on Ant / no-Ant, Benedict. I am very curious how this unfolds. My long-term perception is that changing it to something else is more or less inevitable but if there is a broader consensus to not do that .... well.
> >>>
> >>> ________________________________________
> >>> From: Benedict <be...@apache.org>
> >>> Sent: Friday, November 25, 2022 10:52
> >>> To: dev@cassandra.apache.org
> >>> Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis
> >>>
> >>> NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.
> >>>
> >>>
> >>>
> >>>
> >>> I was in a bit of a rush last night. I should say that I’m of course +1 a general endeavour to clean this up, and to expand our use of linters, and I appreciate your volunteering to help out in this way Maxim.
> >>>
> >>> However, responding to Stefan, I’m pretty -1 migrating from ant to another build system without really good reason. Migration has a real cost to productivity for all existing contributors, and the phantom of increasing new contributions has never paid off historically. I’m all for easing people into participation, but not at penalty to the existing contributor base.
> >>>
> >>> If the only reason is to make it easier to open in a different IDE, we can perhaps have some basic build files outlining code structure for importing, that are compatible with our canonical ant build? We could perhaps even generate them.
> >>>
> >>>
> >>>>> On 25 Nov 2022, at 09:35, Miklosovic, Stefan <St...@netapp.com> wrote:
> >>>>
> >>>> For the record, I was testing that same combo Claude mentioned and it did not work out of the box but it is definitely possible to set up successfully. I do not remember the details.
> >>>>
> >>>> To replay to Maxim, it all seems good to me, roughly, but I humbly think it all boils down to Maven/Gradle refactoring and on top of that we can do all else.
> >>>>
> >>>> For example, there is (1) where the solution, besides fixing the tests, is to introduce an Ant task which would check this on build. That being said, how is that going to look like when we change Ant for something else? That stuff suddenly becomes obsolete.
> >>>>
> >>>> This case maybe applies to other problems we want to solve as well. I do not want to do something tailored for one build system just to rewrite it all or to spend significant amount of time on that again when we switch the build system.
> >>>>
> >>>> For that reason I think changing Ant for something else should be top priority (as I understand that it the hot topic for community for very long time) and then everything else should follow. We should spend time on things mentioned only in case they do not collide with any build system at all.
> >>>>
> >>>> (1) https://issues.apache.org/jira/browse/CASSANDRA-17964
> >>>>
> >>>> Stefan
> >>>>
> >>>> ________________________________________
> >>>> From: Claude Warren, Jr via dev <de...@cassandra.apache.org>
> >>>> Sent: Friday, November 25, 2022 10:16
> >>>> To: dev@cassandra.apache.org
> >>>> Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis
> >>>>
> >>>> NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.
> >>>>
> >>>>
> >>>>
> >>>> +1 for the concept as a whole.  I am certain I could find nits to pick if I looked deeply.
> >>>>
> >>>> @mck -- I did have a problem with Cassandra + Eclipse + Java11 (Classpath).  I gave up and am spending time trying to learn IntelliJ.  I also mentioned it in one of the discussion areas.
> >>>>
> >>>> Claude
> >>>>
> >>>>> On Thu, Nov 24, 2022 at 8:55 PM Mick Semb Wever <mc...@apache.org>> wrote:
> >>>> Thank you for a solid write up Maxim. And welcome to Cassandra, it's
> >>>> very positive to see you here.
> >>>>
> >>>> I whole-heartedly agree with nearly everything you write. Some input
> >>>> and questions inline.
> >>>>
> >>>>
> >>>>>
> >>>>> As you may know, the infrastructure team has disabled public sign-up
> >>>>> to ASF JIRA (the GitHub issues are recommended instead).
> >>>>>
> >>>>
> >>>>
> >>>> I suspect (based on chatter in general, but not on dev@ AFAIK) is to
> >>>> avoid GH issues and stick with jira. The sign-up hurdle we will
> >>>> document on the website: CASSANDRA-18064
> >>>>
> >>>>
> >>>>
> >>>>> == 1. Make the checkstyle config a single point of truth for the
> >>>>> source code style. ==
> >>>>>
> >>>>> The checkstyle is already used and included in the Cassandra project
> >>>>> build lifecycle (ant command line, Jenkins, CircleCI). There is no
> >>>>> need to maintain code style configurations for different types of IDEs
> >>>>> (e.g. IntelliJ inspections configuration) since the checkstyle.xml
> >>>>> file can be directly imported to IDE used by a developer. This is fair
> >>>>> for Intellij Idea, NetBeans, and Eclipse.
> >>>>
> >>>>
> >>>> Big +1
> >>>>
> >>>>
> >>>>> So, I propose to focus on the checks themselves and checking pull
> >>>>> requests with automation scripts, rather than maintaining these
> >>>>> integrations. The benefits here are avoiding all issues with
> >>>>> maintaining configurations for different IDEs. Another good advantage
> >>>>> of this approach would be the ability to add new checkstyle rules
> >>>>> without touching IDE configuration - and such tickets will be LFH and
> >>>>> easy to commit.
> >>>>>
> >>>>> The actions points here are:
> >>>>>
> >>>>> - create an umbrella JIRA ticket for all checkstyle issues e.g. [8]
> >>>>> (or label checkstyle);
> >>>>> - add checkstyle to GitHub pull requests using GitHub actions (execute
> >>>>> ant command);
> >>>>
> >>>>
> >>>> Instead of custom GHA scripting, please use our existing
> >>>> cassandra-artifact.sh (which should already include all such checks).
> >>>>
> >>>> Something like https://github.com/apache/cassandra/compare/cassandra-3.11...thelastpickle:cassandra:mck/github-actions/3.11
> >>>>
> >>>>
> >>>>
> >>>>> == 3. Enable pushing backwards build results for both Jenkins and
> >>>>> CircleCI to GitHub pull requests. ==
> >>>>>
> >>>>> The goal here is to have a "green checkbox" for those GitHub pull
> >>>>> requests that have corresponding Jenkins/CircleCI runs. According to
> >>>>> the following links, it is completely possible to have those.
> >>>>>
> >>>>> https://github.com/jenkinsci/github-branch-source-plugin
> >>>>> https://circleci.com/docs/enable-checks/
> >>>>>
> >>>>> Moreover, the GitHub Branch Source Plugin is already enabled for the
> >>>>> Cassandra project [16]. The same seems should work the same way for
> >>>>> CirleCI, but I have faced the infrastructure team comment [17] that
> >>>>> describes admin permissions are required for that, so the question is
> >>>>> still open here. I could dig a bit more once we've agreed on it.
> >>>>>
> >>>>> The actions points here are:
> >>>>> - enable Jenkins integration for GitHub pull requests;
> >>>>> - enable CircleCI integration for GitHub pull requests;
> >>>>
> >>>>
> >>>> Some folk use CircleCI, some use ci-cassandra. The green checkbox idea
> >>>> is great, so long as it's optional. We don't want PRs triggering the
> >>>> runs either (there are other mechanisms for triggering for now).
> >>>>
> >>>>
> >>>>> The actions points here are:
> >>>>>
> >>>>> - initiate a wide survey for user and dev lists, to get to know about
> >>>>> the usages;
> >>>>> - remove those configurations that are not used anymore;
> >>>>> - force migration from Ant to Gradle/Maven;
> >>>>
> >>>>
> >>>> Let's leave this out for now. There's too many unknowns here. If
> >>>> there's an IDE configuration that's broken, no one has reported it for
> >>>> ages, and no one is around to fix it, then I say we should raise the
> >>>> discussion to remove it.
> >>>>
> >>>> The Gradle/Maven migration is a hot one, contribute to that discussion
> >>>> but let's not tangle this work up with it, IMHO.
> >>>>
> >>>> Totally agree that IDE project files should be as light weight as possible.
> >>>>
> >>>>
> >>>>> Summarizing everything proposed above I think it is possible to
> >>>>> simplify adding small contributions easier to the codebase as well as
> >>>>> save a bunch of committer's time.
> >>>>>
> >>>>> So,
> >>>>> WDYT about the things described above?
> >>>>> Should I create a CEP for this?
> >>>>
> >>>>
> >>>> I see no need for a CEP here. An epic and tickets will work.
> >>>> Again, thanks for the input Maxim!
> >>
>
>
>

Re: [DISCUSSION] Cassandra's code style and source code analysis

[Josh McKenzie <jm...@apache.org>]

My .02 on the build discussion is we should try and keep the guts of that in one place, be it the other email thread or on JIRA. Some insightful points made on this thread but would hate to see this thread derailed on a complex independent topic as well as see some of these points lost on the other discussion.

> I think there needs to be a lot of community consensus on the broad expansion of lints that can reject patches.
+1. It may be worthwhile to configure 2 tiers of lints, optional and required, so we can move to a more gradual process of cleaning up lint violations for those that are interested in that type of work. I know in the past we've seen value in looking at the diff in linting violations even w/a 1k+ noisy violation environment.


On Fri, Nov 25, 2022, at 12:41 PM, scott@paradoxica.net wrote:
> For me, the strongest arguments in favor of adopting a modern build tool like Maven or Gradle are their ecosystems - both explicit (in terms of plugins), and implicit (in terms of nearly all build tooling supporting both of them, but not ant).
> 
> Investment in Ant - and in tooling that integrates with Ant - fell off years ago. This makes integrating build-phase aspects of Cassandra with other tooling a very frustrating task that users of most build tools get for free. Many tools built in the last several years don’t support it, or do so only as an afterthought.
> 
> Two recent examples that have caused pain for me, which I suspect are felt by many:
> 
> – Integration with internal build systems at many companies that develop Cassandra. Because ant has fallen into disuse, this integration is heavily manual instead of automatic and free. It usually requires forking the project’s build.xml, developing custom tooling around it, or creating a mock Gradle build that wraps ant lifecycle tasks (which also requires overriding ant tasks whose names clash).
> 
> – Security toolchain integration. Many users and developers of Cassandra also integrate with security tooling at their respective companies. Because Ant has fallen into disuse, most tooling commonly used by security organizations doesn’t support it. SBOMs are a good example, as their introduction postdates ant’s decline. Maven plugins exist to generate them in CycloneDX and SPDX format, but no such plugins exist for ant. Cassandra users and developers who need them must manually write shims to produce SBOMs that users of modern build tools get for free.
> 
> These might not be use cases anticipated by the project, but they represent work I suspect a large number of contributors to the project are required to perform to make Cassandra usable for them.
> 
> The ecosystem point means that the fix for this has to be external to the project if the project continues to use Ant: lots of one-off scripts and shims unsuitable for contribution with sole maintainers at their respective companies to provide functionality that users of modern build tools get for free. It also demands continuous, incremental work to adapt to changes in security and build tooling in use at many companies that don’t need to be made for projects using well-supported tools like Maven or Gradle.
> 
> – Scott
> 
> > On Nov 25, 2022, at 4:56 AM, Benedict <be...@apache.org> wrote:
> > 
> > I think modules are already fairly well supported - we in effect already have several? FQL, Simulator and others I think.
> > 
> > I can anyway say with absolute certainty that the main impediment to modularising is not build tooling, it’s the difficulty of the task, the cost to the project of undertaking it, and therefore its relative payoff versus other things that could be undertaken by the folk with the relevant expertise to do it.
> > 
> > I’ve also been around long enough to see enough efforts to broaden contributions fail to have an impact, that basing costly decisions on this kind of reasoning doesn’t resonate. The main impediments to contributions are the complexity of the codebase (and problem domain) and our limited capacity to respond promptly to high quality contributions. Until we fix those fundamental issues, this kind of rearranging of chairs seems more like a branding exercise to ourselves than to anyone else.
> > 
> > 
> >> On 25 Nov 2022, at 10:22, Miklosovic, Stefan <St...@netapp.com> wrote:
> >> 
> >> I agree with what you wrote. How I understand it is that migrating to Maven/Gradle makes the project more "attractive" for newcomers. If a project is built on "that old un-cool Ant", it might be a little bit off-putting and questionable if we are "stuck in the past on build systems and not progressing".
> >> 
> >> So in that sense I agree this is more "marketing" rather than technological question but on the other hand, does not Maven/Gradle allow us to modularize the project better? Maybe we would like to modularize but nobody is up to that because build system makes it impossible or at least quite inconvenient to do so. Do you really think there are not any significant benefits to switch even if it "just works" now?
> >> 
> >> ________________________________________
> >> From: Benedict <be...@apache.org>
> >> Sent: Friday, November 25, 2022 11:07
> >> To: dev@cassandra.apache.org
> >> Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis
> >> 
> >> NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.
> >> 
> >> 
> >> 
> >> 
> >> There’s always a handful of people asking for it, but notably few if any of the full time contributors doing the majority of the core development of Cassandra. It strikes me as something very appealing to others, but less so to those wanting to get on with development.
> >> 
> >> I never really see a good argument articulated for the migration, besides general hand waving that ant is old, and people like newer build systems. Ant is working fine, so there isn’t a strong technical reason to replace it, and there are good organisational reasons not to.
> >> 
> >> Why do you consider a migration inevitable?
> >> 
> >> 
> >> 
> >>> On 25 Nov 2022, at 09:58, Miklosovic, Stefan <St...@netapp.com> wrote:
> >>> 
> >>> Interesting take on Ant / no-Ant, Benedict. I am very curious how this unfolds. My long-term perception is that changing it to something else is more or less inevitable but if there is a broader consensus to not do that .... well.
> >>> 
> >>> ________________________________________
> >>> From: Benedict <be...@apache.org>
> >>> Sent: Friday, November 25, 2022 10:52
> >>> To: dev@cassandra.apache.org
> >>> Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis
> >>> 
> >>> NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.
> >>> 
> >>> 
> >>> 
> >>> 
> >>> I was in a bit of a rush last night. I should say that I’m of course +1 a general endeavour to clean this up, and to expand our use of linters, and I appreciate your volunteering to help out in this way Maxim.
> >>> 
> >>> However, responding to Stefan, I’m pretty -1 migrating from ant to another build system without really good reason. Migration has a real cost to productivity for all existing contributors, and the phantom of increasing new contributions has never paid off historically. I’m all for easing people into participation, but not at penalty to the existing contributor base.
> >>> 
> >>> If the only reason is to make it easier to open in a different IDE, we can perhaps have some basic build files outlining code structure for importing, that are compatible with our canonical ant build? We could perhaps even generate them.
> >>> 
> >>> 
> >>>>> On 25 Nov 2022, at 09:35, Miklosovic, Stefan <St...@netapp.com> wrote:
> >>>> 
> >>>> For the record, I was testing that same combo Claude mentioned and it did not work out of the box but it is definitely possible to set up successfully. I do not remember the details.
> >>>> 
> >>>> To replay to Maxim, it all seems good to me, roughly, but I humbly think it all boils down to Maven/Gradle refactoring and on top of that we can do all else.
> >>>> 
> >>>> For example, there is (1) where the solution, besides fixing the tests, is to introduce an Ant task which would check this on build. That being said, how is that going to look like when we change Ant for something else? That stuff suddenly becomes obsolete.
> >>>> 
> >>>> This case maybe applies to other problems we want to solve as well. I do not want to do something tailored for one build system just to rewrite it all or to spend significant amount of time on that again when we switch the build system.
> >>>> 
> >>>> For that reason I think changing Ant for something else should be top priority (as I understand that it the hot topic for community for very long time) and then everything else should follow. We should spend time on things mentioned only in case they do not collide with any build system at all.
> >>>> 
> >>>> (1) https://issues.apache.org/jira/browse/CASSANDRA-17964
> >>>> 
> >>>> Stefan
> >>>> 
> >>>> ________________________________________
> >>>> From: Claude Warren, Jr via dev <de...@cassandra.apache.org>
> >>>> Sent: Friday, November 25, 2022 10:16
> >>>> To: dev@cassandra.apache.org
> >>>> Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis
> >>>> 
> >>>> NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.
> >>>> 
> >>>> 
> >>>> 
> >>>> +1 for the concept as a whole.  I am certain I could find nits to pick if I looked deeply.
> >>>> 
> >>>> @mck -- I did have a problem with Cassandra + Eclipse + Java11 (Classpath).  I gave up and am spending time trying to learn IntelliJ.  I also mentioned it in one of the discussion areas.
> >>>> 
> >>>> Claude
> >>>> 
> >>>>> On Thu, Nov 24, 2022 at 8:55 PM Mick Semb Wever <mc...@apache.org>> wrote:
> >>>> Thank you for a solid write up Maxim. And welcome to Cassandra, it's
> >>>> very positive to see you here.
> >>>> 
> >>>> I whole-heartedly agree with nearly everything you write. Some input
> >>>> and questions inline.
> >>>> 
> >>>> 
> >>>>> 
> >>>>> As you may know, the infrastructure team has disabled public sign-up
> >>>>> to ASF JIRA (the GitHub issues are recommended instead).
> >>>>> 
> >>>> 
> >>>> 
> >>>> I suspect (based on chatter in general, but not on dev@ AFAIK) is to
> >>>> avoid GH issues and stick with jira. The sign-up hurdle we will
> >>>> document on the website: CASSANDRA-18064
> >>>> 
> >>>> 
> >>>> 
> >>>>> == 1. Make the checkstyle config a single point of truth for the
> >>>>> source code style. ==
> >>>>> 
> >>>>> The checkstyle is already used and included in the Cassandra project
> >>>>> build lifecycle (ant command line, Jenkins, CircleCI). There is no
> >>>>> need to maintain code style configurations for different types of IDEs
> >>>>> (e.g. IntelliJ inspections configuration) since the checkstyle.xml
> >>>>> file can be directly imported to IDE used by a developer. This is fair
> >>>>> for Intellij Idea, NetBeans, and Eclipse.
> >>>> 
> >>>> 
> >>>> Big +1
> >>>> 
> >>>> 
> >>>>> So, I propose to focus on the checks themselves and checking pull
> >>>>> requests with automation scripts, rather than maintaining these
> >>>>> integrations. The benefits here are avoiding all issues with
> >>>>> maintaining configurations for different IDEs. Another good advantage
> >>>>> of this approach would be the ability to add new checkstyle rules
> >>>>> without touching IDE configuration - and such tickets will be LFH and
> >>>>> easy to commit.
> >>>>> 
> >>>>> The actions points here are:
> >>>>> 
> >>>>> - create an umbrella JIRA ticket for all checkstyle issues e.g. [8]
> >>>>> (or label checkstyle);
> >>>>> - add checkstyle to GitHub pull requests using GitHub actions (execute
> >>>>> ant command);
> >>>> 
> >>>> 
> >>>> Instead of custom GHA scripting, please use our existing
> >>>> cassandra-artifact.sh (which should already include all such checks).
> >>>> 
> >>>> Something like https://github.com/apache/cassandra/compare/cassandra-3.11...thelastpickle:cassandra:mck/github-actions/3.11
> >>>> 
> >>>> 
> >>>> 
> >>>>> == 3. Enable pushing backwards build results for both Jenkins and
> >>>>> CircleCI to GitHub pull requests. ==
> >>>>> 
> >>>>> The goal here is to have a "green checkbox" for those GitHub pull
> >>>>> requests that have corresponding Jenkins/CircleCI runs. According to
> >>>>> the following links, it is completely possible to have those.
> >>>>> 
> >>>>> https://github.com/jenkinsci/github-branch-source-plugin
> >>>>> https://circleci.com/docs/enable-checks/
> >>>>> 
> >>>>> Moreover, the GitHub Branch Source Plugin is already enabled for the
> >>>>> Cassandra project [16]. The same seems should work the same way for
> >>>>> CirleCI, but I have faced the infrastructure team comment [17] that
> >>>>> describes admin permissions are required for that, so the question is
> >>>>> still open here. I could dig a bit more once we've agreed on it.
> >>>>> 
> >>>>> The actions points here are:
> >>>>> - enable Jenkins integration for GitHub pull requests;
> >>>>> - enable CircleCI integration for GitHub pull requests;
> >>>> 
> >>>> 
> >>>> Some folk use CircleCI, some use ci-cassandra. The green checkbox idea
> >>>> is great, so long as it's optional. We don't want PRs triggering the
> >>>> runs either (there are other mechanisms for triggering for now).
> >>>> 
> >>>> 
> >>>>> The actions points here are:
> >>>>> 
> >>>>> - initiate a wide survey for user and dev lists, to get to know about
> >>>>> the usages;
> >>>>> - remove those configurations that are not used anymore;
> >>>>> - force migration from Ant to Gradle/Maven;
> >>>> 
> >>>> 
> >>>> Let's leave this out for now. There's too many unknowns here. If
> >>>> there's an IDE configuration that's broken, no one has reported it for
> >>>> ages, and no one is around to fix it, then I say we should raise the
> >>>> discussion to remove it.
> >>>> 
> >>>> The Gradle/Maven migration is a hot one, contribute to that discussion
> >>>> but let's not tangle this work up with it, IMHO.
> >>>> 
> >>>> Totally agree that IDE project files should be as light weight as possible.
> >>>> 
> >>>> 
> >>>>> Summarizing everything proposed above I think it is possible to
> >>>>> simplify adding small contributions easier to the codebase as well as
> >>>>> save a bunch of committer's time.
> >>>>> 
> >>>>> So,
> >>>>> WDYT about the things described above?
> >>>>> Should I create a CEP for this?
> >>>> 
> >>>> 
> >>>> I see no need for a CEP here. An epic and tickets will work.
> >>>> Again, thanks for the input Maxim!
> >> 
> 
> 

Re: [DISCUSSION] Cassandra's code style and source code analysis

[sc...@paradoxica.net]

For me, the strongest arguments in favor of adopting a modern build tool like Maven or Gradle are their ecosystems - both explicit (in terms of plugins), and implicit (in terms of nearly all build tooling supporting both of them, but not ant).

Investment in Ant - and in tooling that integrates with Ant - fell off years ago. This makes integrating build-phase aspects of Cassandra with other tooling a very frustrating task that users of most build tools get for free. Many tools built in the last several years don’t support it, or do so only as an afterthought.

Two recent examples that have caused pain for me, which I suspect are felt by many:

– Integration with internal build systems at many companies that develop Cassandra. Because ant has fallen into disuse, this integration is heavily manual instead of automatic and free. It usually requires forking the project’s build.xml, developing custom tooling around it, or creating a mock Gradle build that wraps ant lifecycle tasks (which also requires overriding ant tasks whose names clash).

– Security toolchain integration. Many users and developers of Cassandra also integrate with security tooling at their respective companies. Because Ant has fallen into disuse, most tooling commonly used by security organizations doesn’t support it. SBOMs are a good example, as their introduction postdates ant’s decline. Maven plugins exist to generate them in CycloneDX and SPDX format, but no such plugins exist for ant. Cassandra users and developers who need them must manually write shims to produce SBOMs that users of modern build tools get for free.

These might not be use cases anticipated by the project, but they represent work I suspect a large number of contributors to the project are required to perform to make Cassandra usable for them.

The ecosystem point means that the fix for this has to be external to the project if the project continues to use Ant: lots of one-off scripts and shims unsuitable for contribution with sole maintainers at their respective companies to provide functionality that users of modern build tools get for free. It also demands continuous, incremental work to adapt to changes in security and build tooling in use at many companies that don’t need to be made for projects using well-supported tools like Maven or Gradle.

– Scott

> On Nov 25, 2022, at 4:56 AM, Benedict <be...@apache.org> wrote:
> 
> I think modules are already fairly well supported - we in effect already have several? FQL, Simulator and others I think.
> 
> I can anyway say with absolute certainty that the main impediment to modularising is not build tooling, it’s the difficulty of the task, the cost to the project of undertaking it, and therefore its relative payoff versus other things that could be undertaken by the folk with the relevant expertise to do it.
> 
> I’ve also been around long enough to see enough efforts to broaden contributions fail to have an impact, that basing costly decisions on this kind of reasoning doesn’t resonate. The main impediments to contributions are the complexity of the codebase (and problem domain) and our limited capacity to respond promptly to high quality contributions. Until we fix those fundamental issues, this kind of rearranging of chairs seems more like a branding exercise to ourselves than to anyone else.
> 
> 
>> On 25 Nov 2022, at 10:22, Miklosovic, Stefan <St...@netapp.com> wrote:
>> 
>> I agree with what you wrote. How I understand it is that migrating to Maven/Gradle makes the project more "attractive" for newcomers. If a project is built on "that old un-cool Ant", it might be a little bit off-putting and questionable if we are "stuck in the past on build systems and not progressing".
>> 
>> So in that sense I agree this is more "marketing" rather than technological question but on the other hand, does not Maven/Gradle allow us to modularize the project better? Maybe we would like to modularize but nobody is up to that because build system makes it impossible or at least quite inconvenient to do so. Do you really think there are not any significant benefits to switch even if it "just works" now?
>> 
>> ________________________________________
>> From: Benedict <be...@apache.org>
>> Sent: Friday, November 25, 2022 11:07
>> To: dev@cassandra.apache.org
>> Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis
>> 
>> NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.
>> 
>> 
>> 
>> 
>> There’s always a handful of people asking for it, but notably few if any of the full time contributors doing the majority of the core development of Cassandra. It strikes me as something very appealing to others, but less so to those wanting to get on with development.
>> 
>> I never really see a good argument articulated for the migration, besides general hand waving that ant is old, and people like newer build systems. Ant is working fine, so there isn’t a strong technical reason to replace it, and there are good organisational reasons not to.
>> 
>> Why do you consider a migration inevitable?
>> 
>> 
>> 
>>> On 25 Nov 2022, at 09:58, Miklosovic, Stefan <St...@netapp.com> wrote:
>>> 
>>> Interesting take on Ant / no-Ant, Benedict. I am very curious how this unfolds. My long-term perception is that changing it to something else is more or less inevitable but if there is a broader consensus to not do that .... well.
>>> 
>>> ________________________________________
>>> From: Benedict <be...@apache.org>
>>> Sent: Friday, November 25, 2022 10:52
>>> To: dev@cassandra.apache.org
>>> Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis
>>> 
>>> NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.
>>> 
>>> 
>>> 
>>> 
>>> I was in a bit of a rush last night. I should say that I’m of course +1 a general endeavour to clean this up, and to expand our use of linters, and I appreciate your volunteering to help out in this way Maxim.
>>> 
>>> However, responding to Stefan, I’m pretty -1 migrating from ant to another build system without really good reason. Migration has a real cost to productivity for all existing contributors, and the phantom of increasing new contributions has never paid off historically. I’m all for easing people into participation, but not at penalty to the existing contributor base.
>>> 
>>> If the only reason is to make it easier to open in a different IDE, we can perhaps have some basic build files outlining code structure for importing, that are compatible with our canonical ant build? We could perhaps even generate them.
>>> 
>>> 
>>>>> On 25 Nov 2022, at 09:35, Miklosovic, Stefan <St...@netapp.com> wrote:
>>>> 
>>>> For the record, I was testing that same combo Claude mentioned and it did not work out of the box but it is definitely possible to set up successfully. I do not remember the details.
>>>> 
>>>> To replay to Maxim, it all seems good to me, roughly, but I humbly think it all boils down to Maven/Gradle refactoring and on top of that we can do all else.
>>>> 
>>>> For example, there is (1) where the solution, besides fixing the tests, is to introduce an Ant task which would check this on build. That being said, how is that going to look like when we change Ant for something else? That stuff suddenly becomes obsolete.
>>>> 
>>>> This case maybe applies to other problems we want to solve as well. I do not want to do something tailored for one build system just to rewrite it all or to spend significant amount of time on that again when we switch the build system.
>>>> 
>>>> For that reason I think changing Ant for something else should be top priority (as I understand that it the hot topic for community for very long time) and then everything else should follow. We should spend time on things mentioned only in case they do not collide with any build system at all.
>>>> 
>>>> (1) https://issues.apache.org/jira/browse/CASSANDRA-17964
>>>> 
>>>> Stefan
>>>> 
>>>> ________________________________________
>>>> From: Claude Warren, Jr via dev <de...@cassandra.apache.org>
>>>> Sent: Friday, November 25, 2022 10:16
>>>> To: dev@cassandra.apache.org
>>>> Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis
>>>> 
>>>> NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.
>>>> 
>>>> 
>>>> 
>>>> +1 for the concept as a whole.  I am certain I could find nits to pick if I looked deeply.
>>>> 
>>>> @mck -- I did have a problem with Cassandra + Eclipse + Java11 (Classpath).  I gave up and am spending time trying to learn IntelliJ.  I also mentioned it in one of the discussion areas.
>>>> 
>>>> Claude
>>>> 
>>>>> On Thu, Nov 24, 2022 at 8:55 PM Mick Semb Wever <mc...@apache.org>> wrote:
>>>> Thank you for a solid write up Maxim. And welcome to Cassandra, it's
>>>> very positive to see you here.
>>>> 
>>>> I whole-heartedly agree with nearly everything you write. Some input
>>>> and questions inline.
>>>> 
>>>> 
>>>>> 
>>>>> As you may know, the infrastructure team has disabled public sign-up
>>>>> to ASF JIRA (the GitHub issues are recommended instead).
>>>>> 
>>>> 
>>>> 
>>>> I suspect (based on chatter in general, but not on dev@ AFAIK) is to
>>>> avoid GH issues and stick with jira. The sign-up hurdle we will
>>>> document on the website: CASSANDRA-18064
>>>> 
>>>> 
>>>> 
>>>>> == 1. Make the checkstyle config a single point of truth for the
>>>>> source code style. ==
>>>>> 
>>>>> The checkstyle is already used and included in the Cassandra project
>>>>> build lifecycle (ant command line, Jenkins, CircleCI). There is no
>>>>> need to maintain code style configurations for different types of IDEs
>>>>> (e.g. IntelliJ inspections configuration) since the checkstyle.xml
>>>>> file can be directly imported to IDE used by a developer. This is fair
>>>>> for Intellij Idea, NetBeans, and Eclipse.
>>>> 
>>>> 
>>>> Big +1
>>>> 
>>>> 
>>>>> So, I propose to focus on the checks themselves and checking pull
>>>>> requests with automation scripts, rather than maintaining these
>>>>> integrations. The benefits here are avoiding all issues with
>>>>> maintaining configurations for different IDEs. Another good advantage
>>>>> of this approach would be the ability to add new checkstyle rules
>>>>> without touching IDE configuration - and such tickets will be LFH and
>>>>> easy to commit.
>>>>> 
>>>>> The actions points here are:
>>>>> 
>>>>> - create an umbrella JIRA ticket for all checkstyle issues e.g. [8]
>>>>> (or label checkstyle);
>>>>> - add checkstyle to GitHub pull requests using GitHub actions (execute
>>>>> ant command);
>>>> 
>>>> 
>>>> Instead of custom GHA scripting, please use our existing
>>>> cassandra-artifact.sh (which should already include all such checks).
>>>> 
>>>> Something like https://github.com/apache/cassandra/compare/cassandra-3.11...thelastpickle:cassandra:mck/github-actions/3.11
>>>> 
>>>> 
>>>> 
>>>>> == 3. Enable pushing backwards build results for both Jenkins and
>>>>> CircleCI to GitHub pull requests. ==
>>>>> 
>>>>> The goal here is to have a "green checkbox" for those GitHub pull
>>>>> requests that have corresponding Jenkins/CircleCI runs. According to
>>>>> the following links, it is completely possible to have those.
>>>>> 
>>>>> https://github.com/jenkinsci/github-branch-source-plugin
>>>>> https://circleci.com/docs/enable-checks/
>>>>> 
>>>>> Moreover, the GitHub Branch Source Plugin is already enabled for the
>>>>> Cassandra project [16]. The same seems should work the same way for
>>>>> CirleCI, but I have faced the infrastructure team comment [17] that
>>>>> describes admin permissions are required for that, so the question is
>>>>> still open here. I could dig a bit more once we've agreed on it.
>>>>> 
>>>>> The actions points here are:
>>>>> - enable Jenkins integration for GitHub pull requests;
>>>>> - enable CircleCI integration for GitHub pull requests;
>>>> 
>>>> 
>>>> Some folk use CircleCI, some use ci-cassandra. The green checkbox idea
>>>> is great, so long as it's optional. We don't want PRs triggering the
>>>> runs either (there are other mechanisms for triggering for now).
>>>> 
>>>> 
>>>>> The actions points here are:
>>>>> 
>>>>> - initiate a wide survey for user and dev lists, to get to know about
>>>>> the usages;
>>>>> - remove those configurations that are not used anymore;
>>>>> - force migration from Ant to Gradle/Maven;
>>>> 
>>>> 
>>>> Let's leave this out for now. There's too many unknowns here. If
>>>> there's an IDE configuration that's broken, no one has reported it for
>>>> ages, and no one is around to fix it, then I say we should raise the
>>>> discussion to remove it.
>>>> 
>>>> The Gradle/Maven migration is a hot one, contribute to that discussion
>>>> but let's not tangle this work up with it, IMHO.
>>>> 
>>>> Totally agree that IDE project files should be as light weight as possible.
>>>> 
>>>> 
>>>>> Summarizing everything proposed above I think it is possible to
>>>>> simplify adding small contributions easier to the codebase as well as
>>>>> save a bunch of committer's time.
>>>>> 
>>>>> So,
>>>>> WDYT about the things described above?
>>>>> Should I create a CEP for this?
>>>> 
>>>> 
>>>> I see no need for a CEP here. An epic and tickets will work.
>>>> Again, thanks for the input Maxim!
>> 

Re: [DISCUSSION] Cassandra's code style and source code analysis

[Benedict <be...@apache.org>]

I think modules are already fairly well supported - we in effect already have several? FQL, Simulator and others I think.

I can anyway say with absolute certainty that the main impediment to modularising is not build tooling, it’s the difficulty of the task, the cost to the project of undertaking it, and therefore its relative payoff versus other things that could be undertaken by the folk with the relevant expertise to do it.

I’ve also been around long enough to see enough efforts to broaden contributions fail to have an impact, that basing costly decisions on this kind of reasoning doesn’t resonate. The main impediments to contributions are the complexity of the codebase (and problem domain) and our limited capacity to respond promptly to high quality contributions. Until we fix those fundamental issues, this kind of rearranging of chairs seems more like a branding exercise to ourselves than to anyone else.


> On 25 Nov 2022, at 10:22, Miklosovic, Stefan <St...@netapp.com> wrote:
> 
> I agree with what you wrote. How I understand it is that migrating to Maven/Gradle makes the project more "attractive" for newcomers. If a project is built on "that old un-cool Ant", it might be a little bit off-putting and questionable if we are "stuck in the past on build systems and not progressing".
> 
> So in that sense I agree this is more "marketing" rather than technological question but on the other hand, does not Maven/Gradle allow us to modularize the project better? Maybe we would like to modularize but nobody is up to that because build system makes it impossible or at least quite inconvenient to do so. Do you really think there are not any significant benefits to switch even if it "just works" now?
> 
> ________________________________________
> From: Benedict <be...@apache.org>
> Sent: Friday, November 25, 2022 11:07
> To: dev@cassandra.apache.org
> Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis
> 
> NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.
> 
> 
> 
> 
> There’s always a handful of people asking for it, but notably few if any of the full time contributors doing the majority of the core development of Cassandra. It strikes me as something very appealing to others, but less so to those wanting to get on with development.
> 
> I never really see a good argument articulated for the migration, besides general hand waving that ant is old, and people like newer build systems. Ant is working fine, so there isn’t a strong technical reason to replace it, and there are good organisational reasons not to.
> 
> Why do you consider a migration inevitable?
> 
> 
> 
>> On 25 Nov 2022, at 09:58, Miklosovic, Stefan <St...@netapp.com> wrote:
>> 
>> Interesting take on Ant / no-Ant, Benedict. I am very curious how this unfolds. My long-term perception is that changing it to something else is more or less inevitable but if there is a broader consensus to not do that .... well.
>> 
>> ________________________________________
>> From: Benedict <be...@apache.org>
>> Sent: Friday, November 25, 2022 10:52
>> To: dev@cassandra.apache.org
>> Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis
>> 
>> NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.
>> 
>> 
>> 
>> 
>> I was in a bit of a rush last night. I should say that I’m of course +1 a general endeavour to clean this up, and to expand our use of linters, and I appreciate your volunteering to help out in this way Maxim.
>> 
>> However, responding to Stefan, I’m pretty -1 migrating from ant to another build system without really good reason. Migration has a real cost to productivity for all existing contributors, and the phantom of increasing new contributions has never paid off historically. I’m all for easing people into participation, but not at penalty to the existing contributor base.
>> 
>> If the only reason is to make it easier to open in a different IDE, we can perhaps have some basic build files outlining code structure for importing, that are compatible with our canonical ant build? We could perhaps even generate them.
>> 
>> 
>>>> On 25 Nov 2022, at 09:35, Miklosovic, Stefan <St...@netapp.com> wrote:
>>> 
>>> For the record, I was testing that same combo Claude mentioned and it did not work out of the box but it is definitely possible to set up successfully. I do not remember the details.
>>> 
>>> To replay to Maxim, it all seems good to me, roughly, but I humbly think it all boils down to Maven/Gradle refactoring and on top of that we can do all else.
>>> 
>>> For example, there is (1) where the solution, besides fixing the tests, is to introduce an Ant task which would check this on build. That being said, how is that going to look like when we change Ant for something else? That stuff suddenly becomes obsolete.
>>> 
>>> This case maybe applies to other problems we want to solve as well. I do not want to do something tailored for one build system just to rewrite it all or to spend significant amount of time on that again when we switch the build system.
>>> 
>>> For that reason I think changing Ant for something else should be top priority (as I understand that it the hot topic for community for very long time) and then everything else should follow. We should spend time on things mentioned only in case they do not collide with any build system at all.
>>> 
>>> (1) https://issues.apache.org/jira/browse/CASSANDRA-17964
>>> 
>>> Stefan
>>> 
>>> ________________________________________
>>> From: Claude Warren, Jr via dev <de...@cassandra.apache.org>
>>> Sent: Friday, November 25, 2022 10:16
>>> To: dev@cassandra.apache.org
>>> Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis
>>> 
>>> NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.
>>> 
>>> 
>>> 
>>> +1 for the concept as a whole.  I am certain I could find nits to pick if I looked deeply.
>>> 
>>> @mck -- I did have a problem with Cassandra + Eclipse + Java11 (Classpath).  I gave up and am spending time trying to learn IntelliJ.  I also mentioned it in one of the discussion areas.
>>> 
>>> Claude
>>> 
>>>> On Thu, Nov 24, 2022 at 8:55 PM Mick Semb Wever <mc...@apache.org>> wrote:
>>> Thank you for a solid write up Maxim. And welcome to Cassandra, it's
>>> very positive to see you here.
>>> 
>>> I whole-heartedly agree with nearly everything you write. Some input
>>> and questions inline.
>>> 
>>> 
>>>> 
>>>> As you may know, the infrastructure team has disabled public sign-up
>>>> to ASF JIRA (the GitHub issues are recommended instead).
>>>> 
>>> 
>>> 
>>> I suspect (based on chatter in general, but not on dev@ AFAIK) is to
>>> avoid GH issues and stick with jira. The sign-up hurdle we will
>>> document on the website: CASSANDRA-18064
>>> 
>>> 
>>> 
>>>> == 1. Make the checkstyle config a single point of truth for the
>>>> source code style. ==
>>>> 
>>>> The checkstyle is already used and included in the Cassandra project
>>>> build lifecycle (ant command line, Jenkins, CircleCI). There is no
>>>> need to maintain code style configurations for different types of IDEs
>>>> (e.g. IntelliJ inspections configuration) since the checkstyle.xml
>>>> file can be directly imported to IDE used by a developer. This is fair
>>>> for Intellij Idea, NetBeans, and Eclipse.
>>> 
>>> 
>>> Big +1
>>> 
>>> 
>>>> So, I propose to focus on the checks themselves and checking pull
>>>> requests with automation scripts, rather than maintaining these
>>>> integrations. The benefits here are avoiding all issues with
>>>> maintaining configurations for different IDEs. Another good advantage
>>>> of this approach would be the ability to add new checkstyle rules
>>>> without touching IDE configuration - and such tickets will be LFH and
>>>> easy to commit.
>>>> 
>>>> The actions points here are:
>>>> 
>>>> - create an umbrella JIRA ticket for all checkstyle issues e.g. [8]
>>>> (or label checkstyle);
>>>> - add checkstyle to GitHub pull requests using GitHub actions (execute
>>>> ant command);
>>> 
>>> 
>>> Instead of custom GHA scripting, please use our existing
>>> cassandra-artifact.sh (which should already include all such checks).
>>> 
>>> Something like https://github.com/apache/cassandra/compare/cassandra-3.11...thelastpickle:cassandra:mck/github-actions/3.11
>>> 
>>> 
>>> 
>>>> == 3. Enable pushing backwards build results for both Jenkins and
>>>> CircleCI to GitHub pull requests. ==
>>>> 
>>>> The goal here is to have a "green checkbox" for those GitHub pull
>>>> requests that have corresponding Jenkins/CircleCI runs. According to
>>>> the following links, it is completely possible to have those.
>>>> 
>>>> https://github.com/jenkinsci/github-branch-source-plugin
>>>> https://circleci.com/docs/enable-checks/
>>>> 
>>>> Moreover, the GitHub Branch Source Plugin is already enabled for the
>>>> Cassandra project [16]. The same seems should work the same way for
>>>> CirleCI, but I have faced the infrastructure team comment [17] that
>>>> describes admin permissions are required for that, so the question is
>>>> still open here. I could dig a bit more once we've agreed on it.
>>>> 
>>>> The actions points here are:
>>>> - enable Jenkins integration for GitHub pull requests;
>>>> - enable CircleCI integration for GitHub pull requests;
>>> 
>>> 
>>> Some folk use CircleCI, some use ci-cassandra. The green checkbox idea
>>> is great, so long as it's optional. We don't want PRs triggering the
>>> runs either (there are other mechanisms for triggering for now).
>>> 
>>> 
>>>> The actions points here are:
>>>> 
>>>> - initiate a wide survey for user and dev lists, to get to know about
>>>> the usages;
>>>> - remove those configurations that are not used anymore;
>>>> - force migration from Ant to Gradle/Maven;
>>> 
>>> 
>>> Let's leave this out for now. There's too many unknowns here. If
>>> there's an IDE configuration that's broken, no one has reported it for
>>> ages, and no one is around to fix it, then I say we should raise the
>>> discussion to remove it.
>>> 
>>> The Gradle/Maven migration is a hot one, contribute to that discussion
>>> but let's not tangle this work up with it, IMHO.
>>> 
>>> Totally agree that IDE project files should be as light weight as possible.
>>> 
>>> 
>>>> Summarizing everything proposed above I think it is possible to
>>>> simplify adding small contributions easier to the codebase as well as
>>>> save a bunch of committer's time.
>>>> 
>>>> So,
>>>> WDYT about the things described above?
>>>> Should I create a CEP for this?
>>> 
>>> 
>>> I see no need for a CEP here. An epic and tickets will work.
>>> Again, thanks for the input Maxim!
> 

Re: [DISCUSSION] Cassandra's code style and source code analysis

["Miklosovic, Stefan" <St...@netapp.com>]

I again agree with Benedict when it comes to the code formatters. It will certainly bring a lot of mess to the code base and for what benefit? I think I am looking into the Cassandra code long enough to see that we are progressively making it look better. It seems to me that over last few years there was the influx of top-notch developers to the project (not saying the previous generation was less "talented", that is not my point at all) and I feel that we are taking great care of how the code looks like while doing reviews and so on. I feel like, lets say, 8 years ago, the development was way more "wild" and "how it looks" was more of an afterthought rather than something inherently considerate during reviews / contribution. One can feel the difference when looking into the newer subsystems almost immediately.

Massively formatting the code would just bring unnecessary big-bang effect for what benefit actually ... Lets just do this naturally. It is something different to fix unused imports or order them the same way, same as introducing spotbugs. But I am big -1 for formatting the code as a whole in one go if that is what we are contemplating about to do as well as I am against to try to change the current code and / or make build-time rules about how the code should look like. I doubt we will ever introduce a reasonable set of rules without changing a lot of already existing code in order to comply with that policy.

When it comes to build systems, what I would like to see is to do some "feasability-study" - how much time it would take, what has to change, how Jenkins would look like ... basically what effort we are talking about there. Maybe it could be as simple as trying to have compillable Maven / Gradle project and nothing else so people could gradually add to that. And maybe once we will flip the switch, who knows.

________________________________________
From: Benedict <be...@apache.org>
Sent: Wednesday, November 30, 2022 10:29
To: dev@cassandra.apache.org
Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis

NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.



Why are we still debating build tooling? I think you’re wrong, but I’ve conceded - on the assumption that we can get enough volunteers willing to adopt responsibility for the new world order.

I suggest five long term contributors nominate themselves as the build file maintainers, and collectively manage a safe and painless migration for the rest of us - and agree to maintain and develop the new build file going forwards, and support the community as they adopt it.

On the topic of over-exuberant linting I will continue to push back. I think linting our brace rules could make sense since they are atypical, but more formatting rules than this likely just leads to atrophying style. Authorship involves thinking about how to present your code; I don’t want to either encourage lazy authorship or prevent experimentation with presentation. Both would be bad, and I expect we would struggle to evolve our style guide again in future as the language evolves. Our brace rules are a good example everyone unilaterally ignored when lambdas arrived, as we all recognised they materially harmed the brevity benefits, and we eventually codified this.

On migration: auto formatters applied to code that was not written with the rules in mind will almost unerringly be made a mess of, so having a tool do this is not acceptable IMO.

The idea of checkstyle being the source of truth continues to be untenable and anyone still pushing for this should please engage with my earlier points on this.


On 30 Nov 2022, at 04:06, Patrick McFadin <pm...@gmail.com> wrote:


I'm going to +1 what Stefan has said. I've heard on many occasions from newcomers to the project that having to use Ant is a deterrent. As a matter of fact, a few weeks ago, I spent a Sunday afternoon helping somebody trying to build Cassandra and Ant caused a ton of problems. "Ok. ant really super clean this time"

Sure it still works for people that have been doing this for years. I drive a 20 year old Toyota truck, but I'm reminded by my kids often that it's not cool. So in that spirit, I feel my saying we need to keep Ant is like saying "You kids get off my lawn!" If it's something that will help attract new contributors, I'm all for it.

Patrick

On Fri, Nov 25, 2022 at 2:22 AM Miklosovic, Stefan <St...@netapp.com>> wrote:
I agree with what you wrote. How I understand it is that migrating to Maven/Gradle makes the project more "attractive" for newcomers. If a project is built on "that old un-cool Ant", it might be a little bit off-putting and questionable if we are "stuck in the past on build systems and not progressing".

So in that sense I agree this is more "marketing" rather than technological question but on the other hand, does not Maven/Gradle allow us to modularize the project better? Maybe we would like to modularize but nobody is up to that because build system makes it impossible or at least quite inconvenient to do so. Do you really think there are not any significant benefits to switch even if it "just works" now?

________________________________________
From: Benedict <be...@apache.org>>
Sent: Friday, November 25, 2022 11:07
To: dev@cassandra.apache.org<ma...@cassandra.apache.org>
Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis

NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.




There’s always a handful of people asking for it, but notably few if any of the full time contributors doing the majority of the core development of Cassandra. It strikes me as something very appealing to others, but less so to those wanting to get on with development.

I never really see a good argument articulated for the migration, besides general hand waving that ant is old, and people like newer build systems. Ant is working fine, so there isn’t a strong technical reason to replace it, and there are good organisational reasons not to.

Why do you consider a migration inevitable?



> On 25 Nov 2022, at 09:58, Miklosovic, Stefan <St...@netapp.com>> wrote:
>
> Interesting take on Ant / no-Ant, Benedict. I am very curious how this unfolds. My long-term perception is that changing it to something else is more or less inevitable but if there is a broader consensus to not do that .... well.
>
> ________________________________________
> From: Benedict <be...@apache.org>>
> Sent: Friday, November 25, 2022 10:52
> To: dev@cassandra.apache.org<ma...@cassandra.apache.org>
> Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis
>
> NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.
>
>
>
>
> I was in a bit of a rush last night. I should say that I’m of course +1 a general endeavour to clean this up, and to expand our use of linters, and I appreciate your volunteering to help out in this way Maxim.
>
> However, responding to Stefan, I’m pretty -1 migrating from ant to another build system without really good reason. Migration has a real cost to productivity for all existing contributors, and the phantom of increasing new contributions has never paid off historically. I’m all for easing people into participation, but not at penalty to the existing contributor base.
>
> If the only reason is to make it easier to open in a different IDE, we can perhaps have some basic build files outlining code structure for importing, that are compatible with our canonical ant build? We could perhaps even generate them.
>
>
>> On 25 Nov 2022, at 09:35, Miklosovic, Stefan <St...@netapp.com>> wrote:
>>
>> For the record, I was testing that same combo Claude mentioned and it did not work out of the box but it is definitely possible to set up successfully. I do not remember the details.
>>
>> To replay to Maxim, it all seems good to me, roughly, but I humbly think it all boils down to Maven/Gradle refactoring and on top of that we can do all else.
>>
>> For example, there is (1) where the solution, besides fixing the tests, is to introduce an Ant task which would check this on build. That being said, how is that going to look like when we change Ant for something else? That stuff suddenly becomes obsolete.
>>
>> This case maybe applies to other problems we want to solve as well. I do not want to do something tailored for one build system just to rewrite it all or to spend significant amount of time on that again when we switch the build system.
>>
>> For that reason I think changing Ant for something else should be top priority (as I understand that it the hot topic for community for very long time) and then everything else should follow. We should spend time on things mentioned only in case they do not collide with any build system at all.
>>
>> (1) https://issues.apache.org/jira/browse/CASSANDRA-17964
>>
>> Stefan
>>
>> ________________________________________
>> From: Claude Warren, Jr via dev <de...@cassandra.apache.org>>
>> Sent: Friday, November 25, 2022 10:16
>> To: dev@cassandra.apache.org<ma...@cassandra.apache.org>
>> Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis
>>
>> NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.
>>
>>
>>
>> +1 for the concept as a whole.  I am certain I could find nits to pick if I looked deeply.
>>
>> @mck -- I did have a problem with Cassandra + Eclipse + Java11 (Classpath).  I gave up and am spending time trying to learn IntelliJ.  I also mentioned it in one of the discussion areas.
>>
>> Claude
>>
>> On Thu, Nov 24, 2022 at 8:55 PM Mick Semb Wever <mc...@apache.org>>> wrote:
>> Thank you for a solid write up Maxim. And welcome to Cassandra, it's
>> very positive to see you here.
>>
>> I whole-heartedly agree with nearly everything you write. Some input
>> and questions inline.
>>
>>
>>>
>>> As you may know, the infrastructure team has disabled public sign-up
>>> to ASF JIRA (the GitHub issues are recommended instead).
>>>
>>
>>
>> I suspect (based on chatter in general, but not on dev@ AFAIK) is to
>> avoid GH issues and stick with jira. The sign-up hurdle we will
>> document on the website: CASSANDRA-18064
>>
>>
>>
>>> == 1. Make the checkstyle config a single point of truth for the
>>> source code style. ==
>>>
>>> The checkstyle is already used and included in the Cassandra project
>>> build lifecycle (ant command line, Jenkins, CircleCI). There is no
>>> need to maintain code style configurations for different types of IDEs
>>> (e.g. IntelliJ inspections configuration) since the checkstyle.xml
>>> file can be directly imported to IDE used by a developer. This is fair
>>> for Intellij Idea, NetBeans, and Eclipse.
>>
>>
>> Big +1
>>
>>
>>> So, I propose to focus on the checks themselves and checking pull
>>> requests with automation scripts, rather than maintaining these
>>> integrations. The benefits here are avoiding all issues with
>>> maintaining configurations for different IDEs. Another good advantage
>>> of this approach would be the ability to add new checkstyle rules
>>> without touching IDE configuration - and such tickets will be LFH and
>>> easy to commit.
>>>
>>> The actions points here are:
>>>
>>> - create an umbrella JIRA ticket for all checkstyle issues e.g. [8]
>>> (or label checkstyle);
>>> - add checkstyle to GitHub pull requests using GitHub actions (execute
>>> ant command);
>>
>>
>> Instead of custom GHA scripting, please use our existing
>> cassandra-artifact.sh (which should already include all such checks).
>>
>> Something like https://github.com/apache/cassandra/compare/cassandra-3.11...thelastpickle:cassandra:mck/github-actions/3.11
>>
>>
>>
>>> == 3. Enable pushing backwards build results for both Jenkins and
>>> CircleCI to GitHub pull requests. ==
>>>
>>> The goal here is to have a "green checkbox" for those GitHub pull
>>> requests that have corresponding Jenkins/CircleCI runs. According to
>>> the following links, it is completely possible to have those.
>>>
>>> https://github.com/jenkinsci/github-branch-source-plugin
>>> https://circleci.com/docs/enable-checks/
>>>
>>> Moreover, the GitHub Branch Source Plugin is already enabled for the
>>> Cassandra project [16]. The same seems should work the same way for
>>> CirleCI, but I have faced the infrastructure team comment [17] that
>>> describes admin permissions are required for that, so the question is
>>> still open here. I could dig a bit more once we've agreed on it.
>>>
>>> The actions points here are:
>>> - enable Jenkins integration for GitHub pull requests;
>>> - enable CircleCI integration for GitHub pull requests;
>>
>>
>> Some folk use CircleCI, some use ci-cassandra. The green checkbox idea
>> is great, so long as it's optional. We don't want PRs triggering the
>> runs either (there are other mechanisms for triggering for now).
>>
>>
>>> The actions points here are:
>>>
>>> - initiate a wide survey for user and dev lists, to get to know about
>>> the usages;
>>> - remove those configurations that are not used anymore;
>>> - force migration from Ant to Gradle/Maven;
>>
>>
>> Let's leave this out for now. There's too many unknowns here. If
>> there's an IDE configuration that's broken, no one has reported it for
>> ages, and no one is around to fix it, then I say we should raise the
>> discussion to remove it.
>>
>> The Gradle/Maven migration is a hot one, contribute to that discussion
>> but let's not tangle this work up with it, IMHO.
>>
>> Totally agree that IDE project files should be as light weight as possible.
>>
>>
>>> Summarizing everything proposed above I think it is possible to
>>> simplify adding small contributions easier to the codebase as well as
>>> save a bunch of committer's time.
>>>
>>> So,
>>> WDYT about the things described above?
>>> Should I create a CEP for this?
>>
>>
>> I see no need for a CEP here. An epic and tickets will work.
>> Again, thanks for the input Maxim!

Re: [DISCUSSION] Cassandra's code style and source code analysis

[Benedict <be...@apache.org>]

I like 3 or 4.

We need to be sure we have a way of deactivating the check with code comments tho, as Java 8 has some bug with import order that can rarely break compilation, so we need to have some mechanism for permitting a different import order.

Did we decide any changes to star imports?


> On 22 Dec 2022, at 14:53, Maxim Muzafarov <mm...@apache.org> wrote:
> 
> Hello everyone, have a great vacation and happy holidays to all!
> 
> 
> I've completed a small research about how the classe's import order
> rule are spread in the Apache projects. Some of the projects don't
> have any restrictions over the imports even if they are using the
> checkstyle configuration. The other ones may have only the consensus
> over the imports, but they are not reflected in the checkstyle yet
> (e.g. Kafka). The conclusion here can only be that there is a very
> large variability in the classe's import order, so we have to agree on
> the order on our own.
> 
> You can find the projects, IDEs and frameworks and their corresponding
> classe's import order below:
> https://mmuzaf.github.io/blog/Java_Import_Orders.html
> 
> 
> Most of the time during development in an IDE the classe's imports
> remains collapsed, so from my point of view the following things
> related to the classe's import comes into the first place to consider:
> 
> - a PR review: newly imports must be clearly visible;
> - try to minimize the total amount of affected files;
> - the import order rule must be implemented in a simple way and well
> supported by IDEs and its plugins;
> 
> In addition to the last mentioned option, the checkstyle itself has
> some limitations also. For instance, the ImportOrder has a limitation
> by design to enforce an empty line between groups ("java", "javax"),
> or CustomImportOrder may have only up to 4 custom groups separated by
> a blank line.
> 
> 
> 
> Based on all of the above I can propose the following classe's order.
> All of them are tested on the latest changes from the trunk branch
> (commit hash: b171b4ba294126e985d0ee629744516f19c8644e)
> 
> 
> 1. Total 2 groups, 3072 files to change
> 
> ```
> all other imports
> [blank line]
> static all other imports
> ```
> 
> 2. Total 3 groups, 2345 files to change
> 
> ```
> java.*
> javax.*
> [blank line]
> all other imports
> [blank line]
> static all other imports
> ```
> 
> 3. Total 5 groups, 2968 files to change
> 
> ```
> org.apache.cassandra.*
> [blank line]
> java.*
> [blank line]
> javax.*
> [blank line]
> all other imports
> [blank line]
> static all other imports
> ```
> 
> 4. Total 5 groups, 1792 files to change
> 
> ```
> java.*
> javax.*
> [blank line]
> com.*
> net.*
> org.*
> [blank line]
> org.apache.cassandra.*
> [blank line]
> all other imports
> [blank line]
> static all other imports
> ```
> 
> 5. Total 2 groups, 3114 files to change
> 
> ```
> java.*
> javax.*
> org.apache.cassandra.*
> all other imports
> [blank line]
> static all other imports
> ```
> 
> 
> Of course, any suggestions are really appreciated.
> Please, share your thoughts.
> 
>> On Thu, 15 Dec 2022 at 17:48, Mick Semb Wever <mc...@apache.org> wrote:
>>>> 
>>> Another angle I forgot to mention is that this is quite a big patch and there are quite big pieces of work coming, being it CEP-15, for example. So I am trying to figure out if we are ok to just merge this work first and devs doing CEP-15 will need to rework their imports or we merge this after them so we will fix their stuff. I do not know what is more preferable.
>> 
>> 
>> 
>> Thank you for bringing this point up Stefan.
>> 
>> I would be actively reaching out to all those engaged with current CEPs, asking them the rebase impact this would cause and if they are ok with it. The CEPs are our priority, and we have a significant amount of them in progress compared to anything we've had for many years.
>> 
>> 
>> 

Re: [DISCUSSION] Cassandra's code style and source code analysis

[David Capwell <dc...@apple.com>]

Im good with 3 and 4.

> On Dec 22, 2022, at 10:41 AM, Derek Chen-Becker <de...@chen-becker.org> wrote:
> 
> I vote for #4. I've always used the convention of having stdlib stuff
> first, external stuff second, and same-project imports last. I guess
> increasing order of specificity?
> 
> Happy Holidays!
> 
> Derek
> 
> On Thu, Dec 22, 2022 at 7:52 AM Maxim Muzafarov <mm...@apache.org> wrote:
>> 
>> Hello everyone, have a great vacation and happy holidays to all!
>> 
>> 
>> I've completed a small research about how the classe's import order
>> rule are spread in the Apache projects. Some of the projects don't
>> have any restrictions over the imports even if they are using the
>> checkstyle configuration. The other ones may have only the consensus
>> over the imports, but they are not reflected in the checkstyle yet
>> (e.g. Kafka). The conclusion here can only be that there is a very
>> large variability in the classe's import order, so we have to agree on
>> the order on our own.
>> 
>> You can find the projects, IDEs and frameworks and their corresponding
>> classe's import order below:
>> https://mmuzaf.github.io/blog/Java_Import_Orders.html
>> 
>> 
>> Most of the time during development in an IDE the classe's imports
>> remains collapsed, so from my point of view the following things
>> related to the classe's import comes into the first place to consider:
>> 
>> - a PR review: newly imports must be clearly visible;
>> - try to minimize the total amount of affected files;
>> - the import order rule must be implemented in a simple way and well
>> supported by IDEs and its plugins;
>> 
>> In addition to the last mentioned option, the checkstyle itself has
>> some limitations also. For instance, the ImportOrder has a limitation
>> by design to enforce an empty line between groups ("java", "javax"),
>> or CustomImportOrder may have only up to 4 custom groups separated by
>> a blank line.
>> 
>> 
>> 
>> Based on all of the above I can propose the following classe's order.
>> All of them are tested on the latest changes from the trunk branch
>> (commit hash: b171b4ba294126e985d0ee629744516f19c8644e)
>> 
>> 
>> 1. Total 2 groups, 3072 files to change
>> 
>> ```
>> all other imports
>> [blank line]
>> static all other imports
>> ```
>> 
>> 2. Total 3 groups, 2345 files to change
>> 
>> ```
>> java.*
>> javax.*
>> [blank line]
>> all other imports
>> [blank line]
>> static all other imports
>> ```
>> 
>> 3. Total 5 groups, 2968 files to change
>> 
>> ```
>> org.apache.cassandra.*
>> [blank line]
>> java.*
>> [blank line]
>> javax.*
>> [blank line]
>> all other imports
>> [blank line]
>> static all other imports
>> ```
>> 
>> 4. Total 5 groups, 1792 files to change
>> 
>> ```
>> java.*
>> javax.*
>> [blank line]
>> com.*
>> net.*
>> org.*
>> [blank line]
>> org.apache.cassandra.*
>> [blank line]
>> all other imports
>> [blank line]
>> static all other imports
>> ```
>> 
>> 5. Total 2 groups, 3114 files to change
>> 
>> ```
>> java.*
>> javax.*
>> org.apache.cassandra.*
>> all other imports
>> [blank line]
>> static all other imports
>> ```
>> 
>> 
>> Of course, any suggestions are really appreciated.
>> Please, share your thoughts.
>> 
>> On Thu, 15 Dec 2022 at 17:48, Mick Semb Wever <mc...@apache.org> wrote:
>>>> 
>>>> Another angle I forgot to mention is that this is quite a big patch and there are quite big pieces of work coming, being it CEP-15, for example. So I am trying to figure out if we are ok to just merge this work first and devs doing CEP-15 will need to rework their imports or we merge this after them so we will fix their stuff. I do not know what is more preferable.
>>> 
>>> 
>>> 
>>> Thank you for bringing this point up Stefan.
>>> 
>>> I would be actively reaching out to all those engaged with current CEPs, asking them the rebase impact this would cause and if they are ok with it. The CEPs are our priority, and we have a significant amount of them in progress compared to anything we've had for many years.
>>> 
>>> 
>>> 
> 
> 
> 
> -- 
> +---------------------------------------------------------------+
> | Derek Chen-Becker                                             |
> | GPG Key available at https://keybase.io/dchenbecker and       |
> | https://pgp.mit.edu/pks/lookup?search=derek%40chen-becker.org |
> | Fngrprnt: EB8A 6480 F0A3 C8EB C1E7  7F42 AFC5 AFEE 96E4 6ACC  |
> +---------------------------------------------------------------+

Re: [DISCUSSION] Cassandra's code style and source code analysis

[Derek Chen-Becker <de...@chen-becker.org>]

I vote for #4. I've always used the convention of having stdlib stuff
first, external stuff second, and same-project imports last. I guess
increasing order of specificity?

Happy Holidays!

Derek

On Thu, Dec 22, 2022 at 7:52 AM Maxim Muzafarov <mm...@apache.org> wrote:
>
> Hello everyone, have a great vacation and happy holidays to all!
>
>
> I've completed a small research about how the classe's import order
> rule are spread in the Apache projects. Some of the projects don't
> have any restrictions over the imports even if they are using the
> checkstyle configuration. The other ones may have only the consensus
> over the imports, but they are not reflected in the checkstyle yet
> (e.g. Kafka). The conclusion here can only be that there is a very
> large variability in the classe's import order, so we have to agree on
> the order on our own.
>
> You can find the projects, IDEs and frameworks and their corresponding
> classe's import order below:
> https://mmuzaf.github.io/blog/Java_Import_Orders.html
>
>
> Most of the time during development in an IDE the classe's imports
> remains collapsed, so from my point of view the following things
> related to the classe's import comes into the first place to consider:
>
> - a PR review: newly imports must be clearly visible;
> - try to minimize the total amount of affected files;
> - the import order rule must be implemented in a simple way and well
> supported by IDEs and its plugins;
>
> In addition to the last mentioned option, the checkstyle itself has
> some limitations also. For instance, the ImportOrder has a limitation
> by design to enforce an empty line between groups ("java", "javax"),
> or CustomImportOrder may have only up to 4 custom groups separated by
> a blank line.
>
>
>
> Based on all of the above I can propose the following classe's order.
> All of them are tested on the latest changes from the trunk branch
> (commit hash: b171b4ba294126e985d0ee629744516f19c8644e)
>
>
> 1. Total 2 groups, 3072 files to change
>
> ```
> all other imports
> [blank line]
> static all other imports
> ```
>
> 2. Total 3 groups, 2345 files to change
>
> ```
> java.*
> javax.*
> [blank line]
> all other imports
> [blank line]
> static all other imports
> ```
>
> 3. Total 5 groups, 2968 files to change
>
> ```
> org.apache.cassandra.*
> [blank line]
> java.*
> [blank line]
> javax.*
> [blank line]
> all other imports
> [blank line]
> static all other imports
> ```
>
> 4. Total 5 groups, 1792 files to change
>
> ```
> java.*
> javax.*
> [blank line]
> com.*
> net.*
> org.*
> [blank line]
> org.apache.cassandra.*
> [blank line]
> all other imports
> [blank line]
> static all other imports
> ```
>
> 5. Total 2 groups, 3114 files to change
>
> ```
> java.*
> javax.*
> org.apache.cassandra.*
> all other imports
> [blank line]
> static all other imports
> ```
>
>
> Of course, any suggestions are really appreciated.
> Please, share your thoughts.
>
> On Thu, 15 Dec 2022 at 17:48, Mick Semb Wever <mc...@apache.org> wrote:
> >>
> >> Another angle I forgot to mention is that this is quite a big patch and there are quite big pieces of work coming, being it CEP-15, for example. So I am trying to figure out if we are ok to just merge this work first and devs doing CEP-15 will need to rework their imports or we merge this after them so we will fix their stuff. I do not know what is more preferable.
> >
> >
> >
> > Thank you for bringing this point up Stefan.
> >
> > I would be actively reaching out to all those engaged with current CEPs, asking them the rebase impact this would cause and if they are ok with it. The CEPs are our priority, and we have a significant amount of them in progress compared to anything we've had for many years.
> >
> >
> >



-- 
+---------------------------------------------------------------+
| Derek Chen-Becker                                             |
| GPG Key available at https://keybase.io/dchenbecker and       |
| https://pgp.mit.edu/pks/lookup?search=derek%40chen-becker.org |
| Fngrprnt: EB8A 6480 F0A3 C8EB C1E7  7F42 AFC5 AFEE 96E4 6ACC  |
+---------------------------------------------------------------+

Re: [DISCUSSION] Cassandra's code style and source code analysis

["Claude Warren, Jr via dev" <de...@cassandra.apache.org>]

Turn it on at warning (or lower) level now, so people have some idea of the
size of change to their current code.

On Wed, Jan 25, 2023 at 12:05 PM Miklosovic, Stefan <
Stefan.Miklosovic@netapp.com> wrote:

> Thank you Maxim for doing this.
>
> It is nice to see this effort materialized in a PR.
>
> I would wait until bigger chunks of work are committed to trunk (like
> CEP-15) to not collide too much. I would say we can postpone doing this
> until the actual 5.0 release, last weeks before it so we would not clash
> with any work people would like to include in 5.0. This can go in anytime,
> basically.
>
> Are people on the same page?
>
> Regards
>
> ________________________________________
> From: Maxim Muzafarov <mm...@apache.org>
> Sent: Monday, January 23, 2023 19:46
> To: dev@cassandra.apache.org
> Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis
>
> NetApp Security WARNING: This is an external email. Do not click links or
> open attachments unless you recognize the sender and know the content is
> safe.
>
>
>
>
> Hello everyone,
>
> You can find the changes here:
> https://issues.apache.org/jira/browse/CASSANDRA-17925
>
> While preparing the code style configuration for the Eclipse IDE, I
> discovered that there was no easy way to have complex grouping options
> for the set of packages. So we need to add extra blank lines between
> each group of packages so that all the configurations for Eclipse,
> NetBeans, IntelliJ IDEA and checkstyle are aligned. I should have
> checked this earlier for sure, but I only did it for static imports
> and some groups, my bad. The resultant configuration looks like this:
>
> java.*
> [blank line]
> javax.*
> [blank line]
> com.*
> [blank line]
> net.*
> [blank line]
> org.*
> [blank line]
> org.apache.cassandra.*
> [blank line]
> all other imports
> [blank line]
> static all other imports
>
> The pull request is here:
> https://github.com/apache/cassandra/pull/2108
>
> The configuration-related changes are placed in a dedicated commit, so
> it should be easy to make a review:
>
> https://github.com/apache/cassandra/pull/2108/commits/84e292ddc9671a0be76ceb9304b2b9a051c2d52a
>
> ________________________________
>
> Another important thing to mention is that the total amount of changes
> for organising imports is really big (more than 2000 files!), so we
> need to decide the right time to merge this PR. Although rebasing or
> merging changes to development branches should become much easier
> ("Accept local" + "Organize imports"), we still need to pay extra
> attention here to minimise the impact on major patches for the next
> release.
>
> On Mon, 16 Jan 2023 at 13:16, Maxim Muzafarov <mm...@apache.org> wrote:
> >
> > Stefan,
> >
> > Thank you for bringing this topic up. I'll prepare the PR shortly with
> > option 4, so everyone can take a look at the amount of changes. This
> > does not force us to go exactly this path, but it may shed light on
> > changes in general.
> >
> > What exactly we're planning to do in the PR:
> >
> > 1. Checkstyle AvoidStarImport rule, so no star imports will be allowed.
> > 2. Checkstyle ImportOrder rule, for controlling the order.
> > 3. The IDE code style configuration for Intellij IDEA, NetBeans, and
> > Eclipse (it doesn't exist for Eclipse yet).
> > 4. The import order according to option 4:
> >
> > ```
> > java.*
> > javax.*
> > [blank line]
> > com.*
> > net.*
> > org.*
> > [blank line]
> > org.apache.cassandra.*
> > [blank line]
> > all other imports
> > [blank line]
> > static all other imports
> > ```
> >
> >
> >
> > On Mon, 16 Jan 2023 at 12:39, Miklosovic, Stefan
> > <St...@netapp.com> wrote:
> > >
> > > Based on the voting we should go with option 4?
> > >
> > > Two weeks passed without anybody joining so I guess folks are all
> happy with that or this just went unnoticed?
> > >
> > > Let's give it time until the end of this week (Friday 12:00 UTC).
> > >
> > > Regards
> > >
> > > ________________________________________
> > > From: Maxim Muzafarov <mm...@apache.org>
> > > Sent: Tuesday, January 3, 2023 14:31
> > > To: dev@cassandra.apache.org
> > > Subject: Re: [DISCUSSION] Cassandra's code style and source code
> analysis
> > >
> > > NetApp Security WARNING: This is an external email. Do not click links
> or open attachments unless you recognize the sender and know the content is
> safe.
> > >
> > >
> > >
> > >
> > > Folks,
> > >
> > > Let me update the voting status and put together everything we have so
> > > far. We definitely need more votes to have a solid foundation for this
> > > change, so I encourage everyone to consider the options above and
> > > share them in this thread.
> > >
> > >
> > > Total for each applicable option:
> > >
> > > 4-th option -- 4 votes
> > > 3-rd option -- 3 votes
> > > 5-th option -- 1 vote
> > > 1-st option -- 0 votes
> > > 2-nd option -- 0 votes
> > >
> > > On Thu, 22 Dec 2022 at 22:06, Mick Semb Wever <mc...@apache.org> wrote:
> > > >>
> > > >>
> > > >> 3. Total 5 groups, 2968 files to change
> > > >>
> > > >> ```
> > > >> org.apache.cassandra.*
> > > >> [blank line]
> > > >> java.*
> > > >> [blank line]
> > > >> javax.*
> > > >> [blank line]
> > > >> all other imports
> > > >> [blank line]
> > > >> static all other imports
> > > >> ```
> > > >
> > > >
> > > >
> > > > 3, then 5.
> > > > There's lots under com.*, net.*, org.* that is essentially the same
> as "all other imports", what's the reason to separate those?
> > > >
> > > > My preference for 3 is simply that imports are by default collapsed,
> and if I expand them it's the dependencies on other cassandra stuff I'm
> first grokking. It's also our only imports that lead to cyclic dependencies
> (which we're not good at).
>

Re: [DISCUSSION] Cassandra's code style and source code analysis

[Maxim Muzafarov <mm...@apache.org>]

Hello everyone,

Some updates.
There are issues that we have put on hold, waiting for the CEPs to be
finalized. The java imports are one of these issues, let's do not
forget them ^^

I've created a label to track it:
https://issues.apache.org/jira/issues/?jql=labels%20%3D%20code-polishing

On Tue, 1 Aug 2023 at 10:46, Miklosovic, Stefan
<St...@netapp.com> wrote:
>
> I think we might wait for Accord and transactional metadata as the last big contributions in 5.0 (if I have not forgotten something) and then we can just polish it all just before the release. There will be still some room to do the housekeeping like this after these patches lend. It is not like Accord will be in trunk on Monday and we release Tuesday ...
>
> ________________________________________
> From: Maxim Muzafarov <mm...@apache.org>
> Sent: Monday, July 31, 2023 23:05
> To: dev@cassandra.apache.org
> Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis
>
> NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.
>
>
>
>
> Hello everyone,
>
>
> It's been a long time since the last discussion about the import order
> code style, so I want to give these changes a chance as all the major
> JIRA issues have already landed on the release branch so we won't
> affect anyone. I'd be happy to find any reviewers who are interested
> in helping with the next steps :-) I've updated the changes to reflect
> the latest checkstyle work, so here they are:
>
> https://issues.apache.org/jira/browse/CASSANDRA-17925
> https://github.com/apache/cassandra/pull/2108
>
>
> The changes look scary at first glance, but they're actually quite
> simple and in line with what we've discussed above. In short, we can
> divide all the affected files into two parts: the update of the code
> style configuration files (checkstyle + IDE configs), and the update
> of all the sources to match the code style.
>
> In short:
>
> - "import order" hotkey will work regardless of which IDE you are using;
> - updated checkstyle configuration, and IDEA, Eclipse, NetBeans
> configurations have been updated;
> - AvoidStarImport checkstyle rule applied as well;
>
> The import order we've agreed upon:
>
> java.*
> [blank line]
> javax.*
> [blank line]
> com.*
> [blank line]
> net.*
> [blank line]
> org.*
> [blank line]
> org.apache.cassandra.*
> [blank line]
> all other imports
> [blank line]
> static all other imports
>
> On Mon, 27 Feb 2023 at 13:26, Maxim Muzafarov <mm...@apache.org> wrote:
> >
> > > I suppose it can be easy for the existing feature branches if they have a single commit. Don't we need to adjust each commit for multi-commit feature branches?
> >
> > It depends on how feature branches are maintained and developed, I
> > guess. My thoughts here are that the IDE's hotkeys should just work to
> > resolve any code-style issues that arise during rebase/maintenance.
> > I'm not talking about enforcing all our code-style rules but giving
> > developers good flexibility. The classes import order rule might be a
> > good example here.
> >
> > On Wed, 22 Feb 2023 at 21:27, Jacek Lewandowski
> > <le...@gmail.com> wrote:
> > >
> > > I suppose it can be easy for the existing feature branches if they have a single commit. Don't we need to adjust each commit for multi-commit feature branches?
> > >
> > > śr., 22 lut 2023, 19:48 użytkownik Maxim Muzafarov <mm...@apache.org> napisał:
> > >>
> > >> Hello everyone,
> > >>
> > >> I have created an issue CASSANDRA-18277 that may help us move forward
> > >> with code style changes. It only affects the way we store the IntelliJ
> > >> code style configuration and has no effect on any current (or any)
> > >> releases, so it should be safe to merge. So, once the issue is
> > >> resolved, every developer that checkouts a release branch will use the
> > >> same code style stored in that branch. This in turn makes rebasing a
> > >> big change like the import order [1] a really straightforward matter
> > >> (by pressing Crtl + Opt + O in their local branch to organize
> > >> imports).
> > >>
> > >> See:
> > >>
> > >> Move the IntelliJ Idea code style and inspections configuration to the
> > >> project's root .idea directory
> > >> https://issues.apache.org/jira/browse/CASSANDRA-18277
> > >>
> > >>
> > >>
> > >> [1] https://issues.apache.org/jira/browse/CASSANDRA-17925
> > >>
> > >> On Wed, 25 Jan 2023 at 13:05, Miklosovic, Stefan
> > >> <St...@netapp.com> wrote:
> > >> >
> > >> > Thank you Maxim for doing this.
> > >> >
> > >> > It is nice to see this effort materialized in a PR.
> > >> >
> > >> > I would wait until bigger chunks of work are committed to trunk (like CEP-15) to not collide too much. I would say we can postpone doing this until the actual 5.0 release, last weeks before it so we would not clash with any work people would like to include in 5.0. This can go in anytime, basically.
> > >> >
> > >> > Are people on the same page?
> > >> >
> > >> > Regards
> > >> >
> > >> > ________________________________________
> > >> > From: Maxim Muzafarov <mm...@apache.org>
> > >> > Sent: Monday, January 23, 2023 19:46
> > >> > To: dev@cassandra.apache.org
> > >> > Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis
> > >> >
> > >> > NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.
> > >> >
> > >> >
> > >> >
> > >> >
> > >> > Hello everyone,
> > >> >
> > >> > You can find the changes here:
> > >> > https://issues.apache.org/jira/browse/CASSANDRA-17925
> > >> >
> > >> > While preparing the code style configuration for the Eclipse IDE, I
> > >> > discovered that there was no easy way to have complex grouping options
> > >> > for the set of packages. So we need to add extra blank lines between
> > >> > each group of packages so that all the configurations for Eclipse,
> > >> > NetBeans, IntelliJ IDEA and checkstyle are aligned. I should have
> > >> > checked this earlier for sure, but I only did it for static imports
> > >> > and some groups, my bad. The resultant configuration looks like this:
> > >> >
> > >> > java.*
> > >> > [blank line]
> > >> > javax.*
> > >> > [blank line]
> > >> > com.*
> > >> > [blank line]
> > >> > net.*
> > >> > [blank line]
> > >> > org.*
> > >> > [blank line]
> > >> > org.apache.cassandra.*
> > >> > [blank line]
> > >> > all other imports
> > >> > [blank line]
> > >> > static all other imports
> > >> >
> > >> > The pull request is here:
> > >> > https://github.com/apache/cassandra/pull/2108
> > >> >
> > >> > The configuration-related changes are placed in a dedicated commit, so
> > >> > it should be easy to make a review:
> > >> > https://github.com/apache/cassandra/pull/2108/commits/84e292ddc9671a0be76ceb9304b2b9a051c2d52a
> > >> >
> > >> > ________________________________
> > >> >
> > >> > Another important thing to mention is that the total amount of changes
> > >> > for organising imports is really big (more than 2000 files!), so we
> > >> > need to decide the right time to merge this PR. Although rebasing or
> > >> > merging changes to development branches should become much easier
> > >> > ("Accept local" + "Organize imports"), we still need to pay extra
> > >> > attention here to minimise the impact on major patches for the next
> > >> > release.
> > >> >
> > >> > On Mon, 16 Jan 2023 at 13:16, Maxim Muzafarov <mm...@apache.org> wrote:
> > >> > >
> > >> > > Stefan,
> > >> > >
> > >> > > Thank you for bringing this topic up. I'll prepare the PR shortly with
> > >> > > option 4, so everyone can take a look at the amount of changes. This
> > >> > > does not force us to go exactly this path, but it may shed light on
> > >> > > changes in general.
> > >> > >
> > >> > > What exactly we're planning to do in the PR:
> > >> > >
> > >> > > 1. Checkstyle AvoidStarImport rule, so no star imports will be allowed.
> > >> > > 2. Checkstyle ImportOrder rule, for controlling the order.
> > >> > > 3. The IDE code style configuration for Intellij IDEA, NetBeans, and
> > >> > > Eclipse (it doesn't exist for Eclipse yet).
> > >> > > 4. The import order according to option 4:
> > >> > >
> > >> > > ```
> > >> > > java.*
> > >> > > javax.*
> > >> > > [blank line]
> > >> > > com.*
> > >> > > net.*
> > >> > > org.*
> > >> > > [blank line]
> > >> > > org.apache.cassandra.*
> > >> > > [blank line]
> > >> > > all other imports
> > >> > > [blank line]
> > >> > > static all other imports
> > >> > > ```
> > >> > >
> > >> > >
> > >> > >
> > >> > > On Mon, 16 Jan 2023 at 12:39, Miklosovic, Stefan
> > >> > > <St...@netapp.com> wrote:
> > >> > > >
> > >> > > > Based on the voting we should go with option 4?
> > >> > > >
> > >> > > > Two weeks passed without anybody joining so I guess folks are all happy with that or this just went unnoticed?
> > >> > > >
> > >> > > > Let's give it time until the end of this week (Friday 12:00 UTC).
> > >> > > >
> > >> > > > Regards
> > >> > > >
> > >> > > > ________________________________________
> > >> > > > From: Maxim Muzafarov <mm...@apache.org>
> > >> > > > Sent: Tuesday, January 3, 2023 14:31
> > >> > > > To: dev@cassandra.apache.org
> > >> > > > Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis
> > >> > > >
> > >> > > > NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > > > Folks,
> > >> > > >
> > >> > > > Let me update the voting status and put together everything we have so
> > >> > > > far. We definitely need more votes to have a solid foundation for this
> > >> > > > change, so I encourage everyone to consider the options above and
> > >> > > > share them in this thread.
> > >> > > >
> > >> > > >
> > >> > > > Total for each applicable option:
> > >> > > >
> > >> > > > 4-th option -- 4 votes
> > >> > > > 3-rd option -- 3 votes
> > >> > > > 5-th option -- 1 vote
> > >> > > > 1-st option -- 0 votes
> > >> > > > 2-nd option -- 0 votes
> > >> > > >
> > >> > > > On Thu, 22 Dec 2022 at 22:06, Mick Semb Wever <mc...@apache.org> wrote:
> > >> > > > >>
> > >> > > > >>
> > >> > > > >> 3. Total 5 groups, 2968 files to change
> > >> > > > >>
> > >> > > > >> ```
> > >> > > > >> org.apache.cassandra.*
> > >> > > > >> [blank line]
> > >> > > > >> java.*
> > >> > > > >> [blank line]
> > >> > > > >> javax.*
> > >> > > > >> [blank line]
> > >> > > > >> all other imports
> > >> > > > >> [blank line]
> > >> > > > >> static all other imports
> > >> > > > >> ```
> > >> > > > >
> > >> > > > >
> > >> > > > >
> > >> > > > > 3, then 5.
> > >> > > > > There's lots under com.*, net.*, org.* that is essentially the same as "all other imports", what's the reason to separate those?
> > >> > > > >
> > >> > > > > My preference for 3 is simply that imports are by default collapsed, and if I expand them it's the dependencies on other cassandra stuff I'm first grokking. It's also our only imports that lead to cyclic dependencies (which we're not good at).

Re: [DISCUSSION] Cassandra's code style and source code analysis

["Miklosovic, Stefan" <St...@netapp.com>]

I think we might wait for Accord and transactional metadata as the last big contributions in 5.0 (if I have not forgotten something) and then we can just polish it all just before the release. There will be still some room to do the housekeeping like this after these patches lend. It is not like Accord will be in trunk on Monday and we release Tuesday ...

________________________________________
From: Maxim Muzafarov <mm...@apache.org>
Sent: Monday, July 31, 2023 23:05
To: dev@cassandra.apache.org
Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis

NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.




Hello everyone,


It's been a long time since the last discussion about the import order
code style, so I want to give these changes a chance as all the major
JIRA issues have already landed on the release branch so we won't
affect anyone. I'd be happy to find any reviewers who are interested
in helping with the next steps :-) I've updated the changes to reflect
the latest checkstyle work, so here they are:

https://issues.apache.org/jira/browse/CASSANDRA-17925
https://github.com/apache/cassandra/pull/2108


The changes look scary at first glance, but they're actually quite
simple and in line with what we've discussed above. In short, we can
divide all the affected files into two parts: the update of the code
style configuration files (checkstyle + IDE configs), and the update
of all the sources to match the code style.

In short:

- "import order" hotkey will work regardless of which IDE you are using;
- updated checkstyle configuration, and IDEA, Eclipse, NetBeans
configurations have been updated;
- AvoidStarImport checkstyle rule applied as well;

The import order we've agreed upon:

java.*
[blank line]
javax.*
[blank line]
com.*
[blank line]
net.*
[blank line]
org.*
[blank line]
org.apache.cassandra.*
[blank line]
all other imports
[blank line]
static all other imports

On Mon, 27 Feb 2023 at 13:26, Maxim Muzafarov <mm...@apache.org> wrote:
>
> > I suppose it can be easy for the existing feature branches if they have a single commit. Don't we need to adjust each commit for multi-commit feature branches?
>
> It depends on how feature branches are maintained and developed, I
> guess. My thoughts here are that the IDE's hotkeys should just work to
> resolve any code-style issues that arise during rebase/maintenance.
> I'm not talking about enforcing all our code-style rules but giving
> developers good flexibility. The classes import order rule might be a
> good example here.
>
> On Wed, 22 Feb 2023 at 21:27, Jacek Lewandowski
> <le...@gmail.com> wrote:
> >
> > I suppose it can be easy for the existing feature branches if they have a single commit. Don't we need to adjust each commit for multi-commit feature branches?
> >
> > śr., 22 lut 2023, 19:48 użytkownik Maxim Muzafarov <mm...@apache.org> napisał:
> >>
> >> Hello everyone,
> >>
> >> I have created an issue CASSANDRA-18277 that may help us move forward
> >> with code style changes. It only affects the way we store the IntelliJ
> >> code style configuration and has no effect on any current (or any)
> >> releases, so it should be safe to merge. So, once the issue is
> >> resolved, every developer that checkouts a release branch will use the
> >> same code style stored in that branch. This in turn makes rebasing a
> >> big change like the import order [1] a really straightforward matter
> >> (by pressing Crtl + Opt + O in their local branch to organize
> >> imports).
> >>
> >> See:
> >>
> >> Move the IntelliJ Idea code style and inspections configuration to the
> >> project's root .idea directory
> >> https://issues.apache.org/jira/browse/CASSANDRA-18277
> >>
> >>
> >>
> >> [1] https://issues.apache.org/jira/browse/CASSANDRA-17925
> >>
> >> On Wed, 25 Jan 2023 at 13:05, Miklosovic, Stefan
> >> <St...@netapp.com> wrote:
> >> >
> >> > Thank you Maxim for doing this.
> >> >
> >> > It is nice to see this effort materialized in a PR.
> >> >
> >> > I would wait until bigger chunks of work are committed to trunk (like CEP-15) to not collide too much. I would say we can postpone doing this until the actual 5.0 release, last weeks before it so we would not clash with any work people would like to include in 5.0. This can go in anytime, basically.
> >> >
> >> > Are people on the same page?
> >> >
> >> > Regards
> >> >
> >> > ________________________________________
> >> > From: Maxim Muzafarov <mm...@apache.org>
> >> > Sent: Monday, January 23, 2023 19:46
> >> > To: dev@cassandra.apache.org
> >> > Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis
> >> >
> >> > NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.
> >> >
> >> >
> >> >
> >> >
> >> > Hello everyone,
> >> >
> >> > You can find the changes here:
> >> > https://issues.apache.org/jira/browse/CASSANDRA-17925
> >> >
> >> > While preparing the code style configuration for the Eclipse IDE, I
> >> > discovered that there was no easy way to have complex grouping options
> >> > for the set of packages. So we need to add extra blank lines between
> >> > each group of packages so that all the configurations for Eclipse,
> >> > NetBeans, IntelliJ IDEA and checkstyle are aligned. I should have
> >> > checked this earlier for sure, but I only did it for static imports
> >> > and some groups, my bad. The resultant configuration looks like this:
> >> >
> >> > java.*
> >> > [blank line]
> >> > javax.*
> >> > [blank line]
> >> > com.*
> >> > [blank line]
> >> > net.*
> >> > [blank line]
> >> > org.*
> >> > [blank line]
> >> > org.apache.cassandra.*
> >> > [blank line]
> >> > all other imports
> >> > [blank line]
> >> > static all other imports
> >> >
> >> > The pull request is here:
> >> > https://github.com/apache/cassandra/pull/2108
> >> >
> >> > The configuration-related changes are placed in a dedicated commit, so
> >> > it should be easy to make a review:
> >> > https://github.com/apache/cassandra/pull/2108/commits/84e292ddc9671a0be76ceb9304b2b9a051c2d52a
> >> >
> >> > ________________________________
> >> >
> >> > Another important thing to mention is that the total amount of changes
> >> > for organising imports is really big (more than 2000 files!), so we
> >> > need to decide the right time to merge this PR. Although rebasing or
> >> > merging changes to development branches should become much easier
> >> > ("Accept local" + "Organize imports"), we still need to pay extra
> >> > attention here to minimise the impact on major patches for the next
> >> > release.
> >> >
> >> > On Mon, 16 Jan 2023 at 13:16, Maxim Muzafarov <mm...@apache.org> wrote:
> >> > >
> >> > > Stefan,
> >> > >
> >> > > Thank you for bringing this topic up. I'll prepare the PR shortly with
> >> > > option 4, so everyone can take a look at the amount of changes. This
> >> > > does not force us to go exactly this path, but it may shed light on
> >> > > changes in general.
> >> > >
> >> > > What exactly we're planning to do in the PR:
> >> > >
> >> > > 1. Checkstyle AvoidStarImport rule, so no star imports will be allowed.
> >> > > 2. Checkstyle ImportOrder rule, for controlling the order.
> >> > > 3. The IDE code style configuration for Intellij IDEA, NetBeans, and
> >> > > Eclipse (it doesn't exist for Eclipse yet).
> >> > > 4. The import order according to option 4:
> >> > >
> >> > > ```
> >> > > java.*
> >> > > javax.*
> >> > > [blank line]
> >> > > com.*
> >> > > net.*
> >> > > org.*
> >> > > [blank line]
> >> > > org.apache.cassandra.*
> >> > > [blank line]
> >> > > all other imports
> >> > > [blank line]
> >> > > static all other imports
> >> > > ```
> >> > >
> >> > >
> >> > >
> >> > > On Mon, 16 Jan 2023 at 12:39, Miklosovic, Stefan
> >> > > <St...@netapp.com> wrote:
> >> > > >
> >> > > > Based on the voting we should go with option 4?
> >> > > >
> >> > > > Two weeks passed without anybody joining so I guess folks are all happy with that or this just went unnoticed?
> >> > > >
> >> > > > Let's give it time until the end of this week (Friday 12:00 UTC).
> >> > > >
> >> > > > Regards
> >> > > >
> >> > > > ________________________________________
> >> > > > From: Maxim Muzafarov <mm...@apache.org>
> >> > > > Sent: Tuesday, January 3, 2023 14:31
> >> > > > To: dev@cassandra.apache.org
> >> > > > Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis
> >> > > >
> >> > > > NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.
> >> > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > > Folks,
> >> > > >
> >> > > > Let me update the voting status and put together everything we have so
> >> > > > far. We definitely need more votes to have a solid foundation for this
> >> > > > change, so I encourage everyone to consider the options above and
> >> > > > share them in this thread.
> >> > > >
> >> > > >
> >> > > > Total for each applicable option:
> >> > > >
> >> > > > 4-th option -- 4 votes
> >> > > > 3-rd option -- 3 votes
> >> > > > 5-th option -- 1 vote
> >> > > > 1-st option -- 0 votes
> >> > > > 2-nd option -- 0 votes
> >> > > >
> >> > > > On Thu, 22 Dec 2022 at 22:06, Mick Semb Wever <mc...@apache.org> wrote:
> >> > > > >>
> >> > > > >>
> >> > > > >> 3. Total 5 groups, 2968 files to change
> >> > > > >>
> >> > > > >> ```
> >> > > > >> org.apache.cassandra.*
> >> > > > >> [blank line]
> >> > > > >> java.*
> >> > > > >> [blank line]
> >> > > > >> javax.*
> >> > > > >> [blank line]
> >> > > > >> all other imports
> >> > > > >> [blank line]
> >> > > > >> static all other imports
> >> > > > >> ```
> >> > > > >
> >> > > > >
> >> > > > >
> >> > > > > 3, then 5.
> >> > > > > There's lots under com.*, net.*, org.* that is essentially the same as "all other imports", what's the reason to separate those?
> >> > > > >
> >> > > > > My preference for 3 is simply that imports are by default collapsed, and if I expand them it's the dependencies on other cassandra stuff I'm first grokking. It's also our only imports that lead to cyclic dependencies (which we're not good at).

Re: [DISCUSSION] Cassandra's code style and source code analysis

[Maxim Muzafarov <mm...@apache.org>]

Hello everyone,


It's been a long time since the last discussion about the import order
code style, so I want to give these changes a chance as all the major
JIRA issues have already landed on the release branch so we won't
affect anyone. I'd be happy to find any reviewers who are interested
in helping with the next steps :-) I've updated the changes to reflect
the latest checkstyle work, so here they are:

https://issues.apache.org/jira/browse/CASSANDRA-17925
https://github.com/apache/cassandra/pull/2108


The changes look scary at first glance, but they're actually quite
simple and in line with what we've discussed above. In short, we can
divide all the affected files into two parts: the update of the code
style configuration files (checkstyle + IDE configs), and the update
of all the sources to match the code style.

In short:

- "import order" hotkey will work regardless of which IDE you are using;
- updated checkstyle configuration, and IDEA, Eclipse, NetBeans
configurations have been updated;
- AvoidStarImport checkstyle rule applied as well;

The import order we've agreed upon:

java.*
[blank line]
javax.*
[blank line]
com.*
[blank line]
net.*
[blank line]
org.*
[blank line]
org.apache.cassandra.*
[blank line]
all other imports
[blank line]
static all other imports

On Mon, 27 Feb 2023 at 13:26, Maxim Muzafarov <mm...@apache.org> wrote:
>
> > I suppose it can be easy for the existing feature branches if they have a single commit. Don't we need to adjust each commit for multi-commit feature branches?
>
> It depends on how feature branches are maintained and developed, I
> guess. My thoughts here are that the IDE's hotkeys should just work to
> resolve any code-style issues that arise during rebase/maintenance.
> I'm not talking about enforcing all our code-style rules but giving
> developers good flexibility. The classes import order rule might be a
> good example here.
>
> On Wed, 22 Feb 2023 at 21:27, Jacek Lewandowski
> <le...@gmail.com> wrote:
> >
> > I suppose it can be easy for the existing feature branches if they have a single commit. Don't we need to adjust each commit for multi-commit feature branches?
> >
> > śr., 22 lut 2023, 19:48 użytkownik Maxim Muzafarov <mm...@apache.org> napisał:
> >>
> >> Hello everyone,
> >>
> >> I have created an issue CASSANDRA-18277 that may help us move forward
> >> with code style changes. It only affects the way we store the IntelliJ
> >> code style configuration and has no effect on any current (or any)
> >> releases, so it should be safe to merge. So, once the issue is
> >> resolved, every developer that checkouts a release branch will use the
> >> same code style stored in that branch. This in turn makes rebasing a
> >> big change like the import order [1] a really straightforward matter
> >> (by pressing Crtl + Opt + O in their local branch to organize
> >> imports).
> >>
> >> See:
> >>
> >> Move the IntelliJ Idea code style and inspections configuration to the
> >> project's root .idea directory
> >> https://issues.apache.org/jira/browse/CASSANDRA-18277
> >>
> >>
> >>
> >> [1] https://issues.apache.org/jira/browse/CASSANDRA-17925
> >>
> >> On Wed, 25 Jan 2023 at 13:05, Miklosovic, Stefan
> >> <St...@netapp.com> wrote:
> >> >
> >> > Thank you Maxim for doing this.
> >> >
> >> > It is nice to see this effort materialized in a PR.
> >> >
> >> > I would wait until bigger chunks of work are committed to trunk (like CEP-15) to not collide too much. I would say we can postpone doing this until the actual 5.0 release, last weeks before it so we would not clash with any work people would like to include in 5.0. This can go in anytime, basically.
> >> >
> >> > Are people on the same page?
> >> >
> >> > Regards
> >> >
> >> > ________________________________________
> >> > From: Maxim Muzafarov <mm...@apache.org>
> >> > Sent: Monday, January 23, 2023 19:46
> >> > To: dev@cassandra.apache.org
> >> > Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis
> >> >
> >> > NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.
> >> >
> >> >
> >> >
> >> >
> >> > Hello everyone,
> >> >
> >> > You can find the changes here:
> >> > https://issues.apache.org/jira/browse/CASSANDRA-17925
> >> >
> >> > While preparing the code style configuration for the Eclipse IDE, I
> >> > discovered that there was no easy way to have complex grouping options
> >> > for the set of packages. So we need to add extra blank lines between
> >> > each group of packages so that all the configurations for Eclipse,
> >> > NetBeans, IntelliJ IDEA and checkstyle are aligned. I should have
> >> > checked this earlier for sure, but I only did it for static imports
> >> > and some groups, my bad. The resultant configuration looks like this:
> >> >
> >> > java.*
> >> > [blank line]
> >> > javax.*
> >> > [blank line]
> >> > com.*
> >> > [blank line]
> >> > net.*
> >> > [blank line]
> >> > org.*
> >> > [blank line]
> >> > org.apache.cassandra.*
> >> > [blank line]
> >> > all other imports
> >> > [blank line]
> >> > static all other imports
> >> >
> >> > The pull request is here:
> >> > https://github.com/apache/cassandra/pull/2108
> >> >
> >> > The configuration-related changes are placed in a dedicated commit, so
> >> > it should be easy to make a review:
> >> > https://github.com/apache/cassandra/pull/2108/commits/84e292ddc9671a0be76ceb9304b2b9a051c2d52a
> >> >
> >> > ________________________________
> >> >
> >> > Another important thing to mention is that the total amount of changes
> >> > for organising imports is really big (more than 2000 files!), so we
> >> > need to decide the right time to merge this PR. Although rebasing or
> >> > merging changes to development branches should become much easier
> >> > ("Accept local" + "Organize imports"), we still need to pay extra
> >> > attention here to minimise the impact on major patches for the next
> >> > release.
> >> >
> >> > On Mon, 16 Jan 2023 at 13:16, Maxim Muzafarov <mm...@apache.org> wrote:
> >> > >
> >> > > Stefan,
> >> > >
> >> > > Thank you for bringing this topic up. I'll prepare the PR shortly with
> >> > > option 4, so everyone can take a look at the amount of changes. This
> >> > > does not force us to go exactly this path, but it may shed light on
> >> > > changes in general.
> >> > >
> >> > > What exactly we're planning to do in the PR:
> >> > >
> >> > > 1. Checkstyle AvoidStarImport rule, so no star imports will be allowed.
> >> > > 2. Checkstyle ImportOrder rule, for controlling the order.
> >> > > 3. The IDE code style configuration for Intellij IDEA, NetBeans, and
> >> > > Eclipse (it doesn't exist for Eclipse yet).
> >> > > 4. The import order according to option 4:
> >> > >
> >> > > ```
> >> > > java.*
> >> > > javax.*
> >> > > [blank line]
> >> > > com.*
> >> > > net.*
> >> > > org.*
> >> > > [blank line]
> >> > > org.apache.cassandra.*
> >> > > [blank line]
> >> > > all other imports
> >> > > [blank line]
> >> > > static all other imports
> >> > > ```
> >> > >
> >> > >
> >> > >
> >> > > On Mon, 16 Jan 2023 at 12:39, Miklosovic, Stefan
> >> > > <St...@netapp.com> wrote:
> >> > > >
> >> > > > Based on the voting we should go with option 4?
> >> > > >
> >> > > > Two weeks passed without anybody joining so I guess folks are all happy with that or this just went unnoticed?
> >> > > >
> >> > > > Let's give it time until the end of this week (Friday 12:00 UTC).
> >> > > >
> >> > > > Regards
> >> > > >
> >> > > > ________________________________________
> >> > > > From: Maxim Muzafarov <mm...@apache.org>
> >> > > > Sent: Tuesday, January 3, 2023 14:31
> >> > > > To: dev@cassandra.apache.org
> >> > > > Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis
> >> > > >
> >> > > > NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.
> >> > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > > Folks,
> >> > > >
> >> > > > Let me update the voting status and put together everything we have so
> >> > > > far. We definitely need more votes to have a solid foundation for this
> >> > > > change, so I encourage everyone to consider the options above and
> >> > > > share them in this thread.
> >> > > >
> >> > > >
> >> > > > Total for each applicable option:
> >> > > >
> >> > > > 4-th option -- 4 votes
> >> > > > 3-rd option -- 3 votes
> >> > > > 5-th option -- 1 vote
> >> > > > 1-st option -- 0 votes
> >> > > > 2-nd option -- 0 votes
> >> > > >
> >> > > > On Thu, 22 Dec 2022 at 22:06, Mick Semb Wever <mc...@apache.org> wrote:
> >> > > > >>
> >> > > > >>
> >> > > > >> 3. Total 5 groups, 2968 files to change
> >> > > > >>
> >> > > > >> ```
> >> > > > >> org.apache.cassandra.*
> >> > > > >> [blank line]
> >> > > > >> java.*
> >> > > > >> [blank line]
> >> > > > >> javax.*
> >> > > > >> [blank line]
> >> > > > >> all other imports
> >> > > > >> [blank line]
> >> > > > >> static all other imports
> >> > > > >> ```
> >> > > > >
> >> > > > >
> >> > > > >
> >> > > > > 3, then 5.
> >> > > > > There's lots under com.*, net.*, org.* that is essentially the same as "all other imports", what's the reason to separate those?
> >> > > > >
> >> > > > > My preference for 3 is simply that imports are by default collapsed, and if I expand them it's the dependencies on other cassandra stuff I'm first grokking. It's also our only imports that lead to cyclic dependencies (which we're not good at).

Re: [DISCUSSION] Cassandra's code style and source code analysis

[Maxim Muzafarov <mm...@apache.org>]

> I suppose it can be easy for the existing feature branches if they have a single commit. Don't we need to adjust each commit for multi-commit feature branches?

It depends on how feature branches are maintained and developed, I
guess. My thoughts here are that the IDE's hotkeys should just work to
resolve any code-style issues that arise during rebase/maintenance.
I'm not talking about enforcing all our code-style rules but giving
developers good flexibility. The classes import order rule might be a
good example here.

On Wed, 22 Feb 2023 at 21:27, Jacek Lewandowski
<le...@gmail.com> wrote:
>
> I suppose it can be easy for the existing feature branches if they have a single commit. Don't we need to adjust each commit for multi-commit feature branches?
>
> śr., 22 lut 2023, 19:48 użytkownik Maxim Muzafarov <mm...@apache.org> napisał:
>>
>> Hello everyone,
>>
>> I have created an issue CASSANDRA-18277 that may help us move forward
>> with code style changes. It only affects the way we store the IntelliJ
>> code style configuration and has no effect on any current (or any)
>> releases, so it should be safe to merge. So, once the issue is
>> resolved, every developer that checkouts a release branch will use the
>> same code style stored in that branch. This in turn makes rebasing a
>> big change like the import order [1] a really straightforward matter
>> (by pressing Crtl + Opt + O in their local branch to organize
>> imports).
>>
>> See:
>>
>> Move the IntelliJ Idea code style and inspections configuration to the
>> project's root .idea directory
>> https://issues.apache.org/jira/browse/CASSANDRA-18277
>>
>>
>>
>> [1] https://issues.apache.org/jira/browse/CASSANDRA-17925
>>
>> On Wed, 25 Jan 2023 at 13:05, Miklosovic, Stefan
>> <St...@netapp.com> wrote:
>> >
>> > Thank you Maxim for doing this.
>> >
>> > It is nice to see this effort materialized in a PR.
>> >
>> > I would wait until bigger chunks of work are committed to trunk (like CEP-15) to not collide too much. I would say we can postpone doing this until the actual 5.0 release, last weeks before it so we would not clash with any work people would like to include in 5.0. This can go in anytime, basically.
>> >
>> > Are people on the same page?
>> >
>> > Regards
>> >
>> > ________________________________________
>> > From: Maxim Muzafarov <mm...@apache.org>
>> > Sent: Monday, January 23, 2023 19:46
>> > To: dev@cassandra.apache.org
>> > Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis
>> >
>> > NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.
>> >
>> >
>> >
>> >
>> > Hello everyone,
>> >
>> > You can find the changes here:
>> > https://issues.apache.org/jira/browse/CASSANDRA-17925
>> >
>> > While preparing the code style configuration for the Eclipse IDE, I
>> > discovered that there was no easy way to have complex grouping options
>> > for the set of packages. So we need to add extra blank lines between
>> > each group of packages so that all the configurations for Eclipse,
>> > NetBeans, IntelliJ IDEA and checkstyle are aligned. I should have
>> > checked this earlier for sure, but I only did it for static imports
>> > and some groups, my bad. The resultant configuration looks like this:
>> >
>> > java.*
>> > [blank line]
>> > javax.*
>> > [blank line]
>> > com.*
>> > [blank line]
>> > net.*
>> > [blank line]
>> > org.*
>> > [blank line]
>> > org.apache.cassandra.*
>> > [blank line]
>> > all other imports
>> > [blank line]
>> > static all other imports
>> >
>> > The pull request is here:
>> > https://github.com/apache/cassandra/pull/2108
>> >
>> > The configuration-related changes are placed in a dedicated commit, so
>> > it should be easy to make a review:
>> > https://github.com/apache/cassandra/pull/2108/commits/84e292ddc9671a0be76ceb9304b2b9a051c2d52a
>> >
>> > ________________________________
>> >
>> > Another important thing to mention is that the total amount of changes
>> > for organising imports is really big (more than 2000 files!), so we
>> > need to decide the right time to merge this PR. Although rebasing or
>> > merging changes to development branches should become much easier
>> > ("Accept local" + "Organize imports"), we still need to pay extra
>> > attention here to minimise the impact on major patches for the next
>> > release.
>> >
>> > On Mon, 16 Jan 2023 at 13:16, Maxim Muzafarov <mm...@apache.org> wrote:
>> > >
>> > > Stefan,
>> > >
>> > > Thank you for bringing this topic up. I'll prepare the PR shortly with
>> > > option 4, so everyone can take a look at the amount of changes. This
>> > > does not force us to go exactly this path, but it may shed light on
>> > > changes in general.
>> > >
>> > > What exactly we're planning to do in the PR:
>> > >
>> > > 1. Checkstyle AvoidStarImport rule, so no star imports will be allowed.
>> > > 2. Checkstyle ImportOrder rule, for controlling the order.
>> > > 3. The IDE code style configuration for Intellij IDEA, NetBeans, and
>> > > Eclipse (it doesn't exist for Eclipse yet).
>> > > 4. The import order according to option 4:
>> > >
>> > > ```
>> > > java.*
>> > > javax.*
>> > > [blank line]
>> > > com.*
>> > > net.*
>> > > org.*
>> > > [blank line]
>> > > org.apache.cassandra.*
>> > > [blank line]
>> > > all other imports
>> > > [blank line]
>> > > static all other imports
>> > > ```
>> > >
>> > >
>> > >
>> > > On Mon, 16 Jan 2023 at 12:39, Miklosovic, Stefan
>> > > <St...@netapp.com> wrote:
>> > > >
>> > > > Based on the voting we should go with option 4?
>> > > >
>> > > > Two weeks passed without anybody joining so I guess folks are all happy with that or this just went unnoticed?
>> > > >
>> > > > Let's give it time until the end of this week (Friday 12:00 UTC).
>> > > >
>> > > > Regards
>> > > >
>> > > > ________________________________________
>> > > > From: Maxim Muzafarov <mm...@apache.org>
>> > > > Sent: Tuesday, January 3, 2023 14:31
>> > > > To: dev@cassandra.apache.org
>> > > > Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis
>> > > >
>> > > > NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.
>> > > >
>> > > >
>> > > >
>> > > >
>> > > > Folks,
>> > > >
>> > > > Let me update the voting status and put together everything we have so
>> > > > far. We definitely need more votes to have a solid foundation for this
>> > > > change, so I encourage everyone to consider the options above and
>> > > > share them in this thread.
>> > > >
>> > > >
>> > > > Total for each applicable option:
>> > > >
>> > > > 4-th option -- 4 votes
>> > > > 3-rd option -- 3 votes
>> > > > 5-th option -- 1 vote
>> > > > 1-st option -- 0 votes
>> > > > 2-nd option -- 0 votes
>> > > >
>> > > > On Thu, 22 Dec 2022 at 22:06, Mick Semb Wever <mc...@apache.org> wrote:
>> > > > >>
>> > > > >>
>> > > > >> 3. Total 5 groups, 2968 files to change
>> > > > >>
>> > > > >> ```
>> > > > >> org.apache.cassandra.*
>> > > > >> [blank line]
>> > > > >> java.*
>> > > > >> [blank line]
>> > > > >> javax.*
>> > > > >> [blank line]
>> > > > >> all other imports
>> > > > >> [blank line]
>> > > > >> static all other imports
>> > > > >> ```
>> > > > >
>> > > > >
>> > > > >
>> > > > > 3, then 5.
>> > > > > There's lots under com.*, net.*, org.* that is essentially the same as "all other imports", what's the reason to separate those?
>> > > > >
>> > > > > My preference for 3 is simply that imports are by default collapsed, and if I expand them it's the dependencies on other cassandra stuff I'm first grokking. It's also our only imports that lead to cyclic dependencies (which we're not good at).

Re: [DISCUSSION] Cassandra's code style and source code analysis

[Jacek Lewandowski <le...@gmail.com>]

I suppose it can be easy for the existing feature branches if they have a
single commit. Don't we need to adjust each commit for multi-commit feature
branches?

śr., 22 lut 2023, 19:48 użytkownik Maxim Muzafarov <mm...@apache.org>
napisał:

> Hello everyone,
>
> I have created an issue CASSANDRA-18277 that may help us move forward
> with code style changes. It only affects the way we store the IntelliJ
> code style configuration and has no effect on any current (or any)
> releases, so it should be safe to merge. So, once the issue is
> resolved, every developer that checkouts a release branch will use the
> same code style stored in that branch. This in turn makes rebasing a
> big change like the import order [1] a really straightforward matter
> (by pressing Crtl + Opt + O in their local branch to organize
> imports).
>
> See:
>
> Move the IntelliJ Idea code style and inspections configuration to the
> project's root .idea directory
> https://issues.apache.org/jira/browse/CASSANDRA-18277
>
>
>
> [1] https://issues.apache.org/jira/browse/CASSANDRA-17925
>
> On Wed, 25 Jan 2023 at 13:05, Miklosovic, Stefan
> <St...@netapp.com> wrote:
> >
> > Thank you Maxim for doing this.
> >
> > It is nice to see this effort materialized in a PR.
> >
> > I would wait until bigger chunks of work are committed to trunk (like
> CEP-15) to not collide too much. I would say we can postpone doing this
> until the actual 5.0 release, last weeks before it so we would not clash
> with any work people would like to include in 5.0. This can go in anytime,
> basically.
> >
> > Are people on the same page?
> >
> > Regards
> >
> > ________________________________________
> > From: Maxim Muzafarov <mm...@apache.org>
> > Sent: Monday, January 23, 2023 19:46
> > To: dev@cassandra.apache.org
> > Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis
> >
> > NetApp Security WARNING: This is an external email. Do not click links
> or open attachments unless you recognize the sender and know the content is
> safe.
> >
> >
> >
> >
> > Hello everyone,
> >
> > You can find the changes here:
> > https://issues.apache.org/jira/browse/CASSANDRA-17925
> >
> > While preparing the code style configuration for the Eclipse IDE, I
> > discovered that there was no easy way to have complex grouping options
> > for the set of packages. So we need to add extra blank lines between
> > each group of packages so that all the configurations for Eclipse,
> > NetBeans, IntelliJ IDEA and checkstyle are aligned. I should have
> > checked this earlier for sure, but I only did it for static imports
> > and some groups, my bad. The resultant configuration looks like this:
> >
> > java.*
> > [blank line]
> > javax.*
> > [blank line]
> > com.*
> > [blank line]
> > net.*
> > [blank line]
> > org.*
> > [blank line]
> > org.apache.cassandra.*
> > [blank line]
> > all other imports
> > [blank line]
> > static all other imports
> >
> > The pull request is here:
> > https://github.com/apache/cassandra/pull/2108
> >
> > The configuration-related changes are placed in a dedicated commit, so
> > it should be easy to make a review:
> >
> https://github.com/apache/cassandra/pull/2108/commits/84e292ddc9671a0be76ceb9304b2b9a051c2d52a
> >
> > ________________________________
> >
> > Another important thing to mention is that the total amount of changes
> > for organising imports is really big (more than 2000 files!), so we
> > need to decide the right time to merge this PR. Although rebasing or
> > merging changes to development branches should become much easier
> > ("Accept local" + "Organize imports"), we still need to pay extra
> > attention here to minimise the impact on major patches for the next
> > release.
> >
> > On Mon, 16 Jan 2023 at 13:16, Maxim Muzafarov <mm...@apache.org> wrote:
> > >
> > > Stefan,
> > >
> > > Thank you for bringing this topic up. I'll prepare the PR shortly with
> > > option 4, so everyone can take a look at the amount of changes. This
> > > does not force us to go exactly this path, but it may shed light on
> > > changes in general.
> > >
> > > What exactly we're planning to do in the PR:
> > >
> > > 1. Checkstyle AvoidStarImport rule, so no star imports will be allowed.
> > > 2. Checkstyle ImportOrder rule, for controlling the order.
> > > 3. The IDE code style configuration for Intellij IDEA, NetBeans, and
> > > Eclipse (it doesn't exist for Eclipse yet).
> > > 4. The import order according to option 4:
> > >
> > > ```
> > > java.*
> > > javax.*
> > > [blank line]
> > > com.*
> > > net.*
> > > org.*
> > > [blank line]
> > > org.apache.cassandra.*
> > > [blank line]
> > > all other imports
> > > [blank line]
> > > static all other imports
> > > ```
> > >
> > >
> > >
> > > On Mon, 16 Jan 2023 at 12:39, Miklosovic, Stefan
> > > <St...@netapp.com> wrote:
> > > >
> > > > Based on the voting we should go with option 4?
> > > >
> > > > Two weeks passed without anybody joining so I guess folks are all
> happy with that or this just went unnoticed?
> > > >
> > > > Let's give it time until the end of this week (Friday 12:00 UTC).
> > > >
> > > > Regards
> > > >
> > > > ________________________________________
> > > > From: Maxim Muzafarov <mm...@apache.org>
> > > > Sent: Tuesday, January 3, 2023 14:31
> > > > To: dev@cassandra.apache.org
> > > > Subject: Re: [DISCUSSION] Cassandra's code style and source code
> analysis
> > > >
> > > > NetApp Security WARNING: This is an external email. Do not click
> links or open attachments unless you recognize the sender and know the
> content is safe.
> > > >
> > > >
> > > >
> > > >
> > > > Folks,
> > > >
> > > > Let me update the voting status and put together everything we have
> so
> > > > far. We definitely need more votes to have a solid foundation for
> this
> > > > change, so I encourage everyone to consider the options above and
> > > > share them in this thread.
> > > >
> > > >
> > > > Total for each applicable option:
> > > >
> > > > 4-th option -- 4 votes
> > > > 3-rd option -- 3 votes
> > > > 5-th option -- 1 vote
> > > > 1-st option -- 0 votes
> > > > 2-nd option -- 0 votes
> > > >
> > > > On Thu, 22 Dec 2022 at 22:06, Mick Semb Wever <mc...@apache.org>
> wrote:
> > > > >>
> > > > >>
> > > > >> 3. Total 5 groups, 2968 files to change
> > > > >>
> > > > >> ```
> > > > >> org.apache.cassandra.*
> > > > >> [blank line]
> > > > >> java.*
> > > > >> [blank line]
> > > > >> javax.*
> > > > >> [blank line]
> > > > >> all other imports
> > > > >> [blank line]
> > > > >> static all other imports
> > > > >> ```
> > > > >
> > > > >
> > > > >
> > > > > 3, then 5.
> > > > > There's lots under com.*, net.*, org.* that is essentially the
> same as "all other imports", what's the reason to separate those?
> > > > >
> > > > > My preference for 3 is simply that imports are by default
> collapsed, and if I expand them it's the dependencies on other cassandra
> stuff I'm first grokking. It's also our only imports that lead to cyclic
> dependencies (which we're not good at).
>

Re: [DISCUSSION] Cassandra's code style and source code analysis

[Maxim Muzafarov <mm...@apache.org>]

Hello everyone,

I have created an issue CASSANDRA-18277 that may help us move forward
with code style changes. It only affects the way we store the IntelliJ
code style configuration and has no effect on any current (or any)
releases, so it should be safe to merge. So, once the issue is
resolved, every developer that checkouts a release branch will use the
same code style stored in that branch. This in turn makes rebasing a
big change like the import order [1] a really straightforward matter
(by pressing Crtl + Opt + O in their local branch to organize
imports).

See:

Move the IntelliJ Idea code style and inspections configuration to the
project's root .idea directory
https://issues.apache.org/jira/browse/CASSANDRA-18277



[1] https://issues.apache.org/jira/browse/CASSANDRA-17925

On Wed, 25 Jan 2023 at 13:05, Miklosovic, Stefan
<St...@netapp.com> wrote:
>
> Thank you Maxim for doing this.
>
> It is nice to see this effort materialized in a PR.
>
> I would wait until bigger chunks of work are committed to trunk (like CEP-15) to not collide too much. I would say we can postpone doing this until the actual 5.0 release, last weeks before it so we would not clash with any work people would like to include in 5.0. This can go in anytime, basically.
>
> Are people on the same page?
>
> Regards
>
> ________________________________________
> From: Maxim Muzafarov <mm...@apache.org>
> Sent: Monday, January 23, 2023 19:46
> To: dev@cassandra.apache.org
> Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis
>
> NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.
>
>
>
>
> Hello everyone,
>
> You can find the changes here:
> https://issues.apache.org/jira/browse/CASSANDRA-17925
>
> While preparing the code style configuration for the Eclipse IDE, I
> discovered that there was no easy way to have complex grouping options
> for the set of packages. So we need to add extra blank lines between
> each group of packages so that all the configurations for Eclipse,
> NetBeans, IntelliJ IDEA and checkstyle are aligned. I should have
> checked this earlier for sure, but I only did it for static imports
> and some groups, my bad. The resultant configuration looks like this:
>
> java.*
> [blank line]
> javax.*
> [blank line]
> com.*
> [blank line]
> net.*
> [blank line]
> org.*
> [blank line]
> org.apache.cassandra.*
> [blank line]
> all other imports
> [blank line]
> static all other imports
>
> The pull request is here:
> https://github.com/apache/cassandra/pull/2108
>
> The configuration-related changes are placed in a dedicated commit, so
> it should be easy to make a review:
> https://github.com/apache/cassandra/pull/2108/commits/84e292ddc9671a0be76ceb9304b2b9a051c2d52a
>
> ________________________________
>
> Another important thing to mention is that the total amount of changes
> for organising imports is really big (more than 2000 files!), so we
> need to decide the right time to merge this PR. Although rebasing or
> merging changes to development branches should become much easier
> ("Accept local" + "Organize imports"), we still need to pay extra
> attention here to minimise the impact on major patches for the next
> release.
>
> On Mon, 16 Jan 2023 at 13:16, Maxim Muzafarov <mm...@apache.org> wrote:
> >
> > Stefan,
> >
> > Thank you for bringing this topic up. I'll prepare the PR shortly with
> > option 4, so everyone can take a look at the amount of changes. This
> > does not force us to go exactly this path, but it may shed light on
> > changes in general.
> >
> > What exactly we're planning to do in the PR:
> >
> > 1. Checkstyle AvoidStarImport rule, so no star imports will be allowed.
> > 2. Checkstyle ImportOrder rule, for controlling the order.
> > 3. The IDE code style configuration for Intellij IDEA, NetBeans, and
> > Eclipse (it doesn't exist for Eclipse yet).
> > 4. The import order according to option 4:
> >
> > ```
> > java.*
> > javax.*
> > [blank line]
> > com.*
> > net.*
> > org.*
> > [blank line]
> > org.apache.cassandra.*
> > [blank line]
> > all other imports
> > [blank line]
> > static all other imports
> > ```
> >
> >
> >
> > On Mon, 16 Jan 2023 at 12:39, Miklosovic, Stefan
> > <St...@netapp.com> wrote:
> > >
> > > Based on the voting we should go with option 4?
> > >
> > > Two weeks passed without anybody joining so I guess folks are all happy with that or this just went unnoticed?
> > >
> > > Let's give it time until the end of this week (Friday 12:00 UTC).
> > >
> > > Regards
> > >
> > > ________________________________________
> > > From: Maxim Muzafarov <mm...@apache.org>
> > > Sent: Tuesday, January 3, 2023 14:31
> > > To: dev@cassandra.apache.org
> > > Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis
> > >
> > > NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.
> > >
> > >
> > >
> > >
> > > Folks,
> > >
> > > Let me update the voting status and put together everything we have so
> > > far. We definitely need more votes to have a solid foundation for this
> > > change, so I encourage everyone to consider the options above and
> > > share them in this thread.
> > >
> > >
> > > Total for each applicable option:
> > >
> > > 4-th option -- 4 votes
> > > 3-rd option -- 3 votes
> > > 5-th option -- 1 vote
> > > 1-st option -- 0 votes
> > > 2-nd option -- 0 votes
> > >
> > > On Thu, 22 Dec 2022 at 22:06, Mick Semb Wever <mc...@apache.org> wrote:
> > > >>
> > > >>
> > > >> 3. Total 5 groups, 2968 files to change
> > > >>
> > > >> ```
> > > >> org.apache.cassandra.*
> > > >> [blank line]
> > > >> java.*
> > > >> [blank line]
> > > >> javax.*
> > > >> [blank line]
> > > >> all other imports
> > > >> [blank line]
> > > >> static all other imports
> > > >> ```
> > > >
> > > >
> > > >
> > > > 3, then 5.
> > > > There's lots under com.*, net.*, org.* that is essentially the same as "all other imports", what's the reason to separate those?
> > > >
> > > > My preference for 3 is simply that imports are by default collapsed, and if I expand them it's the dependencies on other cassandra stuff I'm first grokking. It's also our only imports that lead to cyclic dependencies (which we're not good at).

Re: [DISCUSSION] Cassandra's code style and source code analysis

["Miklosovic, Stefan" <St...@netapp.com>]

Thank you Maxim for doing this.

It is nice to see this effort materialized in a PR.

I would wait until bigger chunks of work are committed to trunk (like CEP-15) to not collide too much. I would say we can postpone doing this until the actual 5.0 release, last weeks before it so we would not clash with any work people would like to include in 5.0. This can go in anytime, basically.

Are people on the same page?

Regards

________________________________________
From: Maxim Muzafarov <mm...@apache.org>
Sent: Monday, January 23, 2023 19:46
To: dev@cassandra.apache.org
Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis

NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.




Hello everyone,

You can find the changes here:
https://issues.apache.org/jira/browse/CASSANDRA-17925

While preparing the code style configuration for the Eclipse IDE, I
discovered that there was no easy way to have complex grouping options
for the set of packages. So we need to add extra blank lines between
each group of packages so that all the configurations for Eclipse,
NetBeans, IntelliJ IDEA and checkstyle are aligned. I should have
checked this earlier for sure, but I only did it for static imports
and some groups, my bad. The resultant configuration looks like this:

java.*
[blank line]
javax.*
[blank line]
com.*
[blank line]
net.*
[blank line]
org.*
[blank line]
org.apache.cassandra.*
[blank line]
all other imports
[blank line]
static all other imports

The pull request is here:
https://github.com/apache/cassandra/pull/2108

The configuration-related changes are placed in a dedicated commit, so
it should be easy to make a review:
https://github.com/apache/cassandra/pull/2108/commits/84e292ddc9671a0be76ceb9304b2b9a051c2d52a

________________________________

Another important thing to mention is that the total amount of changes
for organising imports is really big (more than 2000 files!), so we
need to decide the right time to merge this PR. Although rebasing or
merging changes to development branches should become much easier
("Accept local" + "Organize imports"), we still need to pay extra
attention here to minimise the impact on major patches for the next
release.

On Mon, 16 Jan 2023 at 13:16, Maxim Muzafarov <mm...@apache.org> wrote:
>
> Stefan,
>
> Thank you for bringing this topic up. I'll prepare the PR shortly with
> option 4, so everyone can take a look at the amount of changes. This
> does not force us to go exactly this path, but it may shed light on
> changes in general.
>
> What exactly we're planning to do in the PR:
>
> 1. Checkstyle AvoidStarImport rule, so no star imports will be allowed.
> 2. Checkstyle ImportOrder rule, for controlling the order.
> 3. The IDE code style configuration for Intellij IDEA, NetBeans, and
> Eclipse (it doesn't exist for Eclipse yet).
> 4. The import order according to option 4:
>
> ```
> java.*
> javax.*
> [blank line]
> com.*
> net.*
> org.*
> [blank line]
> org.apache.cassandra.*
> [blank line]
> all other imports
> [blank line]
> static all other imports
> ```
>
>
>
> On Mon, 16 Jan 2023 at 12:39, Miklosovic, Stefan
> <St...@netapp.com> wrote:
> >
> > Based on the voting we should go with option 4?
> >
> > Two weeks passed without anybody joining so I guess folks are all happy with that or this just went unnoticed?
> >
> > Let's give it time until the end of this week (Friday 12:00 UTC).
> >
> > Regards
> >
> > ________________________________________
> > From: Maxim Muzafarov <mm...@apache.org>
> > Sent: Tuesday, January 3, 2023 14:31
> > To: dev@cassandra.apache.org
> > Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis
> >
> > NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.
> >
> >
> >
> >
> > Folks,
> >
> > Let me update the voting status and put together everything we have so
> > far. We definitely need more votes to have a solid foundation for this
> > change, so I encourage everyone to consider the options above and
> > share them in this thread.
> >
> >
> > Total for each applicable option:
> >
> > 4-th option -- 4 votes
> > 3-rd option -- 3 votes
> > 5-th option -- 1 vote
> > 1-st option -- 0 votes
> > 2-nd option -- 0 votes
> >
> > On Thu, 22 Dec 2022 at 22:06, Mick Semb Wever <mc...@apache.org> wrote:
> > >>
> > >>
> > >> 3. Total 5 groups, 2968 files to change
> > >>
> > >> ```
> > >> org.apache.cassandra.*
> > >> [blank line]
> > >> java.*
> > >> [blank line]
> > >> javax.*
> > >> [blank line]
> > >> all other imports
> > >> [blank line]
> > >> static all other imports
> > >> ```
> > >
> > >
> > >
> > > 3, then 5.
> > > There's lots under com.*, net.*, org.* that is essentially the same as "all other imports", what's the reason to separate those?
> > >
> > > My preference for 3 is simply that imports are by default collapsed, and if I expand them it's the dependencies on other cassandra stuff I'm first grokking. It's also our only imports that lead to cyclic dependencies (which we're not good at).

Re: [DISCUSSION] Cassandra's code style and source code analysis

[Maxim Muzafarov <mm...@apache.org>]

Hello everyone,

You can find the changes here:
https://issues.apache.org/jira/browse/CASSANDRA-17925

While preparing the code style configuration for the Eclipse IDE, I
discovered that there was no easy way to have complex grouping options
for the set of packages. So we need to add extra blank lines between
each group of packages so that all the configurations for Eclipse,
NetBeans, IntelliJ IDEA and checkstyle are aligned. I should have
checked this earlier for sure, but I only did it for static imports
and some groups, my bad. The resultant configuration looks like this:

java.*
[blank line]
javax.*
[blank line]
com.*
[blank line]
net.*
[blank line]
org.*
[blank line]
org.apache.cassandra.*
[blank line]
all other imports
[blank line]
static all other imports

The pull request is here:
https://github.com/apache/cassandra/pull/2108

The configuration-related changes are placed in a dedicated commit, so
it should be easy to make a review:
https://github.com/apache/cassandra/pull/2108/commits/84e292ddc9671a0be76ceb9304b2b9a051c2d52a

________________________________

Another important thing to mention is that the total amount of changes
for organising imports is really big (more than 2000 files!), so we
need to decide the right time to merge this PR. Although rebasing or
merging changes to development branches should become much easier
("Accept local" + "Organize imports"), we still need to pay extra
attention here to minimise the impact on major patches for the next
release.

On Mon, 16 Jan 2023 at 13:16, Maxim Muzafarov <mm...@apache.org> wrote:
>
> Stefan,
>
> Thank you for bringing this topic up. I'll prepare the PR shortly with
> option 4, so everyone can take a look at the amount of changes. This
> does not force us to go exactly this path, but it may shed light on
> changes in general.
>
> What exactly we're planning to do in the PR:
>
> 1. Checkstyle AvoidStarImport rule, so no star imports will be allowed.
> 2. Checkstyle ImportOrder rule, for controlling the order.
> 3. The IDE code style configuration for Intellij IDEA, NetBeans, and
> Eclipse (it doesn't exist for Eclipse yet).
> 4. The import order according to option 4:
>
> ```
> java.*
> javax.*
> [blank line]
> com.*
> net.*
> org.*
> [blank line]
> org.apache.cassandra.*
> [blank line]
> all other imports
> [blank line]
> static all other imports
> ```
>
>
>
> On Mon, 16 Jan 2023 at 12:39, Miklosovic, Stefan
> <St...@netapp.com> wrote:
> >
> > Based on the voting we should go with option 4?
> >
> > Two weeks passed without anybody joining so I guess folks are all happy with that or this just went unnoticed?
> >
> > Let's give it time until the end of this week (Friday 12:00 UTC).
> >
> > Regards
> >
> > ________________________________________
> > From: Maxim Muzafarov <mm...@apache.org>
> > Sent: Tuesday, January 3, 2023 14:31
> > To: dev@cassandra.apache.org
> > Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis
> >
> > NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.
> >
> >
> >
> >
> > Folks,
> >
> > Let me update the voting status and put together everything we have so
> > far. We definitely need more votes to have a solid foundation for this
> > change, so I encourage everyone to consider the options above and
> > share them in this thread.
> >
> >
> > Total for each applicable option:
> >
> > 4-th option -- 4 votes
> > 3-rd option -- 3 votes
> > 5-th option -- 1 vote
> > 1-st option -- 0 votes
> > 2-nd option -- 0 votes
> >
> > On Thu, 22 Dec 2022 at 22:06, Mick Semb Wever <mc...@apache.org> wrote:
> > >>
> > >>
> > >> 3. Total 5 groups, 2968 files to change
> > >>
> > >> ```
> > >> org.apache.cassandra.*
> > >> [blank line]
> > >> java.*
> > >> [blank line]
> > >> javax.*
> > >> [blank line]
> > >> all other imports
> > >> [blank line]
> > >> static all other imports
> > >> ```
> > >
> > >
> > >
> > > 3, then 5.
> > > There's lots under com.*, net.*, org.* that is essentially the same as "all other imports", what's the reason to separate those?
> > >
> > > My preference for 3 is simply that imports are by default collapsed, and if I expand them it's the dependencies on other cassandra stuff I'm first grokking. It's also our only imports that lead to cyclic dependencies (which we're not good at).

Re: [DISCUSSION] Cassandra's code style and source code analysis

[Maxim Muzafarov <mm...@apache.org>]

Stefan,

Thank you for bringing this topic up. I'll prepare the PR shortly with
option 4, so everyone can take a look at the amount of changes. This
does not force us to go exactly this path, but it may shed light on
changes in general.

What exactly we're planning to do in the PR:

1. Checkstyle AvoidStarImport rule, so no star imports will be allowed.
2. Checkstyle ImportOrder rule, for controlling the order.
3. The IDE code style configuration for Intellij IDEA, NetBeans, and
Eclipse (it doesn't exist for Eclipse yet).
4. The import order according to option 4:

```
java.*
javax.*
[blank line]
com.*
net.*
org.*
[blank line]
org.apache.cassandra.*
[blank line]
all other imports
[blank line]
static all other imports
```



On Mon, 16 Jan 2023 at 12:39, Miklosovic, Stefan
<St...@netapp.com> wrote:
>
> Based on the voting we should go with option 4?
>
> Two weeks passed without anybody joining so I guess folks are all happy with that or this just went unnoticed?
>
> Let's give it time until the end of this week (Friday 12:00 UTC).
>
> Regards
>
> ________________________________________
> From: Maxim Muzafarov <mm...@apache.org>
> Sent: Tuesday, January 3, 2023 14:31
> To: dev@cassandra.apache.org
> Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis
>
> NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.
>
>
>
>
> Folks,
>
> Let me update the voting status and put together everything we have so
> far. We definitely need more votes to have a solid foundation for this
> change, so I encourage everyone to consider the options above and
> share them in this thread.
>
>
> Total for each applicable option:
>
> 4-th option -- 4 votes
> 3-rd option -- 3 votes
> 5-th option -- 1 vote
> 1-st option -- 0 votes
> 2-nd option -- 0 votes
>
> On Thu, 22 Dec 2022 at 22:06, Mick Semb Wever <mc...@apache.org> wrote:
> >>
> >>
> >> 3. Total 5 groups, 2968 files to change
> >>
> >> ```
> >> org.apache.cassandra.*
> >> [blank line]
> >> java.*
> >> [blank line]
> >> javax.*
> >> [blank line]
> >> all other imports
> >> [blank line]
> >> static all other imports
> >> ```
> >
> >
> >
> > 3, then 5.
> > There's lots under com.*, net.*, org.* that is essentially the same as "all other imports", what's the reason to separate those?
> >
> > My preference for 3 is simply that imports are by default collapsed, and if I expand them it's the dependencies on other cassandra stuff I'm first grokking. It's also our only imports that lead to cyclic dependencies (which we're not good at).

Re: [DISCUSSION] Cassandra's code style and source code analysis

["Miklosovic, Stefan" <St...@netapp.com>]

Based on the voting we should go with option 4?

Two weeks passed without anybody joining so I guess folks are all happy with that or this just went unnoticed?

Let's give it time until the end of this week (Friday 12:00 UTC).

Regards

________________________________________
From: Maxim Muzafarov <mm...@apache.org>
Sent: Tuesday, January 3, 2023 14:31
To: dev@cassandra.apache.org
Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis

NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.




Folks,

Let me update the voting status and put together everything we have so
far. We definitely need more votes to have a solid foundation for this
change, so I encourage everyone to consider the options above and
share them in this thread.


Total for each applicable option:

4-th option -- 4 votes
3-rd option -- 3 votes
5-th option -- 1 vote
1-st option -- 0 votes
2-nd option -- 0 votes

On Thu, 22 Dec 2022 at 22:06, Mick Semb Wever <mc...@apache.org> wrote:
>>
>>
>> 3. Total 5 groups, 2968 files to change
>>
>> ```
>> org.apache.cassandra.*
>> [blank line]
>> java.*
>> [blank line]
>> javax.*
>> [blank line]
>> all other imports
>> [blank line]
>> static all other imports
>> ```
>
>
>
> 3, then 5.
> There's lots under com.*, net.*, org.* that is essentially the same as "all other imports", what's the reason to separate those?
>
> My preference for 3 is simply that imports are by default collapsed, and if I expand them it's the dependencies on other cassandra stuff I'm first grokking. It's also our only imports that lead to cyclic dependencies (which we're not good at).

Re: [DISCUSSION] Cassandra's code style and source code analysis

[Maxim Muzafarov <mm...@apache.org>]

Folks,

Let me update the voting status and put together everything we have so
far. We definitely need more votes to have a solid foundation for this
change, so I encourage everyone to consider the options above and
share them in this thread.


Total for each applicable option:

4-th option -- 4 votes
3-rd option -- 3 votes
5-th option -- 1 vote
1-st option -- 0 votes
2-nd option -- 0 votes

On Thu, 22 Dec 2022 at 22:06, Mick Semb Wever <mc...@apache.org> wrote:
>>
>>
>> 3. Total 5 groups, 2968 files to change
>>
>> ```
>> org.apache.cassandra.*
>> [blank line]
>> java.*
>> [blank line]
>> javax.*
>> [blank line]
>> all other imports
>> [blank line]
>> static all other imports
>> ```
>
>
>
> 3, then 5.
> There's lots under com.*, net.*, org.* that is essentially the same as "all other imports", what's the reason to separate those?
>
> My preference for 3 is simply that imports are by default collapsed, and if I expand them it's the dependencies on other cassandra stuff I'm first grokking. It's also our only imports that lead to cyclic dependencies (which we're not good at).

Re: [DISCUSSION] Cassandra's code style and source code analysis

[Mick Semb Wever <mc...@apache.org>]

>
>
> 3. Total 5 groups, 2968 files to change
>
> ```
> org.apache.cassandra.*
> [blank line]
> java.*
> [blank line]
> javax.*
> [blank line]
> all other imports
> [blank line]
> static all other imports
> ```
>


3, then 5.
There's lots under com.*, net.*, org.* that is essentially the same as "all
other imports", what's the reason to separate those?

My preference for 3 is simply that imports are by default collapsed, and if
I expand them it's the dependencies on other cassandra stuff I'm first
grokking. It's also our only imports that lead to cyclic dependencies
(which we're not good at).

Re: [DISCUSSION] Cassandra's code style and source code analysis

["Miklosovic, Stefan" <St...@netapp.com>]

4

I would like to avoid star imports in general. Having them explicitly enumerated seems to be better, it can not happen we use some import wrongly (by accident). There is rational behind this here (1).

This check can be configured in such a way that star imports could be possible on some packages. For example "org.apache.cassandra.tools.NodeTool" has star import "import org.apache.cassandra.tools.nodetool.*;" where all nodetool commands are. This can be configured so it would remain to be star import on that package so we do not need to explicitly import 100+ classes. There might be more cases like this but they would need to be reviewed more closely.

(1) https://checkstyle.org/config_imports.html#AvoidStarImport

________________________________________
From: Maxim Muzafarov <mm...@apache.org>
Sent: Thursday, December 22, 2022 15:52
To: dev@cassandra.apache.org
Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis

NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.




Hello everyone, have a great vacation and happy holidays to all!


I've completed a small research about how the classe's import order
rule are spread in the Apache projects. Some of the projects don't
have any restrictions over the imports even if they are using the
checkstyle configuration. The other ones may have only the consensus
over the imports, but they are not reflected in the checkstyle yet
(e.g. Kafka). The conclusion here can only be that there is a very
large variability in the classe's import order, so we have to agree on
the order on our own.

You can find the projects, IDEs and frameworks and their corresponding
classe's import order below:
https://mmuzaf.github.io/blog/Java_Import_Orders.html


Most of the time during development in an IDE the classe's imports
remains collapsed, so from my point of view the following things
related to the classe's import comes into the first place to consider:

- a PR review: newly imports must be clearly visible;
- try to minimize the total amount of affected files;
- the import order rule must be implemented in a simple way and well
supported by IDEs and its plugins;

In addition to the last mentioned option, the checkstyle itself has
some limitations also. For instance, the ImportOrder has a limitation
by design to enforce an empty line between groups ("java", "javax"),
or CustomImportOrder may have only up to 4 custom groups separated by
a blank line.



Based on all of the above I can propose the following classe's order.
All of them are tested on the latest changes from the trunk branch
(commit hash: b171b4ba294126e985d0ee629744516f19c8644e)


1. Total 2 groups, 3072 files to change

```
all other imports
[blank line]
static all other imports
```

2. Total 3 groups, 2345 files to change

```
java.*
javax.*
[blank line]
all other imports
[blank line]
static all other imports
```

3. Total 5 groups, 2968 files to change

```
org.apache.cassandra.*
[blank line]
java.*
[blank line]
javax.*
[blank line]
all other imports
[blank line]
static all other imports
```

4. Total 5 groups, 1792 files to change

```
java.*
javax.*
[blank line]
com.*
net.*
org.*
[blank line]
org.apache.cassandra.*
[blank line]
all other imports
[blank line]
static all other imports
```

5. Total 2 groups, 3114 files to change

```
java.*
javax.*
org.apache.cassandra.*
all other imports
[blank line]
static all other imports
```


Of course, any suggestions are really appreciated.
Please, share your thoughts.

On Thu, 15 Dec 2022 at 17:48, Mick Semb Wever <mc...@apache.org> wrote:
>>
>> Another angle I forgot to mention is that this is quite a big patch and there are quite big pieces of work coming, being it CEP-15, for example. So I am trying to figure out if we are ok to just merge this work first and devs doing CEP-15 will need to rework their imports or we merge this after them so we will fix their stuff. I do not know what is more preferable.
>
>
>
> Thank you for bringing this point up Stefan.
>
> I would be actively reaching out to all those engaged with current CEPs, asking them the rebase impact this would cause and if they are ok with it. The CEPs are our priority, and we have a significant amount of them in progress compared to anything we've had for many years.
>
>
>

Re: [DISCUSSION] Cassandra's code style and source code analysis

[Maxim Muzafarov <mm...@apache.org>]

Hello everyone, have a great vacation and happy holidays to all!


I've completed a small research about how the classe's import order
rule are spread in the Apache projects. Some of the projects don't
have any restrictions over the imports even if they are using the
checkstyle configuration. The other ones may have only the consensus
over the imports, but they are not reflected in the checkstyle yet
(e.g. Kafka). The conclusion here can only be that there is a very
large variability in the classe's import order, so we have to agree on
the order on our own.

You can find the projects, IDEs and frameworks and their corresponding
classe's import order below:
https://mmuzaf.github.io/blog/Java_Import_Orders.html


Most of the time during development in an IDE the classe's imports
remains collapsed, so from my point of view the following things
related to the classe's import comes into the first place to consider:

- a PR review: newly imports must be clearly visible;
- try to minimize the total amount of affected files;
- the import order rule must be implemented in a simple way and well
supported by IDEs and its plugins;

In addition to the last mentioned option, the checkstyle itself has
some limitations also. For instance, the ImportOrder has a limitation
by design to enforce an empty line between groups ("java", "javax"),
or CustomImportOrder may have only up to 4 custom groups separated by
a blank line.



Based on all of the above I can propose the following classe's order.
All of them are tested on the latest changes from the trunk branch
(commit hash: b171b4ba294126e985d0ee629744516f19c8644e)


1. Total 2 groups, 3072 files to change

```
all other imports
[blank line]
static all other imports
```

2. Total 3 groups, 2345 files to change

```
java.*
javax.*
[blank line]
all other imports
[blank line]
static all other imports
```

3. Total 5 groups, 2968 files to change

```
org.apache.cassandra.*
[blank line]
java.*
[blank line]
javax.*
[blank line]
all other imports
[blank line]
static all other imports
```

4. Total 5 groups, 1792 files to change

```
java.*
javax.*
[blank line]
com.*
net.*
org.*
[blank line]
org.apache.cassandra.*
[blank line]
all other imports
[blank line]
static all other imports
```

5. Total 2 groups, 3114 files to change

```
java.*
javax.*
org.apache.cassandra.*
all other imports
[blank line]
static all other imports
```


Of course, any suggestions are really appreciated.
Please, share your thoughts.

On Thu, 15 Dec 2022 at 17:48, Mick Semb Wever <mc...@apache.org> wrote:
>>
>> Another angle I forgot to mention is that this is quite a big patch and there are quite big pieces of work coming, being it CEP-15, for example. So I am trying to figure out if we are ok to just merge this work first and devs doing CEP-15 will need to rework their imports or we merge this after them so we will fix their stuff. I do not know what is more preferable.
>
>
>
> Thank you for bringing this point up Stefan.
>
> I would be actively reaching out to all those engaged with current CEPs, asking them the rebase impact this would cause and if they are ok with it. The CEPs are our priority, and we have a significant amount of them in progress compared to anything we've had for many years.
>
>
>

Re: [DISCUSSION] Cassandra's code style and source code analysis

[Mick Semb Wever <mc...@apache.org>]

>
> Another angle I forgot to mention is that this is quite a big patch and
> there are quite big pieces of work coming, being it CEP-15, for example. So
> I am trying to figure out if we are ok to just merge this work first and
> devs doing CEP-15 will need to rework their imports or we merge this after
> them so we will fix their stuff. I do not know what is more preferable.
>


Thank you for bringing this point up Stefan.

I would be actively reaching out to all those engaged with current CEPs,
asking them the rebase impact this would cause and if they are ok with it.
The CEPs are our priority, and we have a significant amount of them in
progress compared to anything we've had for many years.

Re: [DISCUSSION] Cassandra's code style and source code analysis

["Miklosovic, Stefan" <St...@netapp.com>]

Hi Maxim,

yes I will help you to review that.

The approach you outlined sounds reasonable to me by I do not think we can merge this based on my opinion only. I would welcome more developers to discuss this.

Another angle I forgot to mention is that this is quite a big patch and there are quite big pieces of work coming, being it CEP-15, for example. So I am trying to figure out if we are ok to just merge this work first and devs doing CEP-15 will need to rework their imports or we merge this after them so we will fix their stuff. I do not know what is more preferable.

I do not think that the order of imports as we having right now is the same we want to go with. Check Benedict's comment here (1). I agree that the current ordering is strange. Like, why is there this "org.cliffc.high_scale_lib" import in particular? That seems to be completely arbitrary here (or it has some intrinsic logic but I do not see it nor that decision is nowhere to find documented).

We need to gather more feedback here. I ll try to take a look at other Apache projects how their import order is.

Regards

Regards

(1) https://issues.apache.org/jira/browse/CASSANDRA-17925

________________________________________
From: Maxim Muzafarov <mm...@apache.org>
Sent: Monday, December 12, 2022 17:13
To: dev@cassandra.apache.org
Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis

NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.




> Technically it can be two commits which would be merged / pushed at once.

I'll prepare a new pull request containing both of the changes. My
previous experience says me that it's really hard to find a reviewer
who will be able to go through huge pull requests, that's why
initially I've split this into AvoidStarImport and CustomImportOrder
rules. So, if you'll help with the review I'm happy to proceed the way
you suggested :-)

> One thing which needs extra care for ordering imports is that if you order it in IDEA by right-clicking on a package and choosing organising imports, it will remove special comments

You're right, but this is quite unusual behaviour for me and this
seems to be a bug, that hasn't been fixed for a long time [1]. I've
tested the same thing for Eclipse and NetBeans and `optimize imports`
working there as we expect (no comments removes), so the issue exists
only for the IntelliJ IDEA [1].
Despite all of that, we are still on the safe side here - if these
comments will be removed by the `optimized import` procedure the build
with checkstyle will fail.

> I think this is a great time to revisit this ordering.

I would say that the imports order is pretty good (probably, except
for the blank lines) and the imports order is not as important as it
is important that it be the same in all files and automation `optimize
imports`.
I suggest going through a "minimum change" strategy here. The IntelliJ
IDEA has the following configuration with the imports order that most
of the classes already fit:

import java
import javax
[blank line]
import com.google.common
import org.apache.log4j
import org.apache.commons
import org.cliffc.high_scale_lib
import org.junit
import org.slf4j
[blank line]
import all other imports
[blank line]
import static all other imports

We can update the documentation page [2] with this order and implement
the same for NetBeans and Eclipse IDE configuration files as well as
for the checkstyle config.


If everyone is OK with the plan above I'll prepare everything for it.

Suggested summary:
- use current IntelliJ IDEA imports order as defaults for other IDEs;
- update the documentation page;
- prepare a single pull request with AvoidStarImport and CustomImportOrder;



[1] https://youtrack.jetbrains.com/issue/IDEA-128133/Optimize-Imports-disregards-line-comments
[2] https://cassandra.apache.org/_/development/code_style.html

On Sun, 11 Dec 2022 at 00:03, Miklosovic, Stefan
<St...@netapp.com> wrote:
>
> Should the source code obey the AvoidStarImport rule?
>
> yes
>
>  Should we implement AvoidStarImport and CustomImportOrder in a
> single pull request or do it one by one?
>
> Technically it can be two commits which would be merged / pushed at once.
>
> One thing which needs extra care for ordering imports is that if you order it in IDEA by right-clicking on a package and choosing organising imports, it will remove special comments which are put at the end of the import statement. We need to be sure you put them back.  Look at changes in CASSANDRA-17055. We need to preserve this.
>
> Also, we need to be sure that the importing style can be (roughly) set in each major IDE. (eclipse / netbeans / idea) so if we require some import style it can be set in IDE like that. I do not know if we have any strong preference when it comes to this but it definitely does not hurt.
>
> Also, I see that the current import style is
>
> java
> [blank line]
> com.google.common
> org.apache.commons
> org.junit
> org.slf4j
> [blank line]
> everything else alphabetically
>
> I think this is a great time to revisit this ordering. I am not particularly persuaded on this order and why it was choosen. Where has that decision come from?
>
> ________________________________________
> From: Maxim Muzafarov <mm...@apache.org>
> Sent: Wednesday, December 7, 2022 18:29
> To: dev@cassandra.apache.org
> Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis
>
> NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.
>
>
>
>
> Dear community,
>
>
> I have created the epic with code-style activities to track the progress:
> https://issues.apache.org/jira/browse/CASSANDRA-18090
>
> In my understanding, there is no need to format whole the code base at
> once according to the code style described on the page [1], and the
> best strategy here is to go forward with small evolutionary changes.
> Thus eventually we will come up with a set of rules convenient for all
> members of the community. In my mind, having one commit per an added
> code style rule should be easy to look at for a reviewer, the git
> commits history as well as rebasing/merging other pull requests that
> may be affected by the new rules.
>
>
> I want to raise one more question related to class imports and the
> classses import order for a wider discussion. The import order is well
> described on the code style page [1], but using wildcard imports is
> not mentioned at all. The wildcard imports with their drawbacks has
> has already been raised in the JIRA issue [2] and didn't get enough
> attention.
>
> The checkstyle has the rules we are interested in for import control
> and they must be considered together. We can implement them in a
> single pull request or one by one, or use only the last one:
> - AvoidStarImport
> - CustomImportOrder
>
> But still, I think that wildcard imports have more disadvantages
> (class names conflicts e.g. java.util.*, java.sql.* or a new version
> of a library has name clashes) than advantages and such problems will
> be found in later CI cycles.
> Currently, I've implemented the AvoidStarImport checkstyle rule in a
> dedicated pull request [3][4], so you will be able to see all amount
> of the changes with removing wildcard imports. The changes are made
> for the checkstyle configuration as well as for code style
> configurations for different IDEs we supported.
>
> So, the open questions here are:
>
> - Should the source code obey the AvoidStarImport rule [3]? (I think yes);
> - Should we implement AvoidStarImport and CustomImportOrder in a
> single pull request or do it one by one?
>
>
> Anyway, I will fix the result of the agreement over the
> AvoidStarImport rule on the documentation page [1].
>
>
>
> [1] https://cassandra.apache.org/_/development/code_style.html
> [2] https://issues.apache.org/jira/browse/CASSANDRA-17925
> [3] https://issues.apache.org/jira/browse/CASSANDRA-18089
> [4] https://github.com/apache/cassandra/pull/2041
>
> On Thu, 1 Dec 2022 at 11:55, Claude Warren, Jr via dev
> <de...@cassandra.apache.org> wrote:
> >
> > The last time I worked on a project that tried to implement a coding style across the project it was "an education".  The short story is that trying to "mitigate" the code base, with respect to style, is either a massive change or a long slow process.
> >
> > Arguments here have stated that earlier attempts to have the tooling reformat the code did not go well.  What we ended up doing was turned on the style checker and looked at the number of issues across the project.  When new code was accepted the number of issues could not rise.  Eventually most of the code was clean, with a few well coded legacy bits still not up to standard.  We could do something similar here.  Much like code coverage, you can't perform a merge unless the number of style errors remains the same or decreases.
> >
> > As with all software rules, this is a strong recommendation as I am certain that there are edge/corner case exceptions to be found.
> >
> >
> >
> >
> > On Wed, Nov 30, 2022 at 3:30 PM Patrick McFadin <pm...@gmail.com> wrote:
> >>
> >> Why are we still debating build tooling? I think you’re wrong, but I’ve conceded - on the assumption that we can get enough volunteers willing to adopt responsibility for the new world order.
> >>
> >> Not debating. I am just throwing in my support since I have been in the Camp of Ant.
> >>
> >> On Wed, Nov 30, 2022 at 1:29 AM Benedict <be...@apache.org> wrote:
> >>>
> >>> Why are we still debating build tooling? I think you’re wrong, but I’ve conceded - on the assumption that we can get enough volunteers willing to adopt responsibility for the new world order.
> >>>
> >>> I suggest five long term contributors nominate themselves as the build file maintainers, and collectively manage a safe and painless migration for the rest of us - and agree to maintain and develop the new build file going forwards, and support the community as they adopt it.
> >>>
> >>> On the topic of over-exuberant linting I will continue to push back. I think linting our brace rules could make sense since they are atypical, but more formatting rules than this likely just leads to atrophying style. Authorship involves thinking about how to present your code; I don’t want to either encourage lazy authorship or prevent experimentation with presentation. Both would be bad, and I expect we would struggle to evolve our style guide again in future as the language evolves. Our brace rules are a good example everyone unilaterally ignored when lambdas arrived, as we all recognised they materially harmed the brevity benefits, and we eventually codified this.
> >>>
> >>> On migration: auto formatters applied to code that was not written with the rules in mind will almost unerringly be made a mess of, so having a tool do this is not acceptable IMO.
> >>>
> >>> The idea of checkstyle being the source of truth continues to be untenable and anyone still pushing for this should please engage with my earlier points on this.
> >>>
> >>>
> >>> On 30 Nov 2022, at 04:06, Patrick McFadin <pm...@gmail.com> wrote:
> >>>
> >>> 
> >>> I'm going to +1 what Stefan has said. I've heard on many occasions from newcomers to the project that having to use Ant is a deterrent. As a matter of fact, a few weeks ago, I spent a Sunday afternoon helping somebody trying to build Cassandra and Ant caused a ton of problems. "Ok. ant really super clean this time"
> >>>
> >>> Sure it still works for people that have been doing this for years. I drive a 20 year old Toyota truck, but I'm reminded by my kids often that it's not cool. So in that spirit, I feel my saying we need to keep Ant is like saying "You kids get off my lawn!" If it's something that will help attract new contributors, I'm all for it.
> >>>
> >>> Patrick
> >>>
> >>> On Fri, Nov 25, 2022 at 2:22 AM Miklosovic, Stefan <St...@netapp.com> wrote:
> >>>>
> >>>> I agree with what you wrote. How I understand it is that migrating to Maven/Gradle makes the project more "attractive" for newcomers. If a project is built on "that old un-cool Ant", it might be a little bit off-putting and questionable if we are "stuck in the past on build systems and not progressing".
> >>>>
> >>>> So in that sense I agree this is more "marketing" rather than technological question but on the other hand, does not Maven/Gradle allow us to modularize the project better? Maybe we would like to modularize but nobody is up to that because build system makes it impossible or at least quite inconvenient to do so. Do you really think there are not any significant benefits to switch even if it "just works" now?
> >>>>
> >>>> ________________________________________
> >>>> From: Benedict <be...@apache.org>
> >>>> Sent: Friday, November 25, 2022 11:07
> >>>> To: dev@cassandra.apache.org
> >>>> Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis
> >>>>
> >>>> NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> There’s always a handful of people asking for it, but notably few if any of the full time contributors doing the majority of the core development of Cassandra. It strikes me as something very appealing to others, but less so to those wanting to get on with development.
> >>>>
> >>>> I never really see a good argument articulated for the migration, besides general hand waving that ant is old, and people like newer build systems. Ant is working fine, so there isn’t a strong technical reason to replace it, and there are good organisational reasons not to.
> >>>>
> >>>> Why do you consider a migration inevitable?
> >>>>
> >>>>
> >>>>
> >>>> > On 25 Nov 2022, at 09:58, Miklosovic, Stefan <St...@netapp.com> wrote:
> >>>> >
> >>>> > Interesting take on Ant / no-Ant, Benedict. I am very curious how this unfolds. My long-term perception is that changing it to something else is more or less inevitable but if there is a broader consensus to not do that .... well.
> >>>> >
> >>>> > ________________________________________
> >>>> > From: Benedict <be...@apache.org>
> >>>> > Sent: Friday, November 25, 2022 10:52
> >>>> > To: dev@cassandra.apache.org
> >>>> > Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis
> >>>> >
> >>>> > NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.
> >>>> >
> >>>> >
> >>>> >
> >>>> >
> >>>> > I was in a bit of a rush last night. I should say that I’m of course +1 a general endeavour to clean this up, and to expand our use of linters, and I appreciate your volunteering to help out in this way Maxim.
> >>>> >
> >>>> > However, responding to Stefan, I’m pretty -1 migrating from ant to another build system without really good reason. Migration has a real cost to productivity for all existing contributors, and the phantom of increasing new contributions has never paid off historically. I’m all for easing people into participation, but not at penalty to the existing contributor base.
> >>>> >
> >>>> > If the only reason is to make it easier to open in a different IDE, we can perhaps have some basic build files outlining code structure for importing, that are compatible with our canonical ant build? We could perhaps even generate them.
> >>>> >
> >>>> >
> >>>> >> On 25 Nov 2022, at 09:35, Miklosovic, Stefan <St...@netapp.com> wrote:
> >>>> >>
> >>>> >> For the record, I was testing that same combo Claude mentioned and it did not work out of the box but it is definitely possible to set up successfully. I do not remember the details.
> >>>> >>
> >>>> >> To replay to Maxim, it all seems good to me, roughly, but I humbly think it all boils down to Maven/Gradle refactoring and on top of that we can do all else.
> >>>> >>
> >>>> >> For example, there is (1) where the solution, besides fixing the tests, is to introduce an Ant task which would check this on build. That being said, how is that going to look like when we change Ant for something else? That stuff suddenly becomes obsolete.
> >>>> >>
> >>>> >> This case maybe applies to other problems we want to solve as well. I do not want to do something tailored for one build system just to rewrite it all or to spend significant amount of time on that again when we switch the build system.
> >>>> >>
> >>>> >> For that reason I think changing Ant for something else should be top priority (as I understand that it the hot topic for community for very long time) and then everything else should follow. We should spend time on things mentioned only in case they do not collide with any build system at all.
> >>>> >>
> >>>> >> (1) https://issues.apache.org/jira/browse/CASSANDRA-17964
> >>>> >>
> >>>> >> Stefan
> >>>> >>
> >>>> >> ________________________________________
> >>>> >> From: Claude Warren, Jr via dev <de...@cassandra.apache.org>
> >>>> >> Sent: Friday, November 25, 2022 10:16
> >>>> >> To: dev@cassandra.apache.org
> >>>> >> Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis
> >>>> >>
> >>>> >> NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.
> >>>> >>
> >>>> >>
> >>>> >>
> >>>> >> +1 for the concept as a whole.  I am certain I could find nits to pick if I looked deeply.
> >>>> >>
> >>>> >> @mck -- I did have a problem with Cassandra + Eclipse + Java11 (Classpath).  I gave up and am spending time trying to learn IntelliJ.  I also mentioned it in one of the discussion areas.
> >>>> >>
> >>>> >> Claude
> >>>> >>
> >>>> >> On Thu, Nov 24, 2022 at 8:55 PM Mick Semb Wever <mc...@apache.org>> wrote:
> >>>> >> Thank you for a solid write up Maxim. And welcome to Cassandra, it's
> >>>> >> very positive to see you here.
> >>>> >>
> >>>> >> I whole-heartedly agree with nearly everything you write. Some input
> >>>> >> and questions inline.
> >>>> >>
> >>>> >>
> >>>> >>>
> >>>> >>> As you may know, the infrastructure team has disabled public sign-up
> >>>> >>> to ASF JIRA (the GitHub issues are recommended instead).
> >>>> >>>
> >>>> >>
> >>>> >>
> >>>> >> I suspect (based on chatter in general, but not on dev@ AFAIK) is to
> >>>> >> avoid GH issues and stick with jira. The sign-up hurdle we will
> >>>> >> document on the website: CASSANDRA-18064
> >>>> >>
> >>>> >>
> >>>> >>
> >>>> >>> == 1. Make the checkstyle config a single point of truth for the
> >>>> >>> source code style. ==
> >>>> >>>
> >>>> >>> The checkstyle is already used and included in the Cassandra project
> >>>> >>> build lifecycle (ant command line, Jenkins, CircleCI). There is no
> >>>> >>> need to maintain code style configurations for different types of IDEs
> >>>> >>> (e.g. IntelliJ inspections configuration) since the checkstyle.xml
> >>>> >>> file can be directly imported to IDE used by a developer. This is fair
> >>>> >>> for Intellij Idea, NetBeans, and Eclipse.
> >>>> >>
> >>>> >>
> >>>> >> Big +1
> >>>> >>
> >>>> >>
> >>>> >>> So, I propose to focus on the checks themselves and checking pull
> >>>> >>> requests with automation scripts, rather than maintaining these
> >>>> >>> integrations. The benefits here are avoiding all issues with
> >>>> >>> maintaining configurations for different IDEs. Another good advantage
> >>>> >>> of this approach would be the ability to add new checkstyle rules
> >>>> >>> without touching IDE configuration - and such tickets will be LFH and
> >>>> >>> easy to commit.
> >>>> >>>
> >>>> >>> The actions points here are:
> >>>> >>>
> >>>> >>> - create an umbrella JIRA ticket for all checkstyle issues e.g. [8]
> >>>> >>> (or label checkstyle);
> >>>> >>> - add checkstyle to GitHub pull requests using GitHub actions (execute
> >>>> >>> ant command);
> >>>> >>
> >>>> >>
> >>>> >> Instead of custom GHA scripting, please use our existing
> >>>> >> cassandra-artifact.sh (which should already include all such checks).
> >>>> >>
> >>>> >> Something like https://github.com/apache/cassandra/compare/cassandra-3.11...thelastpickle:cassandra:mck/github-actions/3.11
> >>>> >>
> >>>> >>
> >>>> >>
> >>>> >>> == 3. Enable pushing backwards build results for both Jenkins and
> >>>> >>> CircleCI to GitHub pull requests. ==
> >>>> >>>
> >>>> >>> The goal here is to have a "green checkbox" for those GitHub pull
> >>>> >>> requests that have corresponding Jenkins/CircleCI runs. According to
> >>>> >>> the following links, it is completely possible to have those.
> >>>> >>>
> >>>> >>> https://github.com/jenkinsci/github-branch-source-plugin
> >>>> >>> https://circleci.com/docs/enable-checks/
> >>>> >>>
> >>>> >>> Moreover, the GitHub Branch Source Plugin is already enabled for the
> >>>> >>> Cassandra project [16]. The same seems should work the same way for
> >>>> >>> CirleCI, but I have faced the infrastructure team comment [17] that
> >>>> >>> describes admin permissions are required for that, so the question is
> >>>> >>> still open here. I could dig a bit more once we've agreed on it.
> >>>> >>>
> >>>> >>> The actions points here are:
> >>>> >>> - enable Jenkins integration for GitHub pull requests;
> >>>> >>> - enable CircleCI integration for GitHub pull requests;
> >>>> >>
> >>>> >>
> >>>> >> Some folk use CircleCI, some use ci-cassandra. The green checkbox idea
> >>>> >> is great, so long as it's optional. We don't want PRs triggering the
> >>>> >> runs either (there are other mechanisms for triggering for now).
> >>>> >>
> >>>> >>
> >>>> >>> The actions points here are:
> >>>> >>>
> >>>> >>> - initiate a wide survey for user and dev lists, to get to know about
> >>>> >>> the usages;
> >>>> >>> - remove those configurations that are not used anymore;
> >>>> >>> - force migration from Ant to Gradle/Maven;
> >>>> >>
> >>>> >>
> >>>> >> Let's leave this out for now. There's too many unknowns here. If
> >>>> >> there's an IDE configuration that's broken, no one has reported it for
> >>>> >> ages, and no one is around to fix it, then I say we should raise the
> >>>> >> discussion to remove it.
> >>>> >>
> >>>> >> The Gradle/Maven migration is a hot one, contribute to that discussion
> >>>> >> but let's not tangle this work up with it, IMHO.
> >>>> >>
> >>>> >> Totally agree that IDE project files should be as light weight as possible.
> >>>> >>
> >>>> >>
> >>>> >>> Summarizing everything proposed above I think it is possible to
> >>>> >>> simplify adding small contributions easier to the codebase as well as
> >>>> >>> save a bunch of committer's time.
> >>>> >>>
> >>>> >>> So,
> >>>> >>> WDYT about the things described above?
> >>>> >>> Should I create a CEP for this?
> >>>> >>
> >>>> >>
> >>>> >> I see no need for a CEP here. An epic and tickets will work.
> >>>> >> Again, thanks for the input Maxim!
> >>>>

Re: [DISCUSSION] Cassandra's code style and source code analysis

[Maxim Muzafarov <mm...@apache.org>]

> Technically it can be two commits which would be merged / pushed at once.

I'll prepare a new pull request containing both of the changes. My
previous experience says me that it's really hard to find a reviewer
who will be able to go through huge pull requests, that's why
initially I've split this into AvoidStarImport and CustomImportOrder
rules. So, if you'll help with the review I'm happy to proceed the way
you suggested :-)

> One thing which needs extra care for ordering imports is that if you order it in IDEA by right-clicking on a package and choosing organising imports, it will remove special comments

You're right, but this is quite unusual behaviour for me and this
seems to be a bug, that hasn't been fixed for a long time [1]. I've
tested the same thing for Eclipse and NetBeans and `optimize imports`
working there as we expect (no comments removes), so the issue exists
only for the IntelliJ IDEA [1].
Despite all of that, we are still on the safe side here - if these
comments will be removed by the `optimized import` procedure the build
with checkstyle will fail.

> I think this is a great time to revisit this ordering.

I would say that the imports order is pretty good (probably, except
for the blank lines) and the imports order is not as important as it
is important that it be the same in all files and automation `optimize
imports`.
I suggest going through a "minimum change" strategy here. The IntelliJ
IDEA has the following configuration with the imports order that most
of the classes already fit:

import java
import javax
[blank line]
import com.google.common
import org.apache.log4j
import org.apache.commons
import org.cliffc.high_scale_lib
import org.junit
import org.slf4j
[blank line]
import all other imports
[blank line]
import static all other imports

We can update the documentation page [2] with this order and implement
the same for NetBeans and Eclipse IDE configuration files as well as
for the checkstyle config.


If everyone is OK with the plan above I'll prepare everything for it.

Suggested summary:
- use current IntelliJ IDEA imports order as defaults for other IDEs;
- update the documentation page;
- prepare a single pull request with AvoidStarImport and CustomImportOrder;



[1] https://youtrack.jetbrains.com/issue/IDEA-128133/Optimize-Imports-disregards-line-comments
[2] https://cassandra.apache.org/_/development/code_style.html

On Sun, 11 Dec 2022 at 00:03, Miklosovic, Stefan
<St...@netapp.com> wrote:
>
> Should the source code obey the AvoidStarImport rule?
>
> yes
>
>  Should we implement AvoidStarImport and CustomImportOrder in a
> single pull request or do it one by one?
>
> Technically it can be two commits which would be merged / pushed at once.
>
> One thing which needs extra care for ordering imports is that if you order it in IDEA by right-clicking on a package and choosing organising imports, it will remove special comments which are put at the end of the import statement. We need to be sure you put them back.  Look at changes in CASSANDRA-17055. We need to preserve this.
>
> Also, we need to be sure that the importing style can be (roughly) set in each major IDE. (eclipse / netbeans / idea) so if we require some import style it can be set in IDE like that. I do not know if we have any strong preference when it comes to this but it definitely does not hurt.
>
> Also, I see that the current import style is
>
> java
> [blank line]
> com.google.common
> org.apache.commons
> org.junit
> org.slf4j
> [blank line]
> everything else alphabetically
>
> I think this is a great time to revisit this ordering. I am not particularly persuaded on this order and why it was choosen. Where has that decision come from?
>
> ________________________________________
> From: Maxim Muzafarov <mm...@apache.org>
> Sent: Wednesday, December 7, 2022 18:29
> To: dev@cassandra.apache.org
> Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis
>
> NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.
>
>
>
>
> Dear community,
>
>
> I have created the epic with code-style activities to track the progress:
> https://issues.apache.org/jira/browse/CASSANDRA-18090
>
> In my understanding, there is no need to format whole the code base at
> once according to the code style described on the page [1], and the
> best strategy here is to go forward with small evolutionary changes.
> Thus eventually we will come up with a set of rules convenient for all
> members of the community. In my mind, having one commit per an added
> code style rule should be easy to look at for a reviewer, the git
> commits history as well as rebasing/merging other pull requests that
> may be affected by the new rules.
>
>
> I want to raise one more question related to class imports and the
> classses import order for a wider discussion. The import order is well
> described on the code style page [1], but using wildcard imports is
> not mentioned at all. The wildcard imports with their drawbacks has
> has already been raised in the JIRA issue [2] and didn't get enough
> attention.
>
> The checkstyle has the rules we are interested in for import control
> and they must be considered together. We can implement them in a
> single pull request or one by one, or use only the last one:
> - AvoidStarImport
> - CustomImportOrder
>
> But still, I think that wildcard imports have more disadvantages
> (class names conflicts e.g. java.util.*, java.sql.* or a new version
> of a library has name clashes) than advantages and such problems will
> be found in later CI cycles.
> Currently, I've implemented the AvoidStarImport checkstyle rule in a
> dedicated pull request [3][4], so you will be able to see all amount
> of the changes with removing wildcard imports. The changes are made
> for the checkstyle configuration as well as for code style
> configurations for different IDEs we supported.
>
> So, the open questions here are:
>
> - Should the source code obey the AvoidStarImport rule [3]? (I think yes);
> - Should we implement AvoidStarImport and CustomImportOrder in a
> single pull request or do it one by one?
>
>
> Anyway, I will fix the result of the agreement over the
> AvoidStarImport rule on the documentation page [1].
>
>
>
> [1] https://cassandra.apache.org/_/development/code_style.html
> [2] https://issues.apache.org/jira/browse/CASSANDRA-17925
> [3] https://issues.apache.org/jira/browse/CASSANDRA-18089
> [4] https://github.com/apache/cassandra/pull/2041
>
> On Thu, 1 Dec 2022 at 11:55, Claude Warren, Jr via dev
> <de...@cassandra.apache.org> wrote:
> >
> > The last time I worked on a project that tried to implement a coding style across the project it was "an education".  The short story is that trying to "mitigate" the code base, with respect to style, is either a massive change or a long slow process.
> >
> > Arguments here have stated that earlier attempts to have the tooling reformat the code did not go well.  What we ended up doing was turned on the style checker and looked at the number of issues across the project.  When new code was accepted the number of issues could not rise.  Eventually most of the code was clean, with a few well coded legacy bits still not up to standard.  We could do something similar here.  Much like code coverage, you can't perform a merge unless the number of style errors remains the same or decreases.
> >
> > As with all software rules, this is a strong recommendation as I am certain that there are edge/corner case exceptions to be found.
> >
> >
> >
> >
> > On Wed, Nov 30, 2022 at 3:30 PM Patrick McFadin <pm...@gmail.com> wrote:
> >>
> >> Why are we still debating build tooling? I think you’re wrong, but I’ve conceded - on the assumption that we can get enough volunteers willing to adopt responsibility for the new world order.
> >>
> >> Not debating. I am just throwing in my support since I have been in the Camp of Ant.
> >>
> >> On Wed, Nov 30, 2022 at 1:29 AM Benedict <be...@apache.org> wrote:
> >>>
> >>> Why are we still debating build tooling? I think you’re wrong, but I’ve conceded - on the assumption that we can get enough volunteers willing to adopt responsibility for the new world order.
> >>>
> >>> I suggest five long term contributors nominate themselves as the build file maintainers, and collectively manage a safe and painless migration for the rest of us - and agree to maintain and develop the new build file going forwards, and support the community as they adopt it.
> >>>
> >>> On the topic of over-exuberant linting I will continue to push back. I think linting our brace rules could make sense since they are atypical, but more formatting rules than this likely just leads to atrophying style. Authorship involves thinking about how to present your code; I don’t want to either encourage lazy authorship or prevent experimentation with presentation. Both would be bad, and I expect we would struggle to evolve our style guide again in future as the language evolves. Our brace rules are a good example everyone unilaterally ignored when lambdas arrived, as we all recognised they materially harmed the brevity benefits, and we eventually codified this.
> >>>
> >>> On migration: auto formatters applied to code that was not written with the rules in mind will almost unerringly be made a mess of, so having a tool do this is not acceptable IMO.
> >>>
> >>> The idea of checkstyle being the source of truth continues to be untenable and anyone still pushing for this should please engage with my earlier points on this.
> >>>
> >>>
> >>> On 30 Nov 2022, at 04:06, Patrick McFadin <pm...@gmail.com> wrote:
> >>>
> >>> 
> >>> I'm going to +1 what Stefan has said. I've heard on many occasions from newcomers to the project that having to use Ant is a deterrent. As a matter of fact, a few weeks ago, I spent a Sunday afternoon helping somebody trying to build Cassandra and Ant caused a ton of problems. "Ok. ant really super clean this time"
> >>>
> >>> Sure it still works for people that have been doing this for years. I drive a 20 year old Toyota truck, but I'm reminded by my kids often that it's not cool. So in that spirit, I feel my saying we need to keep Ant is like saying "You kids get off my lawn!" If it's something that will help attract new contributors, I'm all for it.
> >>>
> >>> Patrick
> >>>
> >>> On Fri, Nov 25, 2022 at 2:22 AM Miklosovic, Stefan <St...@netapp.com> wrote:
> >>>>
> >>>> I agree with what you wrote. How I understand it is that migrating to Maven/Gradle makes the project more "attractive" for newcomers. If a project is built on "that old un-cool Ant", it might be a little bit off-putting and questionable if we are "stuck in the past on build systems and not progressing".
> >>>>
> >>>> So in that sense I agree this is more "marketing" rather than technological question but on the other hand, does not Maven/Gradle allow us to modularize the project better? Maybe we would like to modularize but nobody is up to that because build system makes it impossible or at least quite inconvenient to do so. Do you really think there are not any significant benefits to switch even if it "just works" now?
> >>>>
> >>>> ________________________________________
> >>>> From: Benedict <be...@apache.org>
> >>>> Sent: Friday, November 25, 2022 11:07
> >>>> To: dev@cassandra.apache.org
> >>>> Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis
> >>>>
> >>>> NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> There’s always a handful of people asking for it, but notably few if any of the full time contributors doing the majority of the core development of Cassandra. It strikes me as something very appealing to others, but less so to those wanting to get on with development.
> >>>>
> >>>> I never really see a good argument articulated for the migration, besides general hand waving that ant is old, and people like newer build systems. Ant is working fine, so there isn’t a strong technical reason to replace it, and there are good organisational reasons not to.
> >>>>
> >>>> Why do you consider a migration inevitable?
> >>>>
> >>>>
> >>>>
> >>>> > On 25 Nov 2022, at 09:58, Miklosovic, Stefan <St...@netapp.com> wrote:
> >>>> >
> >>>> > Interesting take on Ant / no-Ant, Benedict. I am very curious how this unfolds. My long-term perception is that changing it to something else is more or less inevitable but if there is a broader consensus to not do that .... well.
> >>>> >
> >>>> > ________________________________________
> >>>> > From: Benedict <be...@apache.org>
> >>>> > Sent: Friday, November 25, 2022 10:52
> >>>> > To: dev@cassandra.apache.org
> >>>> > Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis
> >>>> >
> >>>> > NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.
> >>>> >
> >>>> >
> >>>> >
> >>>> >
> >>>> > I was in a bit of a rush last night. I should say that I’m of course +1 a general endeavour to clean this up, and to expand our use of linters, and I appreciate your volunteering to help out in this way Maxim.
> >>>> >
> >>>> > However, responding to Stefan, I’m pretty -1 migrating from ant to another build system without really good reason. Migration has a real cost to productivity for all existing contributors, and the phantom of increasing new contributions has never paid off historically. I’m all for easing people into participation, but not at penalty to the existing contributor base.
> >>>> >
> >>>> > If the only reason is to make it easier to open in a different IDE, we can perhaps have some basic build files outlining code structure for importing, that are compatible with our canonical ant build? We could perhaps even generate them.
> >>>> >
> >>>> >
> >>>> >> On 25 Nov 2022, at 09:35, Miklosovic, Stefan <St...@netapp.com> wrote:
> >>>> >>
> >>>> >> For the record, I was testing that same combo Claude mentioned and it did not work out of the box but it is definitely possible to set up successfully. I do not remember the details.
> >>>> >>
> >>>> >> To replay to Maxim, it all seems good to me, roughly, but I humbly think it all boils down to Maven/Gradle refactoring and on top of that we can do all else.
> >>>> >>
> >>>> >> For example, there is (1) where the solution, besides fixing the tests, is to introduce an Ant task which would check this on build. That being said, how is that going to look like when we change Ant for something else? That stuff suddenly becomes obsolete.
> >>>> >>
> >>>> >> This case maybe applies to other problems we want to solve as well. I do not want to do something tailored for one build system just to rewrite it all or to spend significant amount of time on that again when we switch the build system.
> >>>> >>
> >>>> >> For that reason I think changing Ant for something else should be top priority (as I understand that it the hot topic for community for very long time) and then everything else should follow. We should spend time on things mentioned only in case they do not collide with any build system at all.
> >>>> >>
> >>>> >> (1) https://issues.apache.org/jira/browse/CASSANDRA-17964
> >>>> >>
> >>>> >> Stefan
> >>>> >>
> >>>> >> ________________________________________
> >>>> >> From: Claude Warren, Jr via dev <de...@cassandra.apache.org>
> >>>> >> Sent: Friday, November 25, 2022 10:16
> >>>> >> To: dev@cassandra.apache.org
> >>>> >> Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis
> >>>> >>
> >>>> >> NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.
> >>>> >>
> >>>> >>
> >>>> >>
> >>>> >> +1 for the concept as a whole.  I am certain I could find nits to pick if I looked deeply.
> >>>> >>
> >>>> >> @mck -- I did have a problem with Cassandra + Eclipse + Java11 (Classpath).  I gave up and am spending time trying to learn IntelliJ.  I also mentioned it in one of the discussion areas.
> >>>> >>
> >>>> >> Claude
> >>>> >>
> >>>> >> On Thu, Nov 24, 2022 at 8:55 PM Mick Semb Wever <mc...@apache.org>> wrote:
> >>>> >> Thank you for a solid write up Maxim. And welcome to Cassandra, it's
> >>>> >> very positive to see you here.
> >>>> >>
> >>>> >> I whole-heartedly agree with nearly everything you write. Some input
> >>>> >> and questions inline.
> >>>> >>
> >>>> >>
> >>>> >>>
> >>>> >>> As you may know, the infrastructure team has disabled public sign-up
> >>>> >>> to ASF JIRA (the GitHub issues are recommended instead).
> >>>> >>>
> >>>> >>
> >>>> >>
> >>>> >> I suspect (based on chatter in general, but not on dev@ AFAIK) is to
> >>>> >> avoid GH issues and stick with jira. The sign-up hurdle we will
> >>>> >> document on the website: CASSANDRA-18064
> >>>> >>
> >>>> >>
> >>>> >>
> >>>> >>> == 1. Make the checkstyle config a single point of truth for the
> >>>> >>> source code style. ==
> >>>> >>>
> >>>> >>> The checkstyle is already used and included in the Cassandra project
> >>>> >>> build lifecycle (ant command line, Jenkins, CircleCI). There is no
> >>>> >>> need to maintain code style configurations for different types of IDEs
> >>>> >>> (e.g. IntelliJ inspections configuration) since the checkstyle.xml
> >>>> >>> file can be directly imported to IDE used by a developer. This is fair
> >>>> >>> for Intellij Idea, NetBeans, and Eclipse.
> >>>> >>
> >>>> >>
> >>>> >> Big +1
> >>>> >>
> >>>> >>
> >>>> >>> So, I propose to focus on the checks themselves and checking pull
> >>>> >>> requests with automation scripts, rather than maintaining these
> >>>> >>> integrations. The benefits here are avoiding all issues with
> >>>> >>> maintaining configurations for different IDEs. Another good advantage
> >>>> >>> of this approach would be the ability to add new checkstyle rules
> >>>> >>> without touching IDE configuration - and such tickets will be LFH and
> >>>> >>> easy to commit.
> >>>> >>>
> >>>> >>> The actions points here are:
> >>>> >>>
> >>>> >>> - create an umbrella JIRA ticket for all checkstyle issues e.g. [8]
> >>>> >>> (or label checkstyle);
> >>>> >>> - add checkstyle to GitHub pull requests using GitHub actions (execute
> >>>> >>> ant command);
> >>>> >>
> >>>> >>
> >>>> >> Instead of custom GHA scripting, please use our existing
> >>>> >> cassandra-artifact.sh (which should already include all such checks).
> >>>> >>
> >>>> >> Something like https://github.com/apache/cassandra/compare/cassandra-3.11...thelastpickle:cassandra:mck/github-actions/3.11
> >>>> >>
> >>>> >>
> >>>> >>
> >>>> >>> == 3. Enable pushing backwards build results for both Jenkins and
> >>>> >>> CircleCI to GitHub pull requests. ==
> >>>> >>>
> >>>> >>> The goal here is to have a "green checkbox" for those GitHub pull
> >>>> >>> requests that have corresponding Jenkins/CircleCI runs. According to
> >>>> >>> the following links, it is completely possible to have those.
> >>>> >>>
> >>>> >>> https://github.com/jenkinsci/github-branch-source-plugin
> >>>> >>> https://circleci.com/docs/enable-checks/
> >>>> >>>
> >>>> >>> Moreover, the GitHub Branch Source Plugin is already enabled for the
> >>>> >>> Cassandra project [16]. The same seems should work the same way for
> >>>> >>> CirleCI, but I have faced the infrastructure team comment [17] that
> >>>> >>> describes admin permissions are required for that, so the question is
> >>>> >>> still open here. I could dig a bit more once we've agreed on it.
> >>>> >>>
> >>>> >>> The actions points here are:
> >>>> >>> - enable Jenkins integration for GitHub pull requests;
> >>>> >>> - enable CircleCI integration for GitHub pull requests;
> >>>> >>
> >>>> >>
> >>>> >> Some folk use CircleCI, some use ci-cassandra. The green checkbox idea
> >>>> >> is great, so long as it's optional. We don't want PRs triggering the
> >>>> >> runs either (there are other mechanisms for triggering for now).
> >>>> >>
> >>>> >>
> >>>> >>> The actions points here are:
> >>>> >>>
> >>>> >>> - initiate a wide survey for user and dev lists, to get to know about
> >>>> >>> the usages;
> >>>> >>> - remove those configurations that are not used anymore;
> >>>> >>> - force migration from Ant to Gradle/Maven;
> >>>> >>
> >>>> >>
> >>>> >> Let's leave this out for now. There's too many unknowns here. If
> >>>> >> there's an IDE configuration that's broken, no one has reported it for
> >>>> >> ages, and no one is around to fix it, then I say we should raise the
> >>>> >> discussion to remove it.
> >>>> >>
> >>>> >> The Gradle/Maven migration is a hot one, contribute to that discussion
> >>>> >> but let's not tangle this work up with it, IMHO.
> >>>> >>
> >>>> >> Totally agree that IDE project files should be as light weight as possible.
> >>>> >>
> >>>> >>
> >>>> >>> Summarizing everything proposed above I think it is possible to
> >>>> >>> simplify adding small contributions easier to the codebase as well as
> >>>> >>> save a bunch of committer's time.
> >>>> >>>
> >>>> >>> So,
> >>>> >>> WDYT about the things described above?
> >>>> >>> Should I create a CEP for this?
> >>>> >>
> >>>> >>
> >>>> >> I see no need for a CEP here. An epic and tickets will work.
> >>>> >> Again, thanks for the input Maxim!
> >>>>

Re: [DISCUSSION] Cassandra's code style and source code analysis

["Miklosovic, Stefan" <St...@netapp.com>]

Should the source code obey the AvoidStarImport rule?

yes

 Should we implement AvoidStarImport and CustomImportOrder in a
single pull request or do it one by one?

Technically it can be two commits which would be merged / pushed at once.

One thing which needs extra care for ordering imports is that if you order it in IDEA by right-clicking on a package and choosing organising imports, it will remove special comments which are put at the end of the import statement. We need to be sure you put them back.  Look at changes in CASSANDRA-17055. We need to preserve this.

Also, we need to be sure that the importing style can be (roughly) set in each major IDE. (eclipse / netbeans / idea) so if we require some import style it can be set in IDE like that. I do not know if we have any strong preference when it comes to this but it definitely does not hurt.

Also, I see that the current import style is

java
[blank line]
com.google.common
org.apache.commons
org.junit
org.slf4j
[blank line]
everything else alphabetically

I think this is a great time to revisit this ordering. I am not particularly persuaded on this order and why it was choosen. Where has that decision come from?

________________________________________
From: Maxim Muzafarov <mm...@apache.org>
Sent: Wednesday, December 7, 2022 18:29
To: dev@cassandra.apache.org
Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis

NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.




Dear community,


I have created the epic with code-style activities to track the progress:
https://issues.apache.org/jira/browse/CASSANDRA-18090

In my understanding, there is no need to format whole the code base at
once according to the code style described on the page [1], and the
best strategy here is to go forward with small evolutionary changes.
Thus eventually we will come up with a set of rules convenient for all
members of the community. In my mind, having one commit per an added
code style rule should be easy to look at for a reviewer, the git
commits history as well as rebasing/merging other pull requests that
may be affected by the new rules.


I want to raise one more question related to class imports and the
classses import order for a wider discussion. The import order is well
described on the code style page [1], but using wildcard imports is
not mentioned at all. The wildcard imports with their drawbacks has
has already been raised in the JIRA issue [2] and didn't get enough
attention.

The checkstyle has the rules we are interested in for import control
and they must be considered together. We can implement them in a
single pull request or one by one, or use only the last one:
- AvoidStarImport
- CustomImportOrder

But still, I think that wildcard imports have more disadvantages
(class names conflicts e.g. java.util.*, java.sql.* or a new version
of a library has name clashes) than advantages and such problems will
be found in later CI cycles.
Currently, I've implemented the AvoidStarImport checkstyle rule in a
dedicated pull request [3][4], so you will be able to see all amount
of the changes with removing wildcard imports. The changes are made
for the checkstyle configuration as well as for code style
configurations for different IDEs we supported.

So, the open questions here are:

- Should the source code obey the AvoidStarImport rule [3]? (I think yes);
- Should we implement AvoidStarImport and CustomImportOrder in a
single pull request or do it one by one?


Anyway, I will fix the result of the agreement over the
AvoidStarImport rule on the documentation page [1].



[1] https://cassandra.apache.org/_/development/code_style.html
[2] https://issues.apache.org/jira/browse/CASSANDRA-17925
[3] https://issues.apache.org/jira/browse/CASSANDRA-18089
[4] https://github.com/apache/cassandra/pull/2041

On Thu, 1 Dec 2022 at 11:55, Claude Warren, Jr via dev
<de...@cassandra.apache.org> wrote:
>
> The last time I worked on a project that tried to implement a coding style across the project it was "an education".  The short story is that trying to "mitigate" the code base, with respect to style, is either a massive change or a long slow process.
>
> Arguments here have stated that earlier attempts to have the tooling reformat the code did not go well.  What we ended up doing was turned on the style checker and looked at the number of issues across the project.  When new code was accepted the number of issues could not rise.  Eventually most of the code was clean, with a few well coded legacy bits still not up to standard.  We could do something similar here.  Much like code coverage, you can't perform a merge unless the number of style errors remains the same or decreases.
>
> As with all software rules, this is a strong recommendation as I am certain that there are edge/corner case exceptions to be found.
>
>
>
>
> On Wed, Nov 30, 2022 at 3:30 PM Patrick McFadin <pm...@gmail.com> wrote:
>>
>> Why are we still debating build tooling? I think you’re wrong, but I’ve conceded - on the assumption that we can get enough volunteers willing to adopt responsibility for the new world order.
>>
>> Not debating. I am just throwing in my support since I have been in the Camp of Ant.
>>
>> On Wed, Nov 30, 2022 at 1:29 AM Benedict <be...@apache.org> wrote:
>>>
>>> Why are we still debating build tooling? I think you’re wrong, but I’ve conceded - on the assumption that we can get enough volunteers willing to adopt responsibility for the new world order.
>>>
>>> I suggest five long term contributors nominate themselves as the build file maintainers, and collectively manage a safe and painless migration for the rest of us - and agree to maintain and develop the new build file going forwards, and support the community as they adopt it.
>>>
>>> On the topic of over-exuberant linting I will continue to push back. I think linting our brace rules could make sense since they are atypical, but more formatting rules than this likely just leads to atrophying style. Authorship involves thinking about how to present your code; I don’t want to either encourage lazy authorship or prevent experimentation with presentation. Both would be bad, and I expect we would struggle to evolve our style guide again in future as the language evolves. Our brace rules are a good example everyone unilaterally ignored when lambdas arrived, as we all recognised they materially harmed the brevity benefits, and we eventually codified this.
>>>
>>> On migration: auto formatters applied to code that was not written with the rules in mind will almost unerringly be made a mess of, so having a tool do this is not acceptable IMO.
>>>
>>> The idea of checkstyle being the source of truth continues to be untenable and anyone still pushing for this should please engage with my earlier points on this.
>>>
>>>
>>> On 30 Nov 2022, at 04:06, Patrick McFadin <pm...@gmail.com> wrote:
>>>
>>> 
>>> I'm going to +1 what Stefan has said. I've heard on many occasions from newcomers to the project that having to use Ant is a deterrent. As a matter of fact, a few weeks ago, I spent a Sunday afternoon helping somebody trying to build Cassandra and Ant caused a ton of problems. "Ok. ant really super clean this time"
>>>
>>> Sure it still works for people that have been doing this for years. I drive a 20 year old Toyota truck, but I'm reminded by my kids often that it's not cool. So in that spirit, I feel my saying we need to keep Ant is like saying "You kids get off my lawn!" If it's something that will help attract new contributors, I'm all for it.
>>>
>>> Patrick
>>>
>>> On Fri, Nov 25, 2022 at 2:22 AM Miklosovic, Stefan <St...@netapp.com> wrote:
>>>>
>>>> I agree with what you wrote. How I understand it is that migrating to Maven/Gradle makes the project more "attractive" for newcomers. If a project is built on "that old un-cool Ant", it might be a little bit off-putting and questionable if we are "stuck in the past on build systems and not progressing".
>>>>
>>>> So in that sense I agree this is more "marketing" rather than technological question but on the other hand, does not Maven/Gradle allow us to modularize the project better? Maybe we would like to modularize but nobody is up to that because build system makes it impossible or at least quite inconvenient to do so. Do you really think there are not any significant benefits to switch even if it "just works" now?
>>>>
>>>> ________________________________________
>>>> From: Benedict <be...@apache.org>
>>>> Sent: Friday, November 25, 2022 11:07
>>>> To: dev@cassandra.apache.org
>>>> Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis
>>>>
>>>> NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.
>>>>
>>>>
>>>>
>>>>
>>>> There’s always a handful of people asking for it, but notably few if any of the full time contributors doing the majority of the core development of Cassandra. It strikes me as something very appealing to others, but less so to those wanting to get on with development.
>>>>
>>>> I never really see a good argument articulated for the migration, besides general hand waving that ant is old, and people like newer build systems. Ant is working fine, so there isn’t a strong technical reason to replace it, and there are good organisational reasons not to.
>>>>
>>>> Why do you consider a migration inevitable?
>>>>
>>>>
>>>>
>>>> > On 25 Nov 2022, at 09:58, Miklosovic, Stefan <St...@netapp.com> wrote:
>>>> >
>>>> > Interesting take on Ant / no-Ant, Benedict. I am very curious how this unfolds. My long-term perception is that changing it to something else is more or less inevitable but if there is a broader consensus to not do that .... well.
>>>> >
>>>> > ________________________________________
>>>> > From: Benedict <be...@apache.org>
>>>> > Sent: Friday, November 25, 2022 10:52
>>>> > To: dev@cassandra.apache.org
>>>> > Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis
>>>> >
>>>> > NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.
>>>> >
>>>> >
>>>> >
>>>> >
>>>> > I was in a bit of a rush last night. I should say that I’m of course +1 a general endeavour to clean this up, and to expand our use of linters, and I appreciate your volunteering to help out in this way Maxim.
>>>> >
>>>> > However, responding to Stefan, I’m pretty -1 migrating from ant to another build system without really good reason. Migration has a real cost to productivity for all existing contributors, and the phantom of increasing new contributions has never paid off historically. I’m all for easing people into participation, but not at penalty to the existing contributor base.
>>>> >
>>>> > If the only reason is to make it easier to open in a different IDE, we can perhaps have some basic build files outlining code structure for importing, that are compatible with our canonical ant build? We could perhaps even generate them.
>>>> >
>>>> >
>>>> >> On 25 Nov 2022, at 09:35, Miklosovic, Stefan <St...@netapp.com> wrote:
>>>> >>
>>>> >> For the record, I was testing that same combo Claude mentioned and it did not work out of the box but it is definitely possible to set up successfully. I do not remember the details.
>>>> >>
>>>> >> To replay to Maxim, it all seems good to me, roughly, but I humbly think it all boils down to Maven/Gradle refactoring and on top of that we can do all else.
>>>> >>
>>>> >> For example, there is (1) where the solution, besides fixing the tests, is to introduce an Ant task which would check this on build. That being said, how is that going to look like when we change Ant for something else? That stuff suddenly becomes obsolete.
>>>> >>
>>>> >> This case maybe applies to other problems we want to solve as well. I do not want to do something tailored for one build system just to rewrite it all or to spend significant amount of time on that again when we switch the build system.
>>>> >>
>>>> >> For that reason I think changing Ant for something else should be top priority (as I understand that it the hot topic for community for very long time) and then everything else should follow. We should spend time on things mentioned only in case they do not collide with any build system at all.
>>>> >>
>>>> >> (1) https://issues.apache.org/jira/browse/CASSANDRA-17964
>>>> >>
>>>> >> Stefan
>>>> >>
>>>> >> ________________________________________
>>>> >> From: Claude Warren, Jr via dev <de...@cassandra.apache.org>
>>>> >> Sent: Friday, November 25, 2022 10:16
>>>> >> To: dev@cassandra.apache.org
>>>> >> Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis
>>>> >>
>>>> >> NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.
>>>> >>
>>>> >>
>>>> >>
>>>> >> +1 for the concept as a whole.  I am certain I could find nits to pick if I looked deeply.
>>>> >>
>>>> >> @mck -- I did have a problem with Cassandra + Eclipse + Java11 (Classpath).  I gave up and am spending time trying to learn IntelliJ.  I also mentioned it in one of the discussion areas.
>>>> >>
>>>> >> Claude
>>>> >>
>>>> >> On Thu, Nov 24, 2022 at 8:55 PM Mick Semb Wever <mc...@apache.org>> wrote:
>>>> >> Thank you for a solid write up Maxim. And welcome to Cassandra, it's
>>>> >> very positive to see you here.
>>>> >>
>>>> >> I whole-heartedly agree with nearly everything you write. Some input
>>>> >> and questions inline.
>>>> >>
>>>> >>
>>>> >>>
>>>> >>> As you may know, the infrastructure team has disabled public sign-up
>>>> >>> to ASF JIRA (the GitHub issues are recommended instead).
>>>> >>>
>>>> >>
>>>> >>
>>>> >> I suspect (based on chatter in general, but not on dev@ AFAIK) is to
>>>> >> avoid GH issues and stick with jira. The sign-up hurdle we will
>>>> >> document on the website: CASSANDRA-18064
>>>> >>
>>>> >>
>>>> >>
>>>> >>> == 1. Make the checkstyle config a single point of truth for the
>>>> >>> source code style. ==
>>>> >>>
>>>> >>> The checkstyle is already used and included in the Cassandra project
>>>> >>> build lifecycle (ant command line, Jenkins, CircleCI). There is no
>>>> >>> need to maintain code style configurations for different types of IDEs
>>>> >>> (e.g. IntelliJ inspections configuration) since the checkstyle.xml
>>>> >>> file can be directly imported to IDE used by a developer. This is fair
>>>> >>> for Intellij Idea, NetBeans, and Eclipse.
>>>> >>
>>>> >>
>>>> >> Big +1
>>>> >>
>>>> >>
>>>> >>> So, I propose to focus on the checks themselves and checking pull
>>>> >>> requests with automation scripts, rather than maintaining these
>>>> >>> integrations. The benefits here are avoiding all issues with
>>>> >>> maintaining configurations for different IDEs. Another good advantage
>>>> >>> of this approach would be the ability to add new checkstyle rules
>>>> >>> without touching IDE configuration - and such tickets will be LFH and
>>>> >>> easy to commit.
>>>> >>>
>>>> >>> The actions points here are:
>>>> >>>
>>>> >>> - create an umbrella JIRA ticket for all checkstyle issues e.g. [8]
>>>> >>> (or label checkstyle);
>>>> >>> - add checkstyle to GitHub pull requests using GitHub actions (execute
>>>> >>> ant command);
>>>> >>
>>>> >>
>>>> >> Instead of custom GHA scripting, please use our existing
>>>> >> cassandra-artifact.sh (which should already include all such checks).
>>>> >>
>>>> >> Something like https://github.com/apache/cassandra/compare/cassandra-3.11...thelastpickle:cassandra:mck/github-actions/3.11
>>>> >>
>>>> >>
>>>> >>
>>>> >>> == 3. Enable pushing backwards build results for both Jenkins and
>>>> >>> CircleCI to GitHub pull requests. ==
>>>> >>>
>>>> >>> The goal here is to have a "green checkbox" for those GitHub pull
>>>> >>> requests that have corresponding Jenkins/CircleCI runs. According to
>>>> >>> the following links, it is completely possible to have those.
>>>> >>>
>>>> >>> https://github.com/jenkinsci/github-branch-source-plugin
>>>> >>> https://circleci.com/docs/enable-checks/
>>>> >>>
>>>> >>> Moreover, the GitHub Branch Source Plugin is already enabled for the
>>>> >>> Cassandra project [16]. The same seems should work the same way for
>>>> >>> CirleCI, but I have faced the infrastructure team comment [17] that
>>>> >>> describes admin permissions are required for that, so the question is
>>>> >>> still open here. I could dig a bit more once we've agreed on it.
>>>> >>>
>>>> >>> The actions points here are:
>>>> >>> - enable Jenkins integration for GitHub pull requests;
>>>> >>> - enable CircleCI integration for GitHub pull requests;
>>>> >>
>>>> >>
>>>> >> Some folk use CircleCI, some use ci-cassandra. The green checkbox idea
>>>> >> is great, so long as it's optional. We don't want PRs triggering the
>>>> >> runs either (there are other mechanisms for triggering for now).
>>>> >>
>>>> >>
>>>> >>> The actions points here are:
>>>> >>>
>>>> >>> - initiate a wide survey for user and dev lists, to get to know about
>>>> >>> the usages;
>>>> >>> - remove those configurations that are not used anymore;
>>>> >>> - force migration from Ant to Gradle/Maven;
>>>> >>
>>>> >>
>>>> >> Let's leave this out for now. There's too many unknowns here. If
>>>> >> there's an IDE configuration that's broken, no one has reported it for
>>>> >> ages, and no one is around to fix it, then I say we should raise the
>>>> >> discussion to remove it.
>>>> >>
>>>> >> The Gradle/Maven migration is a hot one, contribute to that discussion
>>>> >> but let's not tangle this work up with it, IMHO.
>>>> >>
>>>> >> Totally agree that IDE project files should be as light weight as possible.
>>>> >>
>>>> >>
>>>> >>> Summarizing everything proposed above I think it is possible to
>>>> >>> simplify adding small contributions easier to the codebase as well as
>>>> >>> save a bunch of committer's time.
>>>> >>>
>>>> >>> So,
>>>> >>> WDYT about the things described above?
>>>> >>> Should I create a CEP for this?
>>>> >>
>>>> >>
>>>> >> I see no need for a CEP here. An epic and tickets will work.
>>>> >> Again, thanks for the input Maxim!
>>>>

Re: [DISCUSSION] Cassandra's code style and source code analysis

["Claude Warren, Jr via dev" <de...@cassandra.apache.org>]

+1 avoid star rule

On Wed, Dec 7, 2022 at 5:30 PM Maxim Muzafarov <mm...@apache.org> wrote:

> Dear community,
>
>
> I have created the epic with code-style activities to track the progress:
> https://issues.apache.org/jira/browse/CASSANDRA-18090
>
> In my understanding, there is no need to format whole the code base at
> once according to the code style described on the page [1], and the
> best strategy here is to go forward with small evolutionary changes.
> Thus eventually we will come up with a set of rules convenient for all
> members of the community. In my mind, having one commit per an added
> code style rule should be easy to look at for a reviewer, the git
> commits history as well as rebasing/merging other pull requests that
> may be affected by the new rules.
>
>
> I want to raise one more question related to class imports and the
> classses import order for a wider discussion. The import order is well
> described on the code style page [1], but using wildcard imports is
> not mentioned at all. The wildcard imports with their drawbacks has
> has already been raised in the JIRA issue [2] and didn't get enough
> attention.
>
> The checkstyle has the rules we are interested in for import control
> and they must be considered together. We can implement them in a
> single pull request or one by one, or use only the last one:
> - AvoidStarImport
> - CustomImportOrder
>
> But still, I think that wildcard imports have more disadvantages
> (class names conflicts e.g. java.util.*, java.sql.* or a new version
> of a library has name clashes) than advantages and such problems will
> be found in later CI cycles.
> Currently, I've implemented the AvoidStarImport checkstyle rule in a
> dedicated pull request [3][4], so you will be able to see all amount
> of the changes with removing wildcard imports. The changes are made
> for the checkstyle configuration as well as for code style
> configurations for different IDEs we supported.
>
> So, the open questions here are:
>
> - Should the source code obey the AvoidStarImport rule [3]? (I think yes);
> - Should we implement AvoidStarImport and CustomImportOrder in a
> single pull request or do it one by one?
>
>
> Anyway, I will fix the result of the agreement over the
> AvoidStarImport rule on the documentation page [1].
>
>
>
> [1] https://cassandra.apache.org/_/development/code_style.html
> [2] https://issues.apache.org/jira/browse/CASSANDRA-17925
> [3] https://issues.apache.org/jira/browse/CASSANDRA-18089
> [4] https://github.com/apache/cassandra/pull/2041
>
> On Thu, 1 Dec 2022 at 11:55, Claude Warren, Jr via dev
> <de...@cassandra.apache.org> wrote:
> >
> > The last time I worked on a project that tried to implement a coding
> style across the project it was "an education".  The short story is that
> trying to "mitigate" the code base, with respect to style, is either a
> massive change or a long slow process.
> >
> > Arguments here have stated that earlier attempts to have the tooling
> reformat the code did not go well.  What we ended up doing was turned on
> the style checker and looked at the number of issues across the project.
> When new code was accepted the number of issues could not rise.  Eventually
> most of the code was clean, with a few well coded legacy bits still not up
> to standard.  We could do something similar here.  Much like code coverage,
> you can't perform a merge unless the number of style errors remains the
> same or decreases.
> >
> > As with all software rules, this is a strong recommendation as I am
> certain that there are edge/corner case exceptions to be found.
> >
> >
> >
> >
> > On Wed, Nov 30, 2022 at 3:30 PM Patrick McFadin <pm...@gmail.com>
> wrote:
> >>
> >> Why are we still debating build tooling? I think you’re wrong, but I’ve
> conceded - on the assumption that we can get enough volunteers willing to
> adopt responsibility for the new world order.
> >>
> >> Not debating. I am just throwing in my support since I have been in the
> Camp of Ant.
> >>
> >> On Wed, Nov 30, 2022 at 1:29 AM Benedict <be...@apache.org> wrote:
> >>>
> >>> Why are we still debating build tooling? I think you’re wrong, but
> I’ve conceded - on the assumption that we can get enough volunteers willing
> to adopt responsibility for the new world order.
> >>>
> >>> I suggest five long term contributors nominate themselves as the build
> file maintainers, and collectively manage a safe and painless migration for
> the rest of us - and agree to maintain and develop the new build file going
> forwards, and support the community as they adopt it.
> >>>
> >>> On the topic of over-exuberant linting I will continue to push back. I
> think linting our brace rules could make sense since they are atypical, but
> more formatting rules than this likely just leads to atrophying style.
> Authorship involves thinking about how to present your code; I don’t want
> to either encourage lazy authorship or prevent experimentation with
> presentation. Both would be bad, and I expect we would struggle to evolve
> our style guide again in future as the language evolves. Our brace rules
> are a good example everyone unilaterally ignored when lambdas arrived, as
> we all recognised they materially harmed the brevity benefits, and we
> eventually codified this.
> >>>
> >>> On migration: auto formatters applied to code that was not written
> with the rules in mind will almost unerringly be made a mess of, so having
> a tool do this is not acceptable IMO.
> >>>
> >>> The idea of checkstyle being the source of truth continues to be
> untenable and anyone still pushing for this should please engage with my
> earlier points on this.
> >>>
> >>>
> >>> On 30 Nov 2022, at 04:06, Patrick McFadin <pm...@gmail.com> wrote:
> >>>
> >>> 
> >>> I'm going to +1 what Stefan has said. I've heard on many occasions
> from newcomers to the project that having to use Ant is a deterrent. As a
> matter of fact, a few weeks ago, I spent a Sunday afternoon helping
> somebody trying to build Cassandra and Ant caused a ton of problems. "Ok.
> ant really super clean this time"
> >>>
> >>> Sure it still works for people that have been doing this for years. I
> drive a 20 year old Toyota truck, but I'm reminded by my kids often that
> it's not cool. So in that spirit, I feel my saying we need to keep Ant is
> like saying "You kids get off my lawn!" If it's something that will help
> attract new contributors, I'm all for it.
> >>>
> >>> Patrick
> >>>
> >>> On Fri, Nov 25, 2022 at 2:22 AM Miklosovic, Stefan <
> Stefan.Miklosovic@netapp.com> wrote:
> >>>>
> >>>> I agree with what you wrote. How I understand it is that migrating to
> Maven/Gradle makes the project more "attractive" for newcomers. If a
> project is built on "that old un-cool Ant", it might be a little bit
> off-putting and questionable if we are "stuck in the past on build systems
> and not progressing".
> >>>>
> >>>> So in that sense I agree this is more "marketing" rather than
> technological question but on the other hand, does not Maven/Gradle allow
> us to modularize the project better? Maybe we would like to modularize but
> nobody is up to that because build system makes it impossible or at least
> quite inconvenient to do so. Do you really think there are not any
> significant benefits to switch even if it "just works" now?
> >>>>
> >>>> ________________________________________
> >>>> From: Benedict <be...@apache.org>
> >>>> Sent: Friday, November 25, 2022 11:07
> >>>> To: dev@cassandra.apache.org
> >>>> Subject: Re: [DISCUSSION] Cassandra's code style and source code
> analysis
> >>>>
> >>>> NetApp Security WARNING: This is an external email. Do not click
> links or open attachments unless you recognize the sender and know the
> content is safe.
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> There’s always a handful of people asking for it, but notably few if
> any of the full time contributors doing the majority of the core
> development of Cassandra. It strikes me as something very appealing to
> others, but less so to those wanting to get on with development.
> >>>>
> >>>> I never really see a good argument articulated for the migration,
> besides general hand waving that ant is old, and people like newer build
> systems. Ant is working fine, so there isn’t a strong technical reason to
> replace it, and there are good organisational reasons not to.
> >>>>
> >>>> Why do you consider a migration inevitable?
> >>>>
> >>>>
> >>>>
> >>>> > On 25 Nov 2022, at 09:58, Miklosovic, Stefan <
> Stefan.Miklosovic@netapp.com> wrote:
> >>>> >
> >>>> > Interesting take on Ant / no-Ant, Benedict. I am very curious how
> this unfolds. My long-term perception is that changing it to something else
> is more or less inevitable but if there is a broader consensus to not do
> that .... well.
> >>>> >
> >>>> > ________________________________________
> >>>> > From: Benedict <be...@apache.org>
> >>>> > Sent: Friday, November 25, 2022 10:52
> >>>> > To: dev@cassandra.apache.org
> >>>> > Subject: Re: [DISCUSSION] Cassandra's code style and source code
> analysis
> >>>> >
> >>>> > NetApp Security WARNING: This is an external email. Do not click
> links or open attachments unless you recognize the sender and know the
> content is safe.
> >>>> >
> >>>> >
> >>>> >
> >>>> >
> >>>> > I was in a bit of a rush last night. I should say that I’m of
> course +1 a general endeavour to clean this up, and to expand our use of
> linters, and I appreciate your volunteering to help out in this way Maxim.
> >>>> >
> >>>> > However, responding to Stefan, I’m pretty -1 migrating from ant to
> another build system without really good reason. Migration has a real cost
> to productivity for all existing contributors, and the phantom of
> increasing new contributions has never paid off historically. I’m all for
> easing people into participation, but not at penalty to the existing
> contributor base.
> >>>> >
> >>>> > If the only reason is to make it easier to open in a different IDE,
> we can perhaps have some basic build files outlining code structure for
> importing, that are compatible with our canonical ant build? We could
> perhaps even generate them.
> >>>> >
> >>>> >
> >>>> >> On 25 Nov 2022, at 09:35, Miklosovic, Stefan <
> Stefan.Miklosovic@netapp.com> wrote:
> >>>> >>
> >>>> >> For the record, I was testing that same combo Claude mentioned
> and it did not work out of the box but it is definitely possible to set up
> successfully. I do not remember the details.
> >>>> >>
> >>>> >> To replay to Maxim, it all seems good to me, roughly, but I humbly
> think it all boils down to Maven/Gradle refactoring and on top of that we
> can do all else.
> >>>> >>
> >>>> >> For example, there is (1) where the solution, besides fixing the
> tests, is to introduce an Ant task which would check this on build. That
> being said, how is that going to look like when we change Ant for something
> else? That stuff suddenly becomes obsolete.
> >>>> >>
> >>>> >> This case maybe applies to other problems we want to solve as
> well. I do not want to do something tailored for one build system just to
> rewrite it all or to spend significant amount of time on that again when we
> switch the build system.
> >>>> >>
> >>>> >> For that reason I think changing Ant for something else should be
> top priority (as I understand that it the hot topic for community for very
> long time) and then everything else should follow. We should spend time on
> things mentioned only in case they do not collide with any build system at
> all.
> >>>> >>
> >>>> >> (1) https://issues.apache.org/jira/browse/CASSANDRA-17964
> >>>> >>
> >>>> >> Stefan
> >>>> >>
> >>>> >> ________________________________________
> >>>> >> From: Claude Warren, Jr via dev <de...@cassandra.apache.org>
> >>>> >> Sent: Friday, November 25, 2022 10:16
> >>>> >> To: dev@cassandra.apache.org
> >>>> >> Subject: Re: [DISCUSSION] Cassandra's code style and source code
> analysis
> >>>> >>
> >>>> >> NetApp Security WARNING: This is an external email. Do not click
> links or open attachments unless you recognize the sender and know the
> content is safe.
> >>>> >>
> >>>> >>
> >>>> >>
> >>>> >> +1 for the concept as a whole.  I am certain I could find nits to
> pick if I looked deeply.
> >>>> >>
> >>>> >> @mck -- I did have a problem with Cassandra + Eclipse + Java11
> (Classpath).  I gave up and am spending time trying to learn IntelliJ.  I
> also mentioned it in one of the discussion areas.
> >>>> >>
> >>>> >> Claude
> >>>> >>
> >>>> >> On Thu, Nov 24, 2022 at 8:55 PM Mick Semb Wever <mck@apache.org
> <ma...@apache.org>> wrote:
> >>>> >> Thank you for a solid write up Maxim. And welcome to Cassandra,
> it's
> >>>> >> very positive to see you here.
> >>>> >>
> >>>> >> I whole-heartedly agree with nearly everything you write. Some
> input
> >>>> >> and questions inline.
> >>>> >>
> >>>> >>
> >>>> >>>
> >>>> >>> As you may know, the infrastructure team has disabled public
> sign-up
> >>>> >>> to ASF JIRA (the GitHub issues are recommended instead).
> >>>> >>>
> >>>> >>
> >>>> >>
> >>>> >> I suspect (based on chatter in general, but not on dev@ AFAIK) is
> to
> >>>> >> avoid GH issues and stick with jira. The sign-up hurdle we will
> >>>> >> document on the website: CASSANDRA-18064
> >>>> >>
> >>>> >>
> >>>> >>
> >>>> >>> == 1. Make the checkstyle config a single point of truth for the
> >>>> >>> source code style. ==
> >>>> >>>
> >>>> >>> The checkstyle is already used and included in the Cassandra
> project
> >>>> >>> build lifecycle (ant command line, Jenkins, CircleCI). There is no
> >>>> >>> need to maintain code style configurations for different types of
> IDEs
> >>>> >>> (e.g. IntelliJ inspections configuration) since the checkstyle.xml
> >>>> >>> file can be directly imported to IDE used by a developer. This is
> fair
> >>>> >>> for Intellij Idea, NetBeans, and Eclipse.
> >>>> >>
> >>>> >>
> >>>> >> Big +1
> >>>> >>
> >>>> >>
> >>>> >>> So, I propose to focus on the checks themselves and checking pull
> >>>> >>> requests with automation scripts, rather than maintaining these
> >>>> >>> integrations. The benefits here are avoiding all issues with
> >>>> >>> maintaining configurations for different IDEs. Another good
> advantage
> >>>> >>> of this approach would be the ability to add new checkstyle rules
> >>>> >>> without touching IDE configuration - and such tickets will be LFH
> and
> >>>> >>> easy to commit.
> >>>> >>>
> >>>> >>> The actions points here are:
> >>>> >>>
> >>>> >>> - create an umbrella JIRA ticket for all checkstyle issues e.g.
> [8]
> >>>> >>> (or label checkstyle);
> >>>> >>> - add checkstyle to GitHub pull requests using GitHub actions
> (execute
> >>>> >>> ant command);
> >>>> >>
> >>>> >>
> >>>> >> Instead of custom GHA scripting, please use our existing
> >>>> >> cassandra-artifact.sh (which should already include all such
> checks).
> >>>> >>
> >>>> >> Something like
> https://github.com/apache/cassandra/compare/cassandra-3.11...thelastpickle:cassandra:mck/github-actions/3.11
> >>>> >>
> >>>> >>
> >>>> >>
> >>>> >>> == 3. Enable pushing backwards build results for both Jenkins and
> >>>> >>> CircleCI to GitHub pull requests. ==
> >>>> >>>
> >>>> >>> The goal here is to have a "green checkbox" for those GitHub pull
> >>>> >>> requests that have corresponding Jenkins/CircleCI runs. According
> to
> >>>> >>> the following links, it is completely possible to have those.
> >>>> >>>
> >>>> >>> https://github.com/jenkinsci/github-branch-source-plugin
> >>>> >>> https://circleci.com/docs/enable-checks/
> >>>> >>>
> >>>> >>> Moreover, the GitHub Branch Source Plugin is already enabled for
> the
> >>>> >>> Cassandra project [16]. The same seems should work the same way
> for
> >>>> >>> CirleCI, but I have faced the infrastructure team comment [17]
> that
> >>>> >>> describes admin permissions are required for that, so the
> question is
> >>>> >>> still open here. I could dig a bit more once we've agreed on it.
> >>>> >>>
> >>>> >>> The actions points here are:
> >>>> >>> - enable Jenkins integration for GitHub pull requests;
> >>>> >>> - enable CircleCI integration for GitHub pull requests;
> >>>> >>
> >>>> >>
> >>>> >> Some folk use CircleCI, some use ci-cassandra. The green checkbox
> idea
> >>>> >> is great, so long as it's optional. We don't want PRs triggering
> the
> >>>> >> runs either (there are other mechanisms for triggering for now).
> >>>> >>
> >>>> >>
> >>>> >>> The actions points here are:
> >>>> >>>
> >>>> >>> - initiate a wide survey for user and dev lists, to get to know
> about
> >>>> >>> the usages;
> >>>> >>> - remove those configurations that are not used anymore;
> >>>> >>> - force migration from Ant to Gradle/Maven;
> >>>> >>
> >>>> >>
> >>>> >> Let's leave this out for now. There's too many unknowns here. If
> >>>> >> there's an IDE configuration that's broken, no one has reported it
> for
> >>>> >> ages, and no one is around to fix it, then I say we should raise
> the
> >>>> >> discussion to remove it.
> >>>> >>
> >>>> >> The Gradle/Maven migration is a hot one, contribute to that
> discussion
> >>>> >> but let's not tangle this work up with it, IMHO.
> >>>> >>
> >>>> >> Totally agree that IDE project files should be as light weight as
> possible.
> >>>> >>
> >>>> >>
> >>>> >>> Summarizing everything proposed above I think it is possible to
> >>>> >>> simplify adding small contributions easier to the codebase as
> well as
> >>>> >>> save a bunch of committer's time.
> >>>> >>>
> >>>> >>> So,
> >>>> >>> WDYT about the things described above?
> >>>> >>> Should I create a CEP for this?
> >>>> >>
> >>>> >>
> >>>> >> I see no need for a CEP here. An epic and tickets will work.
> >>>> >> Again, thanks for the input Maxim!
> >>>>
>

Re: [DISCUSSION] Cassandra's code style and source code analysis

[Maxim Muzafarov <mm...@apache.org>]

Dear community,


I have created the epic with code-style activities to track the progress:
https://issues.apache.org/jira/browse/CASSANDRA-18090

In my understanding, there is no need to format whole the code base at
once according to the code style described on the page [1], and the
best strategy here is to go forward with small evolutionary changes.
Thus eventually we will come up with a set of rules convenient for all
members of the community. In my mind, having one commit per an added
code style rule should be easy to look at for a reviewer, the git
commits history as well as rebasing/merging other pull requests that
may be affected by the new rules.


I want to raise one more question related to class imports and the
classses import order for a wider discussion. The import order is well
described on the code style page [1], but using wildcard imports is
not mentioned at all. The wildcard imports with their drawbacks has
has already been raised in the JIRA issue [2] and didn't get enough
attention.

The checkstyle has the rules we are interested in for import control
and they must be considered together. We can implement them in a
single pull request or one by one, or use only the last one:
- AvoidStarImport
- CustomImportOrder

But still, I think that wildcard imports have more disadvantages
(class names conflicts e.g. java.util.*, java.sql.* or a new version
of a library has name clashes) than advantages and such problems will
be found in later CI cycles.
Currently, I've implemented the AvoidStarImport checkstyle rule in a
dedicated pull request [3][4], so you will be able to see all amount
of the changes with removing wildcard imports. The changes are made
for the checkstyle configuration as well as for code style
configurations for different IDEs we supported.

So, the open questions here are:

- Should the source code obey the AvoidStarImport rule [3]? (I think yes);
- Should we implement AvoidStarImport and CustomImportOrder in a
single pull request or do it one by one?


Anyway, I will fix the result of the agreement over the
AvoidStarImport rule on the documentation page [1].



[1] https://cassandra.apache.org/_/development/code_style.html
[2] https://issues.apache.org/jira/browse/CASSANDRA-17925
[3] https://issues.apache.org/jira/browse/CASSANDRA-18089
[4] https://github.com/apache/cassandra/pull/2041

On Thu, 1 Dec 2022 at 11:55, Claude Warren, Jr via dev
<de...@cassandra.apache.org> wrote:
>
> The last time I worked on a project that tried to implement a coding style across the project it was "an education".  The short story is that trying to "mitigate" the code base, with respect to style, is either a massive change or a long slow process.
>
> Arguments here have stated that earlier attempts to have the tooling reformat the code did not go well.  What we ended up doing was turned on the style checker and looked at the number of issues across the project.  When new code was accepted the number of issues could not rise.  Eventually most of the code was clean, with a few well coded legacy bits still not up to standard.  We could do something similar here.  Much like code coverage, you can't perform a merge unless the number of style errors remains the same or decreases.
>
> As with all software rules, this is a strong recommendation as I am certain that there are edge/corner case exceptions to be found.
>
>
>
>
> On Wed, Nov 30, 2022 at 3:30 PM Patrick McFadin <pm...@gmail.com> wrote:
>>
>> Why are we still debating build tooling? I think you’re wrong, but I’ve conceded - on the assumption that we can get enough volunteers willing to adopt responsibility for the new world order.
>>
>> Not debating. I am just throwing in my support since I have been in the Camp of Ant.
>>
>> On Wed, Nov 30, 2022 at 1:29 AM Benedict <be...@apache.org> wrote:
>>>
>>> Why are we still debating build tooling? I think you’re wrong, but I’ve conceded - on the assumption that we can get enough volunteers willing to adopt responsibility for the new world order.
>>>
>>> I suggest five long term contributors nominate themselves as the build file maintainers, and collectively manage a safe and painless migration for the rest of us - and agree to maintain and develop the new build file going forwards, and support the community as they adopt it.
>>>
>>> On the topic of over-exuberant linting I will continue to push back. I think linting our brace rules could make sense since they are atypical, but more formatting rules than this likely just leads to atrophying style. Authorship involves thinking about how to present your code; I don’t want to either encourage lazy authorship or prevent experimentation with presentation. Both would be bad, and I expect we would struggle to evolve our style guide again in future as the language evolves. Our brace rules are a good example everyone unilaterally ignored when lambdas arrived, as we all recognised they materially harmed the brevity benefits, and we eventually codified this.
>>>
>>> On migration: auto formatters applied to code that was not written with the rules in mind will almost unerringly be made a mess of, so having a tool do this is not acceptable IMO.
>>>
>>> The idea of checkstyle being the source of truth continues to be untenable and anyone still pushing for this should please engage with my earlier points on this.
>>>
>>>
>>> On 30 Nov 2022, at 04:06, Patrick McFadin <pm...@gmail.com> wrote:
>>>
>>> 
>>> I'm going to +1 what Stefan has said. I've heard on many occasions from newcomers to the project that having to use Ant is a deterrent. As a matter of fact, a few weeks ago, I spent a Sunday afternoon helping somebody trying to build Cassandra and Ant caused a ton of problems. "Ok. ant really super clean this time"
>>>
>>> Sure it still works for people that have been doing this for years. I drive a 20 year old Toyota truck, but I'm reminded by my kids often that it's not cool. So in that spirit, I feel my saying we need to keep Ant is like saying "You kids get off my lawn!" If it's something that will help attract new contributors, I'm all for it.
>>>
>>> Patrick
>>>
>>> On Fri, Nov 25, 2022 at 2:22 AM Miklosovic, Stefan <St...@netapp.com> wrote:
>>>>
>>>> I agree with what you wrote. How I understand it is that migrating to Maven/Gradle makes the project more "attractive" for newcomers. If a project is built on "that old un-cool Ant", it might be a little bit off-putting and questionable if we are "stuck in the past on build systems and not progressing".
>>>>
>>>> So in that sense I agree this is more "marketing" rather than technological question but on the other hand, does not Maven/Gradle allow us to modularize the project better? Maybe we would like to modularize but nobody is up to that because build system makes it impossible or at least quite inconvenient to do so. Do you really think there are not any significant benefits to switch even if it "just works" now?
>>>>
>>>> ________________________________________
>>>> From: Benedict <be...@apache.org>
>>>> Sent: Friday, November 25, 2022 11:07
>>>> To: dev@cassandra.apache.org
>>>> Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis
>>>>
>>>> NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.
>>>>
>>>>
>>>>
>>>>
>>>> There’s always a handful of people asking for it, but notably few if any of the full time contributors doing the majority of the core development of Cassandra. It strikes me as something very appealing to others, but less so to those wanting to get on with development.
>>>>
>>>> I never really see a good argument articulated for the migration, besides general hand waving that ant is old, and people like newer build systems. Ant is working fine, so there isn’t a strong technical reason to replace it, and there are good organisational reasons not to.
>>>>
>>>> Why do you consider a migration inevitable?
>>>>
>>>>
>>>>
>>>> > On 25 Nov 2022, at 09:58, Miklosovic, Stefan <St...@netapp.com> wrote:
>>>> >
>>>> > Interesting take on Ant / no-Ant, Benedict. I am very curious how this unfolds. My long-term perception is that changing it to something else is more or less inevitable but if there is a broader consensus to not do that .... well.
>>>> >
>>>> > ________________________________________
>>>> > From: Benedict <be...@apache.org>
>>>> > Sent: Friday, November 25, 2022 10:52
>>>> > To: dev@cassandra.apache.org
>>>> > Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis
>>>> >
>>>> > NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.
>>>> >
>>>> >
>>>> >
>>>> >
>>>> > I was in a bit of a rush last night. I should say that I’m of course +1 a general endeavour to clean this up, and to expand our use of linters, and I appreciate your volunteering to help out in this way Maxim.
>>>> >
>>>> > However, responding to Stefan, I’m pretty -1 migrating from ant to another build system without really good reason. Migration has a real cost to productivity for all existing contributors, and the phantom of increasing new contributions has never paid off historically. I’m all for easing people into participation, but not at penalty to the existing contributor base.
>>>> >
>>>> > If the only reason is to make it easier to open in a different IDE, we can perhaps have some basic build files outlining code structure for importing, that are compatible with our canonical ant build? We could perhaps even generate them.
>>>> >
>>>> >
>>>> >> On 25 Nov 2022, at 09:35, Miklosovic, Stefan <St...@netapp.com> wrote:
>>>> >>
>>>> >> For the record, I was testing that same combo Claude mentioned and it did not work out of the box but it is definitely possible to set up successfully. I do not remember the details.
>>>> >>
>>>> >> To replay to Maxim, it all seems good to me, roughly, but I humbly think it all boils down to Maven/Gradle refactoring and on top of that we can do all else.
>>>> >>
>>>> >> For example, there is (1) where the solution, besides fixing the tests, is to introduce an Ant task which would check this on build. That being said, how is that going to look like when we change Ant for something else? That stuff suddenly becomes obsolete.
>>>> >>
>>>> >> This case maybe applies to other problems we want to solve as well. I do not want to do something tailored for one build system just to rewrite it all or to spend significant amount of time on that again when we switch the build system.
>>>> >>
>>>> >> For that reason I think changing Ant for something else should be top priority (as I understand that it the hot topic for community for very long time) and then everything else should follow. We should spend time on things mentioned only in case they do not collide with any build system at all.
>>>> >>
>>>> >> (1) https://issues.apache.org/jira/browse/CASSANDRA-17964
>>>> >>
>>>> >> Stefan
>>>> >>
>>>> >> ________________________________________
>>>> >> From: Claude Warren, Jr via dev <de...@cassandra.apache.org>
>>>> >> Sent: Friday, November 25, 2022 10:16
>>>> >> To: dev@cassandra.apache.org
>>>> >> Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis
>>>> >>
>>>> >> NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.
>>>> >>
>>>> >>
>>>> >>
>>>> >> +1 for the concept as a whole.  I am certain I could find nits to pick if I looked deeply.
>>>> >>
>>>> >> @mck -- I did have a problem with Cassandra + Eclipse + Java11 (Classpath).  I gave up and am spending time trying to learn IntelliJ.  I also mentioned it in one of the discussion areas.
>>>> >>
>>>> >> Claude
>>>> >>
>>>> >> On Thu, Nov 24, 2022 at 8:55 PM Mick Semb Wever <mc...@apache.org>> wrote:
>>>> >> Thank you for a solid write up Maxim. And welcome to Cassandra, it's
>>>> >> very positive to see you here.
>>>> >>
>>>> >> I whole-heartedly agree with nearly everything you write. Some input
>>>> >> and questions inline.
>>>> >>
>>>> >>
>>>> >>>
>>>> >>> As you may know, the infrastructure team has disabled public sign-up
>>>> >>> to ASF JIRA (the GitHub issues are recommended instead).
>>>> >>>
>>>> >>
>>>> >>
>>>> >> I suspect (based on chatter in general, but not on dev@ AFAIK) is to
>>>> >> avoid GH issues and stick with jira. The sign-up hurdle we will
>>>> >> document on the website: CASSANDRA-18064
>>>> >>
>>>> >>
>>>> >>
>>>> >>> == 1. Make the checkstyle config a single point of truth for the
>>>> >>> source code style. ==
>>>> >>>
>>>> >>> The checkstyle is already used and included in the Cassandra project
>>>> >>> build lifecycle (ant command line, Jenkins, CircleCI). There is no
>>>> >>> need to maintain code style configurations for different types of IDEs
>>>> >>> (e.g. IntelliJ inspections configuration) since the checkstyle.xml
>>>> >>> file can be directly imported to IDE used by a developer. This is fair
>>>> >>> for Intellij Idea, NetBeans, and Eclipse.
>>>> >>
>>>> >>
>>>> >> Big +1
>>>> >>
>>>> >>
>>>> >>> So, I propose to focus on the checks themselves and checking pull
>>>> >>> requests with automation scripts, rather than maintaining these
>>>> >>> integrations. The benefits here are avoiding all issues with
>>>> >>> maintaining configurations for different IDEs. Another good advantage
>>>> >>> of this approach would be the ability to add new checkstyle rules
>>>> >>> without touching IDE configuration - and such tickets will be LFH and
>>>> >>> easy to commit.
>>>> >>>
>>>> >>> The actions points here are:
>>>> >>>
>>>> >>> - create an umbrella JIRA ticket for all checkstyle issues e.g. [8]
>>>> >>> (or label checkstyle);
>>>> >>> - add checkstyle to GitHub pull requests using GitHub actions (execute
>>>> >>> ant command);
>>>> >>
>>>> >>
>>>> >> Instead of custom GHA scripting, please use our existing
>>>> >> cassandra-artifact.sh (which should already include all such checks).
>>>> >>
>>>> >> Something like https://github.com/apache/cassandra/compare/cassandra-3.11...thelastpickle:cassandra:mck/github-actions/3.11
>>>> >>
>>>> >>
>>>> >>
>>>> >>> == 3. Enable pushing backwards build results for both Jenkins and
>>>> >>> CircleCI to GitHub pull requests. ==
>>>> >>>
>>>> >>> The goal here is to have a "green checkbox" for those GitHub pull
>>>> >>> requests that have corresponding Jenkins/CircleCI runs. According to
>>>> >>> the following links, it is completely possible to have those.
>>>> >>>
>>>> >>> https://github.com/jenkinsci/github-branch-source-plugin
>>>> >>> https://circleci.com/docs/enable-checks/
>>>> >>>
>>>> >>> Moreover, the GitHub Branch Source Plugin is already enabled for the
>>>> >>> Cassandra project [16]. The same seems should work the same way for
>>>> >>> CirleCI, but I have faced the infrastructure team comment [17] that
>>>> >>> describes admin permissions are required for that, so the question is
>>>> >>> still open here. I could dig a bit more once we've agreed on it.
>>>> >>>
>>>> >>> The actions points here are:
>>>> >>> - enable Jenkins integration for GitHub pull requests;
>>>> >>> - enable CircleCI integration for GitHub pull requests;
>>>> >>
>>>> >>
>>>> >> Some folk use CircleCI, some use ci-cassandra. The green checkbox idea
>>>> >> is great, so long as it's optional. We don't want PRs triggering the
>>>> >> runs either (there are other mechanisms for triggering for now).
>>>> >>
>>>> >>
>>>> >>> The actions points here are:
>>>> >>>
>>>> >>> - initiate a wide survey for user and dev lists, to get to know about
>>>> >>> the usages;
>>>> >>> - remove those configurations that are not used anymore;
>>>> >>> - force migration from Ant to Gradle/Maven;
>>>> >>
>>>> >>
>>>> >> Let's leave this out for now. There's too many unknowns here. If
>>>> >> there's an IDE configuration that's broken, no one has reported it for
>>>> >> ages, and no one is around to fix it, then I say we should raise the
>>>> >> discussion to remove it.
>>>> >>
>>>> >> The Gradle/Maven migration is a hot one, contribute to that discussion
>>>> >> but let's not tangle this work up with it, IMHO.
>>>> >>
>>>> >> Totally agree that IDE project files should be as light weight as possible.
>>>> >>
>>>> >>
>>>> >>> Summarizing everything proposed above I think it is possible to
>>>> >>> simplify adding small contributions easier to the codebase as well as
>>>> >>> save a bunch of committer's time.
>>>> >>>
>>>> >>> So,
>>>> >>> WDYT about the things described above?
>>>> >>> Should I create a CEP for this?
>>>> >>
>>>> >>
>>>> >> I see no need for a CEP here. An epic and tickets will work.
>>>> >> Again, thanks for the input Maxim!
>>>>

Re: [DISCUSSION] Cassandra's code style and source code analysis

["Claude Warren, Jr via dev" <de...@cassandra.apache.org>]

The last time I worked on a project that tried to implement a coding style
across the project it was "an education".  The short story is that trying
to "mitigate" the code base, with respect to style, is either a massive
change or a long slow process.

Arguments here have stated that earlier attempts to have the tooling
reformat the code did not go well.  What we ended up doing was turned on
the style checker and looked at the number of issues across the project.
When new code was accepted the number of issues could not rise.  Eventually
most of the code was clean, with a few well coded legacy bits still not up
to standard.  We could do something similar here.  Much like code coverage,
you can't perform a merge unless the number of style errors remains the
same or decreases.

As with all software rules, this is a strong recommendation as I am certain
that there are edge/corner case exceptions to be found.




On Wed, Nov 30, 2022 at 3:30 PM Patrick McFadin <pm...@gmail.com> wrote:

> Why are we still debating build tooling? I think you’re wrong, but I’ve
> conceded - on the assumption that we can get enough volunteers willing to
> adopt responsibility for the new world order.
>
> Not debating. I am just throwing in my support since I have been in the
> Camp of Ant.
>
> On Wed, Nov 30, 2022 at 1:29 AM Benedict <be...@apache.org> wrote:
>
>> Why are we still debating build tooling? I think you’re wrong, but I’ve
>> conceded - on the assumption that we can get enough volunteers willing to
>> adopt responsibility for the new world order.
>>
>> I suggest five long term contributors nominate themselves as the build
>> file maintainers, and collectively manage a safe and painless migration for
>> the rest of us - and agree to maintain and develop the new build file going
>> forwards, and support the community as they adopt it.
>>
>> On the topic of over-exuberant linting I will continue to push back. I
>> think linting our brace rules could make sense since they are atypical, but
>> more formatting rules than this likely just leads to atrophying style.
>> Authorship involves thinking about how to present your code; I don’t want
>> to either encourage lazy authorship or prevent experimentation with
>> presentation. Both would be bad, and I expect we would struggle to evolve
>> our style guide again in future as the language evolves. Our brace rules
>> are a good example everyone unilaterally ignored when lambdas arrived, as
>> we all recognised they materially harmed the brevity benefits, and we
>> eventually codified this.
>>
>> On migration: auto formatters applied to code that was not written with
>> the rules in mind will almost unerringly be made a mess of, so having a
>> tool do this is not acceptable IMO.
>>
>> The idea of checkstyle being the source of truth continues to be
>> untenable and anyone still pushing for this should please engage with my
>> earlier points on this.
>>
>>
>> On 30 Nov 2022, at 04:06, Patrick McFadin <pm...@gmail.com> wrote:
>>
>> 
>> I'm going to +1 what Stefan has said. I've heard on many occasions from
>> newcomers to the project that having to use Ant is a deterrent. As a matter
>> of fact, a few weeks ago, I spent a Sunday afternoon helping somebody
>> trying to build Cassandra and Ant caused a ton of problems. "Ok. ant really
>> super clean this time"
>>
>> Sure it still works for people that have been doing this for years. I
>> drive a 20 year old Toyota truck, but I'm reminded by my kids often that
>> it's not cool. So in that spirit, I feel my saying we need to keep Ant is
>> like saying "You kids get off my lawn!" If it's something that will help
>> attract new contributors, I'm all for it.
>>
>> Patrick
>>
>> On Fri, Nov 25, 2022 at 2:22 AM Miklosovic, Stefan <
>> Stefan.Miklosovic@netapp.com> wrote:
>>
>>> I agree with what you wrote. How I understand it is that migrating to
>>> Maven/Gradle makes the project more "attractive" for newcomers. If a
>>> project is built on "that old un-cool Ant", it might be a little bit
>>> off-putting and questionable if we are "stuck in the past on build systems
>>> and not progressing".
>>>
>>> So in that sense I agree this is more "marketing" rather than
>>> technological question but on the other hand, does not Maven/Gradle allow
>>> us to modularize the project better? Maybe we would like to modularize but
>>> nobody is up to that because build system makes it impossible or at least
>>> quite inconvenient to do so. Do you really think there are not any
>>> significant benefits to switch even if it "just works" now?
>>>
>>> ________________________________________
>>> From: Benedict <be...@apache.org>
>>> Sent: Friday, November 25, 2022 11:07
>>> To: dev@cassandra.apache.org
>>> Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis
>>>
>>> NetApp Security WARNING: This is an external email. Do not click links
>>> or open attachments unless you recognize the sender and know the content is
>>> safe.
>>>
>>>
>>>
>>>
>>> There’s always a handful of people asking for it, but notably few if any
>>> of the full time contributors doing the majority of the core development of
>>> Cassandra. It strikes me as something very appealing to others, but less so
>>> to those wanting to get on with development.
>>>
>>> I never really see a good argument articulated for the migration,
>>> besides general hand waving that ant is old, and people like newer build
>>> systems. Ant is working fine, so there isn’t a strong technical reason to
>>> replace it, and there are good organisational reasons not to.
>>>
>>> Why do you consider a migration inevitable?
>>>
>>>
>>>
>>> > On 25 Nov 2022, at 09:58, Miklosovic, Stefan <
>>> Stefan.Miklosovic@netapp.com> wrote:
>>> >
>>> > Interesting take on Ant / no-Ant, Benedict. I am very curious how
>>> this unfolds. My long-term perception is that changing it to something else
>>> is more or less inevitable but if there is a broader consensus to not do
>>> that .... well.
>>> >
>>> > ________________________________________
>>> > From: Benedict <be...@apache.org>
>>> > Sent: Friday, November 25, 2022 10:52
>>> > To: dev@cassandra.apache.org
>>> > Subject: Re: [DISCUSSION] Cassandra's code style and source code
>>> analysis
>>> >
>>> > NetApp Security WARNING: This is an external email. Do not click links
>>> or open attachments unless you recognize the sender and know the content is
>>> safe.
>>> >
>>> >
>>> >
>>> >
>>> > I was in a bit of a rush last night. I should say that I’m of course
>>> +1 a general endeavour to clean this up, and to expand our use of linters,
>>> and I appreciate your volunteering to help out in this way Maxim.
>>> >
>>> > However, responding to Stefan, I’m pretty -1 migrating from ant to
>>> another build system without really good reason. Migration has a real cost
>>> to productivity for all existing contributors, and the phantom of
>>> increasing new contributions has never paid off historically. I’m all for
>>> easing people into participation, but not at penalty to the existing
>>> contributor base.
>>> >
>>> > If the only reason is to make it easier to open in a different IDE, we
>>> can perhaps have some basic build files outlining code structure for
>>> importing, that are compatible with our canonical ant build? We could
>>> perhaps even generate them.
>>> >
>>> >
>>> >> On 25 Nov 2022, at 09:35, Miklosovic, Stefan <
>>> Stefan.Miklosovic@netapp.com> wrote:
>>> >>
>>> >> For the record, I was testing that same combo Claude mentioned and
>>> it did not work out of the box but it is definitely possible to set up
>>> successfully. I do not remember the details.
>>> >>
>>> >> To replay to Maxim, it all seems good to me, roughly, but I humbly
>>> think it all boils down to Maven/Gradle refactoring and on top of that we
>>> can do all else.
>>> >>
>>> >> For example, there is (1) where the solution, besides fixing the
>>> tests, is to introduce an Ant task which would check this on build. That
>>> being said, how is that going to look like when we change Ant for something
>>> else? That stuff suddenly becomes obsolete.
>>> >>
>>> >> This case maybe applies to other problems we want to solve as well. I
>>> do not want to do something tailored for one build system just to rewrite
>>> it all or to spend significant amount of time on that again when we switch
>>> the build system.
>>> >>
>>> >> For that reason I think changing Ant for something else should be top
>>> priority (as I understand that it the hot topic for community for very long
>>> time) and then everything else should follow. We should spend time on
>>> things mentioned only in case they do not collide with any build system at
>>> all.
>>> >>
>>> >> (1) https://issues.apache.org/jira/browse/CASSANDRA-17964
>>> >>
>>> >> Stefan
>>> >>
>>> >> ________________________________________
>>> >> From: Claude Warren, Jr via dev <de...@cassandra.apache.org>
>>> >> Sent: Friday, November 25, 2022 10:16
>>> >> To: dev@cassandra.apache.org
>>> >> Subject: Re: [DISCUSSION] Cassandra's code style and source code
>>> analysis
>>> >>
>>> >> NetApp Security WARNING: This is an external email. Do not click
>>> links or open attachments unless you recognize the sender and know the
>>> content is safe.
>>> >>
>>> >>
>>> >>
>>> >> +1 for the concept as a whole.  I am certain I could find nits to
>>> pick if I looked deeply.
>>> >>
>>> >> @mck -- I did have a problem with Cassandra + Eclipse + Java11
>>> (Classpath).  I gave up and am spending time trying to learn IntelliJ.  I
>>> also mentioned it in one of the discussion areas.
>>> >>
>>> >> Claude
>>> >>
>>> >> On Thu, Nov 24, 2022 at 8:55 PM Mick Semb Wever <mck@apache.org
>>> <ma...@apache.org>> wrote:
>>> >> Thank you for a solid write up Maxim. And welcome to Cassandra, it's
>>> >> very positive to see you here.
>>> >>
>>> >> I whole-heartedly agree with nearly everything you write. Some input
>>> >> and questions inline.
>>> >>
>>> >>
>>> >>>
>>> >>> As you may know, the infrastructure team has disabled public sign-up
>>> >>> to ASF JIRA (the GitHub issues are recommended instead).
>>> >>>
>>> >>
>>> >>
>>> >> I suspect (based on chatter in general, but not on dev@ AFAIK) is to
>>> >> avoid GH issues and stick with jira. The sign-up hurdle we will
>>> >> document on the website: CASSANDRA-18064
>>> >>
>>> >>
>>> >>
>>> >>> == 1. Make the checkstyle config a single point of truth for the
>>> >>> source code style. ==
>>> >>>
>>> >>> The checkstyle is already used and included in the Cassandra project
>>> >>> build lifecycle (ant command line, Jenkins, CircleCI). There is no
>>> >>> need to maintain code style configurations for different types of
>>> IDEs
>>> >>> (e.g. IntelliJ inspections configuration) since the checkstyle.xml
>>> >>> file can be directly imported to IDE used by a developer. This is
>>> fair
>>> >>> for Intellij Idea, NetBeans, and Eclipse.
>>> >>
>>> >>
>>> >> Big +1
>>> >>
>>> >>
>>> >>> So, I propose to focus on the checks themselves and checking pull
>>> >>> requests with automation scripts, rather than maintaining these
>>> >>> integrations. The benefits here are avoiding all issues with
>>> >>> maintaining configurations for different IDEs. Another good advantage
>>> >>> of this approach would be the ability to add new checkstyle rules
>>> >>> without touching IDE configuration - and such tickets will be LFH and
>>> >>> easy to commit.
>>> >>>
>>> >>> The actions points here are:
>>> >>>
>>> >>> - create an umbrella JIRA ticket for all checkstyle issues e.g. [8]
>>> >>> (or label checkstyle);
>>> >>> - add checkstyle to GitHub pull requests using GitHub actions
>>> (execute
>>> >>> ant command);
>>> >>
>>> >>
>>> >> Instead of custom GHA scripting, please use our existing
>>> >> cassandra-artifact.sh (which should already include all such checks).
>>> >>
>>> >> Something like
>>> https://github.com/apache/cassandra/compare/cassandra-3.11...thelastpickle:cassandra:mck/github-actions/3.11
>>> >>
>>> >>
>>> >>
>>> >>> == 3. Enable pushing backwards build results for both Jenkins and
>>> >>> CircleCI to GitHub pull requests. ==
>>> >>>
>>> >>> The goal here is to have a "green checkbox" for those GitHub pull
>>> >>> requests that have corresponding Jenkins/CircleCI runs. According to
>>> >>> the following links, it is completely possible to have those.
>>> >>>
>>> >>> https://github.com/jenkinsci/github-branch-source-plugin
>>> >>> https://circleci.com/docs/enable-checks/
>>> >>>
>>> >>> Moreover, the GitHub Branch Source Plugin is already enabled for the
>>> >>> Cassandra project [16]. The same seems should work the same way for
>>> >>> CirleCI, but I have faced the infrastructure team comment [17] that
>>> >>> describes admin permissions are required for that, so the question is
>>> >>> still open here. I could dig a bit more once we've agreed on it.
>>> >>>
>>> >>> The actions points here are:
>>> >>> - enable Jenkins integration for GitHub pull requests;
>>> >>> - enable CircleCI integration for GitHub pull requests;
>>> >>
>>> >>
>>> >> Some folk use CircleCI, some use ci-cassandra. The green checkbox idea
>>> >> is great, so long as it's optional. We don't want PRs triggering the
>>> >> runs either (there are other mechanisms for triggering for now).
>>> >>
>>> >>
>>> >>> The actions points here are:
>>> >>>
>>> >>> - initiate a wide survey for user and dev lists, to get to know about
>>> >>> the usages;
>>> >>> - remove those configurations that are not used anymore;
>>> >>> - force migration from Ant to Gradle/Maven;
>>> >>
>>> >>
>>> >> Let's leave this out for now. There's too many unknowns here. If
>>> >> there's an IDE configuration that's broken, no one has reported it for
>>> >> ages, and no one is around to fix it, then I say we should raise the
>>> >> discussion to remove it.
>>> >>
>>> >> The Gradle/Maven migration is a hot one, contribute to that discussion
>>> >> but let's not tangle this work up with it, IMHO.
>>> >>
>>> >> Totally agree that IDE project files should be as light weight as
>>> possible.
>>> >>
>>> >>
>>> >>> Summarizing everything proposed above I think it is possible to
>>> >>> simplify adding small contributions easier to the codebase as well as
>>> >>> save a bunch of committer's time.
>>> >>>
>>> >>> So,
>>> >>> WDYT about the things described above?
>>> >>> Should I create a CEP for this?
>>> >>
>>> >>
>>> >> I see no need for a CEP here. An epic and tickets will work.
>>> >> Again, thanks for the input Maxim!
>>>
>>>

Re: [DISCUSSION] Cassandra's code style and source code analysis

[Patrick McFadin <pm...@gmail.com>]

Why are we still debating build tooling? I think you’re wrong, but I’ve
conceded - on the assumption that we can get enough volunteers willing to
adopt responsibility for the new world order.

Not debating. I am just throwing in my support since I have been in the
Camp of Ant.

On Wed, Nov 30, 2022 at 1:29 AM Benedict <be...@apache.org> wrote:

> Why are we still debating build tooling? I think you’re wrong, but I’ve
> conceded - on the assumption that we can get enough volunteers willing to
> adopt responsibility for the new world order.
>
> I suggest five long term contributors nominate themselves as the build
> file maintainers, and collectively manage a safe and painless migration for
> the rest of us - and agree to maintain and develop the new build file going
> forwards, and support the community as they adopt it.
>
> On the topic of over-exuberant linting I will continue to push back. I
> think linting our brace rules could make sense since they are atypical, but
> more formatting rules than this likely just leads to atrophying style.
> Authorship involves thinking about how to present your code; I don’t want
> to either encourage lazy authorship or prevent experimentation with
> presentation. Both would be bad, and I expect we would struggle to evolve
> our style guide again in future as the language evolves. Our brace rules
> are a good example everyone unilaterally ignored when lambdas arrived, as
> we all recognised they materially harmed the brevity benefits, and we
> eventually codified this.
>
> On migration: auto formatters applied to code that was not written with
> the rules in mind will almost unerringly be made a mess of, so having a
> tool do this is not acceptable IMO.
>
> The idea of checkstyle being the source of truth continues to be untenable
> and anyone still pushing for this should please engage with my earlier
> points on this.
>
>
> On 30 Nov 2022, at 04:06, Patrick McFadin <pm...@gmail.com> wrote:
>
> 
> I'm going to +1 what Stefan has said. I've heard on many occasions from
> newcomers to the project that having to use Ant is a deterrent. As a matter
> of fact, a few weeks ago, I spent a Sunday afternoon helping somebody
> trying to build Cassandra and Ant caused a ton of problems. "Ok. ant really
> super clean this time"
>
> Sure it still works for people that have been doing this for years. I
> drive a 20 year old Toyota truck, but I'm reminded by my kids often that
> it's not cool. So in that spirit, I feel my saying we need to keep Ant is
> like saying "You kids get off my lawn!" If it's something that will help
> attract new contributors, I'm all for it.
>
> Patrick
>
> On Fri, Nov 25, 2022 at 2:22 AM Miklosovic, Stefan <
> Stefan.Miklosovic@netapp.com> wrote:
>
>> I agree with what you wrote. How I understand it is that migrating to
>> Maven/Gradle makes the project more "attractive" for newcomers. If a
>> project is built on "that old un-cool Ant", it might be a little bit
>> off-putting and questionable if we are "stuck in the past on build systems
>> and not progressing".
>>
>> So in that sense I agree this is more "marketing" rather than
>> technological question but on the other hand, does not Maven/Gradle allow
>> us to modularize the project better? Maybe we would like to modularize but
>> nobody is up to that because build system makes it impossible or at least
>> quite inconvenient to do so. Do you really think there are not any
>> significant benefits to switch even if it "just works" now?
>>
>> ________________________________________
>> From: Benedict <be...@apache.org>
>> Sent: Friday, November 25, 2022 11:07
>> To: dev@cassandra.apache.org
>> Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis
>>
>> NetApp Security WARNING: This is an external email. Do not click links or
>> open attachments unless you recognize the sender and know the content is
>> safe.
>>
>>
>>
>>
>> There’s always a handful of people asking for it, but notably few if any
>> of the full time contributors doing the majority of the core development of
>> Cassandra. It strikes me as something very appealing to others, but less so
>> to those wanting to get on with development.
>>
>> I never really see a good argument articulated for the migration, besides
>> general hand waving that ant is old, and people like newer build systems.
>> Ant is working fine, so there isn’t a strong technical reason to replace
>> it, and there are good organisational reasons not to.
>>
>> Why do you consider a migration inevitable?
>>
>>
>>
>> > On 25 Nov 2022, at 09:58, Miklosovic, Stefan <
>> Stefan.Miklosovic@netapp.com> wrote:
>> >
>> > Interesting take on Ant / no-Ant, Benedict. I am very curious how this
>> unfolds. My long-term perception is that changing it to something else is
>> more or less inevitable but if there is a broader consensus to not do that
>> .... well.
>> >
>> > ________________________________________
>> > From: Benedict <be...@apache.org>
>> > Sent: Friday, November 25, 2022 10:52
>> > To: dev@cassandra.apache.org
>> > Subject: Re: [DISCUSSION] Cassandra's code style and source code
>> analysis
>> >
>> > NetApp Security WARNING: This is an external email. Do not click links
>> or open attachments unless you recognize the sender and know the content is
>> safe.
>> >
>> >
>> >
>> >
>> > I was in a bit of a rush last night. I should say that I’m of course +1
>> a general endeavour to clean this up, and to expand our use of linters, and
>> I appreciate your volunteering to help out in this way Maxim.
>> >
>> > However, responding to Stefan, I’m pretty -1 migrating from ant to
>> another build system without really good reason. Migration has a real cost
>> to productivity for all existing contributors, and the phantom of
>> increasing new contributions has never paid off historically. I’m all for
>> easing people into participation, but not at penalty to the existing
>> contributor base.
>> >
>> > If the only reason is to make it easier to open in a different IDE, we
>> can perhaps have some basic build files outlining code structure for
>> importing, that are compatible with our canonical ant build? We could
>> perhaps even generate them.
>> >
>> >
>> >> On 25 Nov 2022, at 09:35, Miklosovic, Stefan <
>> Stefan.Miklosovic@netapp.com> wrote:
>> >>
>> >> For the record, I was testing that same combo Claude mentioned and it
>> did not work out of the box but it is definitely possible to set up
>> successfully. I do not remember the details.
>> >>
>> >> To replay to Maxim, it all seems good to me, roughly, but I humbly
>> think it all boils down to Maven/Gradle refactoring and on top of that we
>> can do all else.
>> >>
>> >> For example, there is (1) where the solution, besides fixing the
>> tests, is to introduce an Ant task which would check this on build. That
>> being said, how is that going to look like when we change Ant for something
>> else? That stuff suddenly becomes obsolete.
>> >>
>> >> This case maybe applies to other problems we want to solve as well. I
>> do not want to do something tailored for one build system just to rewrite
>> it all or to spend significant amount of time on that again when we switch
>> the build system.
>> >>
>> >> For that reason I think changing Ant for something else should be top
>> priority (as I understand that it the hot topic for community for very long
>> time) and then everything else should follow. We should spend time on
>> things mentioned only in case they do not collide with any build system at
>> all.
>> >>
>> >> (1) https://issues.apache.org/jira/browse/CASSANDRA-17964
>> >>
>> >> Stefan
>> >>
>> >> ________________________________________
>> >> From: Claude Warren, Jr via dev <de...@cassandra.apache.org>
>> >> Sent: Friday, November 25, 2022 10:16
>> >> To: dev@cassandra.apache.org
>> >> Subject: Re: [DISCUSSION] Cassandra's code style and source code
>> analysis
>> >>
>> >> NetApp Security WARNING: This is an external email. Do not click links
>> or open attachments unless you recognize the sender and know the content is
>> safe.
>> >>
>> >>
>> >>
>> >> +1 for the concept as a whole.  I am certain I could find nits to pick
>> if I looked deeply.
>> >>
>> >> @mck -- I did have a problem with Cassandra + Eclipse + Java11
>> (Classpath).  I gave up and am spending time trying to learn IntelliJ.  I
>> also mentioned it in one of the discussion areas.
>> >>
>> >> Claude
>> >>
>> >> On Thu, Nov 24, 2022 at 8:55 PM Mick Semb Wever <mck@apache.org
>> <ma...@apache.org>> wrote:
>> >> Thank you for a solid write up Maxim. And welcome to Cassandra, it's
>> >> very positive to see you here.
>> >>
>> >> I whole-heartedly agree with nearly everything you write. Some input
>> >> and questions inline.
>> >>
>> >>
>> >>>
>> >>> As you may know, the infrastructure team has disabled public sign-up
>> >>> to ASF JIRA (the GitHub issues are recommended instead).
>> >>>
>> >>
>> >>
>> >> I suspect (based on chatter in general, but not on dev@ AFAIK) is to
>> >> avoid GH issues and stick with jira. The sign-up hurdle we will
>> >> document on the website: CASSANDRA-18064
>> >>
>> >>
>> >>
>> >>> == 1. Make the checkstyle config a single point of truth for the
>> >>> source code style. ==
>> >>>
>> >>> The checkstyle is already used and included in the Cassandra project
>> >>> build lifecycle (ant command line, Jenkins, CircleCI). There is no
>> >>> need to maintain code style configurations for different types of IDEs
>> >>> (e.g. IntelliJ inspections configuration) since the checkstyle.xml
>> >>> file can be directly imported to IDE used by a developer. This is fair
>> >>> for Intellij Idea, NetBeans, and Eclipse.
>> >>
>> >>
>> >> Big +1
>> >>
>> >>
>> >>> So, I propose to focus on the checks themselves and checking pull
>> >>> requests with automation scripts, rather than maintaining these
>> >>> integrations. The benefits here are avoiding all issues with
>> >>> maintaining configurations for different IDEs. Another good advantage
>> >>> of this approach would be the ability to add new checkstyle rules
>> >>> without touching IDE configuration - and such tickets will be LFH and
>> >>> easy to commit.
>> >>>
>> >>> The actions points here are:
>> >>>
>> >>> - create an umbrella JIRA ticket for all checkstyle issues e.g. [8]
>> >>> (or label checkstyle);
>> >>> - add checkstyle to GitHub pull requests using GitHub actions (execute
>> >>> ant command);
>> >>
>> >>
>> >> Instead of custom GHA scripting, please use our existing
>> >> cassandra-artifact.sh (which should already include all such checks).
>> >>
>> >> Something like
>> https://github.com/apache/cassandra/compare/cassandra-3.11...thelastpickle:cassandra:mck/github-actions/3.11
>> >>
>> >>
>> >>
>> >>> == 3. Enable pushing backwards build results for both Jenkins and
>> >>> CircleCI to GitHub pull requests. ==
>> >>>
>> >>> The goal here is to have a "green checkbox" for those GitHub pull
>> >>> requests that have corresponding Jenkins/CircleCI runs. According to
>> >>> the following links, it is completely possible to have those.
>> >>>
>> >>> https://github.com/jenkinsci/github-branch-source-plugin
>> >>> https://circleci.com/docs/enable-checks/
>> >>>
>> >>> Moreover, the GitHub Branch Source Plugin is already enabled for the
>> >>> Cassandra project [16]. The same seems should work the same way for
>> >>> CirleCI, but I have faced the infrastructure team comment [17] that
>> >>> describes admin permissions are required for that, so the question is
>> >>> still open here. I could dig a bit more once we've agreed on it.
>> >>>
>> >>> The actions points here are:
>> >>> - enable Jenkins integration for GitHub pull requests;
>> >>> - enable CircleCI integration for GitHub pull requests;
>> >>
>> >>
>> >> Some folk use CircleCI, some use ci-cassandra. The green checkbox idea
>> >> is great, so long as it's optional. We don't want PRs triggering the
>> >> runs either (there are other mechanisms for triggering for now).
>> >>
>> >>
>> >>> The actions points here are:
>> >>>
>> >>> - initiate a wide survey for user and dev lists, to get to know about
>> >>> the usages;
>> >>> - remove those configurations that are not used anymore;
>> >>> - force migration from Ant to Gradle/Maven;
>> >>
>> >>
>> >> Let's leave this out for now. There's too many unknowns here. If
>> >> there's an IDE configuration that's broken, no one has reported it for
>> >> ages, and no one is around to fix it, then I say we should raise the
>> >> discussion to remove it.
>> >>
>> >> The Gradle/Maven migration is a hot one, contribute to that discussion
>> >> but let's not tangle this work up with it, IMHO.
>> >>
>> >> Totally agree that IDE project files should be as light weight as
>> possible.
>> >>
>> >>
>> >>> Summarizing everything proposed above I think it is possible to
>> >>> simplify adding small contributions easier to the codebase as well as
>> >>> save a bunch of committer's time.
>> >>>
>> >>> So,
>> >>> WDYT about the things described above?
>> >>> Should I create a CEP for this?
>> >>
>> >>
>> >> I see no need for a CEP here. An epic and tickets will work.
>> >> Again, thanks for the input Maxim!
>>
>>

Re: [DISCUSSION] Cassandra's code style and source code analysis

[Benedict <be...@apache.org>]

Why are we still debating build tooling? I think you’re wrong, but I’ve conceded - on the assumption that we can get enough volunteers willing to adopt responsibility for the new world order.

I suggest five long term contributors nominate themselves as the build file maintainers, and collectively manage a safe and painless migration for the rest of us - and agree to maintain and develop the new build file going forwards, and support the community as they adopt it.

On the topic of over-exuberant linting I will continue to push back. I think linting our brace rules could make sense since they are atypical, but more formatting rules than this likely just leads to atrophying style. Authorship involves thinking about how to present your code; I don’t want to either encourage lazy authorship or prevent experimentation with presentation. Both would be bad, and I expect we would struggle to evolve our style guide again in future as the language evolves. Our brace rules are a good example everyone unilaterally ignored when lambdas arrived, as we all recognised they materially harmed the brevity benefits, and we eventually codified this.

On migration: auto formatters applied to code that was not written with the rules in mind will almost unerringly be made a mess of, so having a tool do this is not acceptable IMO.

The idea of checkstyle being the source of truth continues to be untenable and anyone still pushing for this should please engage with my earlier points on this.


> On 30 Nov 2022, at 04:06, Patrick McFadin <pm...@gmail.com> wrote:
> 
> 
> I'm going to +1 what Stefan has said. I've heard on many occasions from newcomers to the project that having to use Ant is a deterrent. As a matter of fact, a few weeks ago, I spent a Sunday afternoon helping somebody trying to build Cassandra and Ant caused a ton of problems. "Ok. ant really super clean this time"
> 
> Sure it still works for people that have been doing this for years. I drive a 20 year old Toyota truck, but I'm reminded by my kids often that it's not cool. So in that spirit, I feel my saying we need to keep Ant is like saying "You kids get off my lawn!" If it's something that will help attract new contributors, I'm all for it. 
> 
> Patrick
> 
>> On Fri, Nov 25, 2022 at 2:22 AM Miklosovic, Stefan <St...@netapp.com> wrote:
>> I agree with what you wrote. How I understand it is that migrating to Maven/Gradle makes the project more "attractive" for newcomers. If a project is built on "that old un-cool Ant", it might be a little bit off-putting and questionable if we are "stuck in the past on build systems and not progressing".
>> 
>> So in that sense I agree this is more "marketing" rather than technological question but on the other hand, does not Maven/Gradle allow us to modularize the project better? Maybe we would like to modularize but nobody is up to that because build system makes it impossible or at least quite inconvenient to do so. Do you really think there are not any significant benefits to switch even if it "just works" now?
>> 
>> ________________________________________
>> From: Benedict <be...@apache.org>
>> Sent: Friday, November 25, 2022 11:07
>> To: dev@cassandra.apache.org
>> Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis
>> 
>> NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.
>> 
>> 
>> 
>> 
>> There’s always a handful of people asking for it, but notably few if any of the full time contributors doing the majority of the core development of Cassandra. It strikes me as something very appealing to others, but less so to those wanting to get on with development.
>> 
>> I never really see a good argument articulated for the migration, besides general hand waving that ant is old, and people like newer build systems. Ant is working fine, so there isn’t a strong technical reason to replace it, and there are good organisational reasons not to.
>> 
>> Why do you consider a migration inevitable?
>> 
>> 
>> 
>> > On 25 Nov 2022, at 09:58, Miklosovic, Stefan <St...@netapp.com> wrote:
>> >
>> > Interesting take on Ant / no-Ant, Benedict. I am very curious how this unfolds. My long-term perception is that changing it to something else is more or less inevitable but if there is a broader consensus to not do that .... well.
>> >
>> > ________________________________________
>> > From: Benedict <be...@apache.org>
>> > Sent: Friday, November 25, 2022 10:52
>> > To: dev@cassandra.apache.org
>> > Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis
>> >
>> > NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.
>> >
>> >
>> >
>> >
>> > I was in a bit of a rush last night. I should say that I’m of course +1 a general endeavour to clean this up, and to expand our use of linters, and I appreciate your volunteering to help out in this way Maxim.
>> >
>> > However, responding to Stefan, I’m pretty -1 migrating from ant to another build system without really good reason. Migration has a real cost to productivity for all existing contributors, and the phantom of increasing new contributions has never paid off historically. I’m all for easing people into participation, but not at penalty to the existing contributor base.
>> >
>> > If the only reason is to make it easier to open in a different IDE, we can perhaps have some basic build files outlining code structure for importing, that are compatible with our canonical ant build? We could perhaps even generate them.
>> >
>> >
>> >> On 25 Nov 2022, at 09:35, Miklosovic, Stefan <St...@netapp.com> wrote:
>> >>
>> >> For the record, I was testing that same combo Claude mentioned and it did not work out of the box but it is definitely possible to set up successfully. I do not remember the details.
>> >>
>> >> To replay to Maxim, it all seems good to me, roughly, but I humbly think it all boils down to Maven/Gradle refactoring and on top of that we can do all else.
>> >>
>> >> For example, there is (1) where the solution, besides fixing the tests, is to introduce an Ant task which would check this on build. That being said, how is that going to look like when we change Ant for something else? That stuff suddenly becomes obsolete.
>> >>
>> >> This case maybe applies to other problems we want to solve as well. I do not want to do something tailored for one build system just to rewrite it all or to spend significant amount of time on that again when we switch the build system.
>> >>
>> >> For that reason I think changing Ant for something else should be top priority (as I understand that it the hot topic for community for very long time) and then everything else should follow. We should spend time on things mentioned only in case they do not collide with any build system at all.
>> >>
>> >> (1) https://issues.apache.org/jira/browse/CASSANDRA-17964
>> >>
>> >> Stefan
>> >>
>> >> ________________________________________
>> >> From: Claude Warren, Jr via dev <de...@cassandra.apache.org>
>> >> Sent: Friday, November 25, 2022 10:16
>> >> To: dev@cassandra.apache.org
>> >> Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis
>> >>
>> >> NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.
>> >>
>> >>
>> >>
>> >> +1 for the concept as a whole.  I am certain I could find nits to pick if I looked deeply.
>> >>
>> >> @mck -- I did have a problem with Cassandra + Eclipse + Java11 (Classpath).  I gave up and am spending time trying to learn IntelliJ.  I also mentioned it in one of the discussion areas.
>> >>
>> >> Claude
>> >>
>> >> On Thu, Nov 24, 2022 at 8:55 PM Mick Semb Wever <mc...@apache.org>> wrote:
>> >> Thank you for a solid write up Maxim. And welcome to Cassandra, it's
>> >> very positive to see you here.
>> >>
>> >> I whole-heartedly agree with nearly everything you write. Some input
>> >> and questions inline.
>> >>
>> >>
>> >>>
>> >>> As you may know, the infrastructure team has disabled public sign-up
>> >>> to ASF JIRA (the GitHub issues are recommended instead).
>> >>>
>> >>
>> >>
>> >> I suspect (based on chatter in general, but not on dev@ AFAIK) is to
>> >> avoid GH issues and stick with jira. The sign-up hurdle we will
>> >> document on the website: CASSANDRA-18064
>> >>
>> >>
>> >>
>> >>> == 1. Make the checkstyle config a single point of truth for the
>> >>> source code style. ==
>> >>>
>> >>> The checkstyle is already used and included in the Cassandra project
>> >>> build lifecycle (ant command line, Jenkins, CircleCI). There is no
>> >>> need to maintain code style configurations for different types of IDEs
>> >>> (e.g. IntelliJ inspections configuration) since the checkstyle.xml
>> >>> file can be directly imported to IDE used by a developer. This is fair
>> >>> for Intellij Idea, NetBeans, and Eclipse.
>> >>
>> >>
>> >> Big +1
>> >>
>> >>
>> >>> So, I propose to focus on the checks themselves and checking pull
>> >>> requests with automation scripts, rather than maintaining these
>> >>> integrations. The benefits here are avoiding all issues with
>> >>> maintaining configurations for different IDEs. Another good advantage
>> >>> of this approach would be the ability to add new checkstyle rules
>> >>> without touching IDE configuration - and such tickets will be LFH and
>> >>> easy to commit.
>> >>>
>> >>> The actions points here are:
>> >>>
>> >>> - create an umbrella JIRA ticket for all checkstyle issues e.g. [8]
>> >>> (or label checkstyle);
>> >>> - add checkstyle to GitHub pull requests using GitHub actions (execute
>> >>> ant command);
>> >>
>> >>
>> >> Instead of custom GHA scripting, please use our existing
>> >> cassandra-artifact.sh (which should already include all such checks).
>> >>
>> >> Something like https://github.com/apache/cassandra/compare/cassandra-3.11...thelastpickle:cassandra:mck/github-actions/3.11
>> >>
>> >>
>> >>
>> >>> == 3. Enable pushing backwards build results for both Jenkins and
>> >>> CircleCI to GitHub pull requests. ==
>> >>>
>> >>> The goal here is to have a "green checkbox" for those GitHub pull
>> >>> requests that have corresponding Jenkins/CircleCI runs. According to
>> >>> the following links, it is completely possible to have those.
>> >>>
>> >>> https://github.com/jenkinsci/github-branch-source-plugin
>> >>> https://circleci.com/docs/enable-checks/
>> >>>
>> >>> Moreover, the GitHub Branch Source Plugin is already enabled for the
>> >>> Cassandra project [16]. The same seems should work the same way for
>> >>> CirleCI, but I have faced the infrastructure team comment [17] that
>> >>> describes admin permissions are required for that, so the question is
>> >>> still open here. I could dig a bit more once we've agreed on it.
>> >>>
>> >>> The actions points here are:
>> >>> - enable Jenkins integration for GitHub pull requests;
>> >>> - enable CircleCI integration for GitHub pull requests;
>> >>
>> >>
>> >> Some folk use CircleCI, some use ci-cassandra. The green checkbox idea
>> >> is great, so long as it's optional. We don't want PRs triggering the
>> >> runs either (there are other mechanisms for triggering for now).
>> >>
>> >>
>> >>> The actions points here are:
>> >>>
>> >>> - initiate a wide survey for user and dev lists, to get to know about
>> >>> the usages;
>> >>> - remove those configurations that are not used anymore;
>> >>> - force migration from Ant to Gradle/Maven;
>> >>
>> >>
>> >> Let's leave this out for now. There's too many unknowns here. If
>> >> there's an IDE configuration that's broken, no one has reported it for
>> >> ages, and no one is around to fix it, then I say we should raise the
>> >> discussion to remove it.
>> >>
>> >> The Gradle/Maven migration is a hot one, contribute to that discussion
>> >> but let's not tangle this work up with it, IMHO.
>> >>
>> >> Totally agree that IDE project files should be as light weight as possible.
>> >>
>> >>
>> >>> Summarizing everything proposed above I think it is possible to
>> >>> simplify adding small contributions easier to the codebase as well as
>> >>> save a bunch of committer's time.
>> >>>
>> >>> So,
>> >>> WDYT about the things described above?
>> >>> Should I create a CEP for this?
>> >>
>> >>
>> >> I see no need for a CEP here. An epic and tickets will work.
>> >> Again, thanks for the input Maxim!
>> 

Re: [DISCUSSION] Cassandra's code style and source code analysis

[Patrick McFadin <pm...@gmail.com>]

I'm going to +1 what Stefan has said. I've heard on many occasions from
newcomers to the project that having to use Ant is a deterrent. As a matter
of fact, a few weeks ago, I spent a Sunday afternoon helping somebody
trying to build Cassandra and Ant caused a ton of problems. "Ok. ant really
super clean this time"

Sure it still works for people that have been doing this for years. I drive
a 20 year old Toyota truck, but I'm reminded by my kids often that it's not
cool. So in that spirit, I feel my saying we need to keep Ant is like
saying "You kids get off my lawn!" If it's something that will help attract
new contributors, I'm all for it.

Patrick

On Fri, Nov 25, 2022 at 2:22 AM Miklosovic, Stefan <
Stefan.Miklosovic@netapp.com> wrote:

> I agree with what you wrote. How I understand it is that migrating to
> Maven/Gradle makes the project more "attractive" for newcomers. If a
> project is built on "that old un-cool Ant", it might be a little bit
> off-putting and questionable if we are "stuck in the past on build systems
> and not progressing".
>
> So in that sense I agree this is more "marketing" rather than
> technological question but on the other hand, does not Maven/Gradle allow
> us to modularize the project better? Maybe we would like to modularize but
> nobody is up to that because build system makes it impossible or at least
> quite inconvenient to do so. Do you really think there are not any
> significant benefits to switch even if it "just works" now?
>
> ________________________________________
> From: Benedict <be...@apache.org>
> Sent: Friday, November 25, 2022 11:07
> To: dev@cassandra.apache.org
> Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis
>
> NetApp Security WARNING: This is an external email. Do not click links or
> open attachments unless you recognize the sender and know the content is
> safe.
>
>
>
>
> There’s always a handful of people asking for it, but notably few if any
> of the full time contributors doing the majority of the core development of
> Cassandra. It strikes me as something very appealing to others, but less so
> to those wanting to get on with development.
>
> I never really see a good argument articulated for the migration, besides
> general hand waving that ant is old, and people like newer build systems.
> Ant is working fine, so there isn’t a strong technical reason to replace
> it, and there are good organisational reasons not to.
>
> Why do you consider a migration inevitable?
>
>
>
> > On 25 Nov 2022, at 09:58, Miklosovic, Stefan <
> Stefan.Miklosovic@netapp.com> wrote:
> >
> > Interesting take on Ant / no-Ant, Benedict. I am very curious how this
> unfolds. My long-term perception is that changing it to something else is
> more or less inevitable but if there is a broader consensus to not do that
> .... well.
> >
> > ________________________________________
> > From: Benedict <be...@apache.org>
> > Sent: Friday, November 25, 2022 10:52
> > To: dev@cassandra.apache.org
> > Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis
> >
> > NetApp Security WARNING: This is an external email. Do not click links
> or open attachments unless you recognize the sender and know the content is
> safe.
> >
> >
> >
> >
> > I was in a bit of a rush last night. I should say that I’m of course +1
> a general endeavour to clean this up, and to expand our use of linters, and
> I appreciate your volunteering to help out in this way Maxim.
> >
> > However, responding to Stefan, I’m pretty -1 migrating from ant to
> another build system without really good reason. Migration has a real cost
> to productivity for all existing contributors, and the phantom of
> increasing new contributions has never paid off historically. I’m all for
> easing people into participation, but not at penalty to the existing
> contributor base.
> >
> > If the only reason is to make it easier to open in a different IDE, we
> can perhaps have some basic build files outlining code structure for
> importing, that are compatible with our canonical ant build? We could
> perhaps even generate them.
> >
> >
> >> On 25 Nov 2022, at 09:35, Miklosovic, Stefan <
> Stefan.Miklosovic@netapp.com> wrote:
> >>
> >> For the record, I was testing that same combo Claude mentioned and it
> did not work out of the box but it is definitely possible to set up
> successfully. I do not remember the details.
> >>
> >> To replay to Maxim, it all seems good to me, roughly, but I humbly
> think it all boils down to Maven/Gradle refactoring and on top of that we
> can do all else.
> >>
> >> For example, there is (1) where the solution, besides fixing the tests,
> is to introduce an Ant task which would check this on build. That being
> said, how is that going to look like when we change Ant for something else?
> That stuff suddenly becomes obsolete.
> >>
> >> This case maybe applies to other problems we want to solve as well. I
> do not want to do something tailored for one build system just to rewrite
> it all or to spend significant amount of time on that again when we switch
> the build system.
> >>
> >> For that reason I think changing Ant for something else should be top
> priority (as I understand that it the hot topic for community for very long
> time) and then everything else should follow. We should spend time on
> things mentioned only in case they do not collide with any build system at
> all.
> >>
> >> (1) https://issues.apache.org/jira/browse/CASSANDRA-17964
> >>
> >> Stefan
> >>
> >> ________________________________________
> >> From: Claude Warren, Jr via dev <de...@cassandra.apache.org>
> >> Sent: Friday, November 25, 2022 10:16
> >> To: dev@cassandra.apache.org
> >> Subject: Re: [DISCUSSION] Cassandra's code style and source code
> analysis
> >>
> >> NetApp Security WARNING: This is an external email. Do not click links
> or open attachments unless you recognize the sender and know the content is
> safe.
> >>
> >>
> >>
> >> +1 for the concept as a whole.  I am certain I could find nits to pick
> if I looked deeply.
> >>
> >> @mck -- I did have a problem with Cassandra + Eclipse + Java11
> (Classpath).  I gave up and am spending time trying to learn IntelliJ.  I
> also mentioned it in one of the discussion areas.
> >>
> >> Claude
> >>
> >> On Thu, Nov 24, 2022 at 8:55 PM Mick Semb Wever <mck@apache.org<mailto:
> mck@apache.org>> wrote:
> >> Thank you for a solid write up Maxim. And welcome to Cassandra, it's
> >> very positive to see you here.
> >>
> >> I whole-heartedly agree with nearly everything you write. Some input
> >> and questions inline.
> >>
> >>
> >>>
> >>> As you may know, the infrastructure team has disabled public sign-up
> >>> to ASF JIRA (the GitHub issues are recommended instead).
> >>>
> >>
> >>
> >> I suspect (based on chatter in general, but not on dev@ AFAIK) is to
> >> avoid GH issues and stick with jira. The sign-up hurdle we will
> >> document on the website: CASSANDRA-18064
> >>
> >>
> >>
> >>> == 1. Make the checkstyle config a single point of truth for the
> >>> source code style. ==
> >>>
> >>> The checkstyle is already used and included in the Cassandra project
> >>> build lifecycle (ant command line, Jenkins, CircleCI). There is no
> >>> need to maintain code style configurations for different types of IDEs
> >>> (e.g. IntelliJ inspections configuration) since the checkstyle.xml
> >>> file can be directly imported to IDE used by a developer. This is fair
> >>> for Intellij Idea, NetBeans, and Eclipse.
> >>
> >>
> >> Big +1
> >>
> >>
> >>> So, I propose to focus on the checks themselves and checking pull
> >>> requests with automation scripts, rather than maintaining these
> >>> integrations. The benefits here are avoiding all issues with
> >>> maintaining configurations for different IDEs. Another good advantage
> >>> of this approach would be the ability to add new checkstyle rules
> >>> without touching IDE configuration - and such tickets will be LFH and
> >>> easy to commit.
> >>>
> >>> The actions points here are:
> >>>
> >>> - create an umbrella JIRA ticket for all checkstyle issues e.g. [8]
> >>> (or label checkstyle);
> >>> - add checkstyle to GitHub pull requests using GitHub actions (execute
> >>> ant command);
> >>
> >>
> >> Instead of custom GHA scripting, please use our existing
> >> cassandra-artifact.sh (which should already include all such checks).
> >>
> >> Something like
> https://github.com/apache/cassandra/compare/cassandra-3.11...thelastpickle:cassandra:mck/github-actions/3.11
> >>
> >>
> >>
> >>> == 3. Enable pushing backwards build results for both Jenkins and
> >>> CircleCI to GitHub pull requests. ==
> >>>
> >>> The goal here is to have a "green checkbox" for those GitHub pull
> >>> requests that have corresponding Jenkins/CircleCI runs. According to
> >>> the following links, it is completely possible to have those.
> >>>
> >>> https://github.com/jenkinsci/github-branch-source-plugin
> >>> https://circleci.com/docs/enable-checks/
> >>>
> >>> Moreover, the GitHub Branch Source Plugin is already enabled for the
> >>> Cassandra project [16]. The same seems should work the same way for
> >>> CirleCI, but I have faced the infrastructure team comment [17] that
> >>> describes admin permissions are required for that, so the question is
> >>> still open here. I could dig a bit more once we've agreed on it.
> >>>
> >>> The actions points here are:
> >>> - enable Jenkins integration for GitHub pull requests;
> >>> - enable CircleCI integration for GitHub pull requests;
> >>
> >>
> >> Some folk use CircleCI, some use ci-cassandra. The green checkbox idea
> >> is great, so long as it's optional. We don't want PRs triggering the
> >> runs either (there are other mechanisms for triggering for now).
> >>
> >>
> >>> The actions points here are:
> >>>
> >>> - initiate a wide survey for user and dev lists, to get to know about
> >>> the usages;
> >>> - remove those configurations that are not used anymore;
> >>> - force migration from Ant to Gradle/Maven;
> >>
> >>
> >> Let's leave this out for now. There's too many unknowns here. If
> >> there's an IDE configuration that's broken, no one has reported it for
> >> ages, and no one is around to fix it, then I say we should raise the
> >> discussion to remove it.
> >>
> >> The Gradle/Maven migration is a hot one, contribute to that discussion
> >> but let's not tangle this work up with it, IMHO.
> >>
> >> Totally agree that IDE project files should be as light weight as
> possible.
> >>
> >>
> >>> Summarizing everything proposed above I think it is possible to
> >>> simplify adding small contributions easier to the codebase as well as
> >>> save a bunch of committer's time.
> >>>
> >>> So,
> >>> WDYT about the things described above?
> >>> Should I create a CEP for this?
> >>
> >>
> >> I see no need for a CEP here. An epic and tickets will work.
> >> Again, thanks for the input Maxim!
>
>

Re: [DISCUSSION] Cassandra's code style and source code analysis

["Miklosovic, Stefan" <St...@netapp.com>]

I agree with what you wrote. How I understand it is that migrating to Maven/Gradle makes the project more "attractive" for newcomers. If a project is built on "that old un-cool Ant", it might be a little bit off-putting and questionable if we are "stuck in the past on build systems and not progressing".

So in that sense I agree this is more "marketing" rather than technological question but on the other hand, does not Maven/Gradle allow us to modularize the project better? Maybe we would like to modularize but nobody is up to that because build system makes it impossible or at least quite inconvenient to do so. Do you really think there are not any significant benefits to switch even if it "just works" now?

________________________________________
From: Benedict <be...@apache.org>
Sent: Friday, November 25, 2022 11:07
To: dev@cassandra.apache.org
Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis

NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.




There’s always a handful of people asking for it, but notably few if any of the full time contributors doing the majority of the core development of Cassandra. It strikes me as something very appealing to others, but less so to those wanting to get on with development.

I never really see a good argument articulated for the migration, besides general hand waving that ant is old, and people like newer build systems. Ant is working fine, so there isn’t a strong technical reason to replace it, and there are good organisational reasons not to.

Why do you consider a migration inevitable?



> On 25 Nov 2022, at 09:58, Miklosovic, Stefan <St...@netapp.com> wrote:
>
> Interesting take on Ant / no-Ant, Benedict. I am very curious how this unfolds. My long-term perception is that changing it to something else is more or less inevitable but if there is a broader consensus to not do that .... well.
>
> ________________________________________
> From: Benedict <be...@apache.org>
> Sent: Friday, November 25, 2022 10:52
> To: dev@cassandra.apache.org
> Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis
>
> NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.
>
>
>
>
> I was in a bit of a rush last night. I should say that I’m of course +1 a general endeavour to clean this up, and to expand our use of linters, and I appreciate your volunteering to help out in this way Maxim.
>
> However, responding to Stefan, I’m pretty -1 migrating from ant to another build system without really good reason. Migration has a real cost to productivity for all existing contributors, and the phantom of increasing new contributions has never paid off historically. I’m all for easing people into participation, but not at penalty to the existing contributor base.
>
> If the only reason is to make it easier to open in a different IDE, we can perhaps have some basic build files outlining code structure for importing, that are compatible with our canonical ant build? We could perhaps even generate them.
>
>
>> On 25 Nov 2022, at 09:35, Miklosovic, Stefan <St...@netapp.com> wrote:
>>
>> For the record, I was testing that same combo Claude mentioned and it did not work out of the box but it is definitely possible to set up successfully. I do not remember the details.
>>
>> To replay to Maxim, it all seems good to me, roughly, but I humbly think it all boils down to Maven/Gradle refactoring and on top of that we can do all else.
>>
>> For example, there is (1) where the solution, besides fixing the tests, is to introduce an Ant task which would check this on build. That being said, how is that going to look like when we change Ant for something else? That stuff suddenly becomes obsolete.
>>
>> This case maybe applies to other problems we want to solve as well. I do not want to do something tailored for one build system just to rewrite it all or to spend significant amount of time on that again when we switch the build system.
>>
>> For that reason I think changing Ant for something else should be top priority (as I understand that it the hot topic for community for very long time) and then everything else should follow. We should spend time on things mentioned only in case they do not collide with any build system at all.
>>
>> (1) https://issues.apache.org/jira/browse/CASSANDRA-17964
>>
>> Stefan
>>
>> ________________________________________
>> From: Claude Warren, Jr via dev <de...@cassandra.apache.org>
>> Sent: Friday, November 25, 2022 10:16
>> To: dev@cassandra.apache.org
>> Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis
>>
>> NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.
>>
>>
>>
>> +1 for the concept as a whole.  I am certain I could find nits to pick if I looked deeply.
>>
>> @mck -- I did have a problem with Cassandra + Eclipse + Java11 (Classpath).  I gave up and am spending time trying to learn IntelliJ.  I also mentioned it in one of the discussion areas.
>>
>> Claude
>>
>> On Thu, Nov 24, 2022 at 8:55 PM Mick Semb Wever <mc...@apache.org>> wrote:
>> Thank you for a solid write up Maxim. And welcome to Cassandra, it's
>> very positive to see you here.
>>
>> I whole-heartedly agree with nearly everything you write. Some input
>> and questions inline.
>>
>>
>>>
>>> As you may know, the infrastructure team has disabled public sign-up
>>> to ASF JIRA (the GitHub issues are recommended instead).
>>>
>>
>>
>> I suspect (based on chatter in general, but not on dev@ AFAIK) is to
>> avoid GH issues and stick with jira. The sign-up hurdle we will
>> document on the website: CASSANDRA-18064
>>
>>
>>
>>> == 1. Make the checkstyle config a single point of truth for the
>>> source code style. ==
>>>
>>> The checkstyle is already used and included in the Cassandra project
>>> build lifecycle (ant command line, Jenkins, CircleCI). There is no
>>> need to maintain code style configurations for different types of IDEs
>>> (e.g. IntelliJ inspections configuration) since the checkstyle.xml
>>> file can be directly imported to IDE used by a developer. This is fair
>>> for Intellij Idea, NetBeans, and Eclipse.
>>
>>
>> Big +1
>>
>>
>>> So, I propose to focus on the checks themselves and checking pull
>>> requests with automation scripts, rather than maintaining these
>>> integrations. The benefits here are avoiding all issues with
>>> maintaining configurations for different IDEs. Another good advantage
>>> of this approach would be the ability to add new checkstyle rules
>>> without touching IDE configuration - and such tickets will be LFH and
>>> easy to commit.
>>>
>>> The actions points here are:
>>>
>>> - create an umbrella JIRA ticket for all checkstyle issues e.g. [8]
>>> (or label checkstyle);
>>> - add checkstyle to GitHub pull requests using GitHub actions (execute
>>> ant command);
>>
>>
>> Instead of custom GHA scripting, please use our existing
>> cassandra-artifact.sh (which should already include all such checks).
>>
>> Something like https://github.com/apache/cassandra/compare/cassandra-3.11...thelastpickle:cassandra:mck/github-actions/3.11
>>
>>
>>
>>> == 3. Enable pushing backwards build results for both Jenkins and
>>> CircleCI to GitHub pull requests. ==
>>>
>>> The goal here is to have a "green checkbox" for those GitHub pull
>>> requests that have corresponding Jenkins/CircleCI runs. According to
>>> the following links, it is completely possible to have those.
>>>
>>> https://github.com/jenkinsci/github-branch-source-plugin
>>> https://circleci.com/docs/enable-checks/
>>>
>>> Moreover, the GitHub Branch Source Plugin is already enabled for the
>>> Cassandra project [16]. The same seems should work the same way for
>>> CirleCI, but I have faced the infrastructure team comment [17] that
>>> describes admin permissions are required for that, so the question is
>>> still open here. I could dig a bit more once we've agreed on it.
>>>
>>> The actions points here are:
>>> - enable Jenkins integration for GitHub pull requests;
>>> - enable CircleCI integration for GitHub pull requests;
>>
>>
>> Some folk use CircleCI, some use ci-cassandra. The green checkbox idea
>> is great, so long as it's optional. We don't want PRs triggering the
>> runs either (there are other mechanisms for triggering for now).
>>
>>
>>> The actions points here are:
>>>
>>> - initiate a wide survey for user and dev lists, to get to know about
>>> the usages;
>>> - remove those configurations that are not used anymore;
>>> - force migration from Ant to Gradle/Maven;
>>
>>
>> Let's leave this out for now. There's too many unknowns here. If
>> there's an IDE configuration that's broken, no one has reported it for
>> ages, and no one is around to fix it, then I say we should raise the
>> discussion to remove it.
>>
>> The Gradle/Maven migration is a hot one, contribute to that discussion
>> but let's not tangle this work up with it, IMHO.
>>
>> Totally agree that IDE project files should be as light weight as possible.
>>
>>
>>> Summarizing everything proposed above I think it is possible to
>>> simplify adding small contributions easier to the codebase as well as
>>> save a bunch of committer's time.
>>>
>>> So,
>>> WDYT about the things described above?
>>> Should I create a CEP for this?
>>
>>
>> I see no need for a CEP here. An epic and tickets will work.
>> Again, thanks for the input Maxim!

Re: [DISCUSSION] Cassandra's code style and source code analysis

[Benedict <be...@apache.org>]

There’s always a handful of people asking for it, but notably few if any of the full time contributors doing the majority of the core development of Cassandra. It strikes me as something very appealing to others, but less so to those wanting to get on with development.

I never really see a good argument articulated for the migration, besides general hand waving that ant is old, and people like newer build systems. Ant is working fine, so there isn’t a strong technical reason to replace it, and there are good organisational reasons not to.

Why do you consider a migration inevitable?



> On 25 Nov 2022, at 09:58, Miklosovic, Stefan <St...@netapp.com> wrote:
> 
> Interesting take on Ant / no-Ant, Benedict. I am very curious how this unfolds. My long-term perception is that changing it to something else is more or less inevitable but if there is a broader consensus to not do that .... well.
> 
> ________________________________________
> From: Benedict <be...@apache.org>
> Sent: Friday, November 25, 2022 10:52
> To: dev@cassandra.apache.org
> Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis
> 
> NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.
> 
> 
> 
> 
> I was in a bit of a rush last night. I should say that I’m of course +1 a general endeavour to clean this up, and to expand our use of linters, and I appreciate your volunteering to help out in this way Maxim.
> 
> However, responding to Stefan, I’m pretty -1 migrating from ant to another build system without really good reason. Migration has a real cost to productivity for all existing contributors, and the phantom of increasing new contributions has never paid off historically. I’m all for easing people into participation, but not at penalty to the existing contributor base.
> 
> If the only reason is to make it easier to open in a different IDE, we can perhaps have some basic build files outlining code structure for importing, that are compatible with our canonical ant build? We could perhaps even generate them.
> 
> 
>> On 25 Nov 2022, at 09:35, Miklosovic, Stefan <St...@netapp.com> wrote:
>> 
>> For the record, I was testing that same combo Claude mentioned and it did not work out of the box but it is definitely possible to set up successfully. I do not remember the details.
>> 
>> To replay to Maxim, it all seems good to me, roughly, but I humbly think it all boils down to Maven/Gradle refactoring and on top of that we can do all else.
>> 
>> For example, there is (1) where the solution, besides fixing the tests, is to introduce an Ant task which would check this on build. That being said, how is that going to look like when we change Ant for something else? That stuff suddenly becomes obsolete.
>> 
>> This case maybe applies to other problems we want to solve as well. I do not want to do something tailored for one build system just to rewrite it all or to spend significant amount of time on that again when we switch the build system.
>> 
>> For that reason I think changing Ant for something else should be top priority (as I understand that it the hot topic for community for very long time) and then everything else should follow. We should spend time on things mentioned only in case they do not collide with any build system at all.
>> 
>> (1) https://issues.apache.org/jira/browse/CASSANDRA-17964
>> 
>> Stefan
>> 
>> ________________________________________
>> From: Claude Warren, Jr via dev <de...@cassandra.apache.org>
>> Sent: Friday, November 25, 2022 10:16
>> To: dev@cassandra.apache.org
>> Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis
>> 
>> NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.
>> 
>> 
>> 
>> +1 for the concept as a whole.  I am certain I could find nits to pick if I looked deeply.
>> 
>> @mck -- I did have a problem with Cassandra + Eclipse + Java11 (Classpath).  I gave up and am spending time trying to learn IntelliJ.  I also mentioned it in one of the discussion areas.
>> 
>> Claude
>> 
>> On Thu, Nov 24, 2022 at 8:55 PM Mick Semb Wever <mc...@apache.org>> wrote:
>> Thank you for a solid write up Maxim. And welcome to Cassandra, it's
>> very positive to see you here.
>> 
>> I whole-heartedly agree with nearly everything you write. Some input
>> and questions inline.
>> 
>> 
>>> 
>>> As you may know, the infrastructure team has disabled public sign-up
>>> to ASF JIRA (the GitHub issues are recommended instead).
>>> 
>> 
>> 
>> I suspect (based on chatter in general, but not on dev@ AFAIK) is to
>> avoid GH issues and stick with jira. The sign-up hurdle we will
>> document on the website: CASSANDRA-18064
>> 
>> 
>> 
>>> == 1. Make the checkstyle config a single point of truth for the
>>> source code style. ==
>>> 
>>> The checkstyle is already used and included in the Cassandra project
>>> build lifecycle (ant command line, Jenkins, CircleCI). There is no
>>> need to maintain code style configurations for different types of IDEs
>>> (e.g. IntelliJ inspections configuration) since the checkstyle.xml
>>> file can be directly imported to IDE used by a developer. This is fair
>>> for Intellij Idea, NetBeans, and Eclipse.
>> 
>> 
>> Big +1
>> 
>> 
>>> So, I propose to focus on the checks themselves and checking pull
>>> requests with automation scripts, rather than maintaining these
>>> integrations. The benefits here are avoiding all issues with
>>> maintaining configurations for different IDEs. Another good advantage
>>> of this approach would be the ability to add new checkstyle rules
>>> without touching IDE configuration - and such tickets will be LFH and
>>> easy to commit.
>>> 
>>> The actions points here are:
>>> 
>>> - create an umbrella JIRA ticket for all checkstyle issues e.g. [8]
>>> (or label checkstyle);
>>> - add checkstyle to GitHub pull requests using GitHub actions (execute
>>> ant command);
>> 
>> 
>> Instead of custom GHA scripting, please use our existing
>> cassandra-artifact.sh (which should already include all such checks).
>> 
>> Something like https://github.com/apache/cassandra/compare/cassandra-3.11...thelastpickle:cassandra:mck/github-actions/3.11
>> 
>> 
>> 
>>> == 3. Enable pushing backwards build results for both Jenkins and
>>> CircleCI to GitHub pull requests. ==
>>> 
>>> The goal here is to have a "green checkbox" for those GitHub pull
>>> requests that have corresponding Jenkins/CircleCI runs. According to
>>> the following links, it is completely possible to have those.
>>> 
>>> https://github.com/jenkinsci/github-branch-source-plugin
>>> https://circleci.com/docs/enable-checks/
>>> 
>>> Moreover, the GitHub Branch Source Plugin is already enabled for the
>>> Cassandra project [16]. The same seems should work the same way for
>>> CirleCI, but I have faced the infrastructure team comment [17] that
>>> describes admin permissions are required for that, so the question is
>>> still open here. I could dig a bit more once we've agreed on it.
>>> 
>>> The actions points here are:
>>> - enable Jenkins integration for GitHub pull requests;
>>> - enable CircleCI integration for GitHub pull requests;
>> 
>> 
>> Some folk use CircleCI, some use ci-cassandra. The green checkbox idea
>> is great, so long as it's optional. We don't want PRs triggering the
>> runs either (there are other mechanisms for triggering for now).
>> 
>> 
>>> The actions points here are:
>>> 
>>> - initiate a wide survey for user and dev lists, to get to know about
>>> the usages;
>>> - remove those configurations that are not used anymore;
>>> - force migration from Ant to Gradle/Maven;
>> 
>> 
>> Let's leave this out for now. There's too many unknowns here. If
>> there's an IDE configuration that's broken, no one has reported it for
>> ages, and no one is around to fix it, then I say we should raise the
>> discussion to remove it.
>> 
>> The Gradle/Maven migration is a hot one, contribute to that discussion
>> but let's not tangle this work up with it, IMHO.
>> 
>> Totally agree that IDE project files should be as light weight as possible.
>> 
>> 
>>> Summarizing everything proposed above I think it is possible to
>>> simplify adding small contributions easier to the codebase as well as
>>> save a bunch of committer's time.
>>> 
>>> So,
>>> WDYT about the things described above?
>>> Should I create a CEP for this?
>> 
>> 
>> I see no need for a CEP here. An epic and tickets will work.
>> Again, thanks for the input Maxim!

Re: [DISCUSSION] Cassandra's code style and source code analysis

["Miklosovic, Stefan" <St...@netapp.com>]

Interesting take on Ant / no-Ant, Benedict. I am very curious how this unfolds. My long-term perception is that changing it to something else is more or less inevitable but if there is a broader consensus to not do that .... well.

________________________________________
From: Benedict <be...@apache.org>
Sent: Friday, November 25, 2022 10:52
To: dev@cassandra.apache.org
Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis

NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.




I was in a bit of a rush last night. I should say that I’m of course +1 a general endeavour to clean this up, and to expand our use of linters, and I appreciate your volunteering to help out in this way Maxim.

However, responding to Stefan, I’m pretty -1 migrating from ant to another build system without really good reason. Migration has a real cost to productivity for all existing contributors, and the phantom of increasing new contributions has never paid off historically. I’m all for easing people into participation, but not at penalty to the existing contributor base.

If the only reason is to make it easier to open in a different IDE, we can perhaps have some basic build files outlining code structure for importing, that are compatible with our canonical ant build? We could perhaps even generate them.


> On 25 Nov 2022, at 09:35, Miklosovic, Stefan <St...@netapp.com> wrote:
>
> For the record, I was testing that same combo Claude mentioned and it did not work out of the box but it is definitely possible to set up successfully. I do not remember the details.
>
> To replay to Maxim, it all seems good to me, roughly, but I humbly think it all boils down to Maven/Gradle refactoring and on top of that we can do all else.
>
> For example, there is (1) where the solution, besides fixing the tests, is to introduce an Ant task which would check this on build. That being said, how is that going to look like when we change Ant for something else? That stuff suddenly becomes obsolete.
>
> This case maybe applies to other problems we want to solve as well. I do not want to do something tailored for one build system just to rewrite it all or to spend significant amount of time on that again when we switch the build system.
>
> For that reason I think changing Ant for something else should be top priority (as I understand that it the hot topic for community for very long time) and then everything else should follow. We should spend time on things mentioned only in case they do not collide with any build system at all.
>
> (1) https://issues.apache.org/jira/browse/CASSANDRA-17964
>
> Stefan
>
> ________________________________________
> From: Claude Warren, Jr via dev <de...@cassandra.apache.org>
> Sent: Friday, November 25, 2022 10:16
> To: dev@cassandra.apache.org
> Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis
>
> NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.
>
>
>
> +1 for the concept as a whole.  I am certain I could find nits to pick if I looked deeply.
>
> @mck -- I did have a problem with Cassandra + Eclipse + Java11 (Classpath).  I gave up and am spending time trying to learn IntelliJ.  I also mentioned it in one of the discussion areas.
>
> Claude
>
> On Thu, Nov 24, 2022 at 8:55 PM Mick Semb Wever <mc...@apache.org>> wrote:
> Thank you for a solid write up Maxim. And welcome to Cassandra, it's
> very positive to see you here.
>
> I whole-heartedly agree with nearly everything you write. Some input
> and questions inline.
>
>
>>
>> As you may know, the infrastructure team has disabled public sign-up
>> to ASF JIRA (the GitHub issues are recommended instead).
>>
>
>
> I suspect (based on chatter in general, but not on dev@ AFAIK) is to
> avoid GH issues and stick with jira. The sign-up hurdle we will
> document on the website: CASSANDRA-18064
>
>
>
>> == 1. Make the checkstyle config a single point of truth for the
>> source code style. ==
>>
>> The checkstyle is already used and included in the Cassandra project
>> build lifecycle (ant command line, Jenkins, CircleCI). There is no
>> need to maintain code style configurations for different types of IDEs
>> (e.g. IntelliJ inspections configuration) since the checkstyle.xml
>> file can be directly imported to IDE used by a developer. This is fair
>> for Intellij Idea, NetBeans, and Eclipse.
>
>
> Big +1
>
>
>> So, I propose to focus on the checks themselves and checking pull
>> requests with automation scripts, rather than maintaining these
>> integrations. The benefits here are avoiding all issues with
>> maintaining configurations for different IDEs. Another good advantage
>> of this approach would be the ability to add new checkstyle rules
>> without touching IDE configuration - and such tickets will be LFH and
>> easy to commit.
>>
>> The actions points here are:
>>
>> - create an umbrella JIRA ticket for all checkstyle issues e.g. [8]
>> (or label checkstyle);
>> - add checkstyle to GitHub pull requests using GitHub actions (execute
>> ant command);
>
>
> Instead of custom GHA scripting, please use our existing
> cassandra-artifact.sh (which should already include all such checks).
>
> Something like https://github.com/apache/cassandra/compare/cassandra-3.11...thelastpickle:cassandra:mck/github-actions/3.11
>
>
>
>> == 3. Enable pushing backwards build results for both Jenkins and
>> CircleCI to GitHub pull requests. ==
>>
>> The goal here is to have a "green checkbox" for those GitHub pull
>> requests that have corresponding Jenkins/CircleCI runs. According to
>> the following links, it is completely possible to have those.
>>
>> https://github.com/jenkinsci/github-branch-source-plugin
>> https://circleci.com/docs/enable-checks/
>>
>> Moreover, the GitHub Branch Source Plugin is already enabled for the
>> Cassandra project [16]. The same seems should work the same way for
>> CirleCI, but I have faced the infrastructure team comment [17] that
>> describes admin permissions are required for that, so the question is
>> still open here. I could dig a bit more once we've agreed on it.
>>
>> The actions points here are:
>> - enable Jenkins integration for GitHub pull requests;
>> - enable CircleCI integration for GitHub pull requests;
>
>
> Some folk use CircleCI, some use ci-cassandra. The green checkbox idea
> is great, so long as it's optional. We don't want PRs triggering the
> runs either (there are other mechanisms for triggering for now).
>
>
>> The actions points here are:
>>
>> - initiate a wide survey for user and dev lists, to get to know about
>> the usages;
>> - remove those configurations that are not used anymore;
>> - force migration from Ant to Gradle/Maven;
>
>
> Let's leave this out for now. There's too many unknowns here. If
> there's an IDE configuration that's broken, no one has reported it for
> ages, and no one is around to fix it, then I say we should raise the
> discussion to remove it.
>
> The Gradle/Maven migration is a hot one, contribute to that discussion
> but let's not tangle this work up with it, IMHO.
>
> Totally agree that IDE project files should be as light weight as possible.
>
>
>> Summarizing everything proposed above I think it is possible to
>> simplify adding small contributions easier to the codebase as well as
>> save a bunch of committer's time.
>>
>> So,
>> WDYT about the things described above?
>> Should I create a CEP for this?
>
>
> I see no need for a CEP here. An epic and tickets will work.
> Again, thanks for the input Maxim!

Re: [DISCUSSION] Cassandra's code style and source code analysis

[Benedict <be...@apache.org>]

I was in a bit of a rush last night. I should say that I’m of course +1 a general endeavour to clean this up, and to expand our use of linters, and I appreciate your volunteering to help out in this way Maxim.

However, responding to Stefan, I’m pretty -1 migrating from ant to another build system without really good reason. Migration has a real cost to productivity for all existing contributors, and the phantom of increasing new contributions has never paid off historically. I’m all for easing people into participation, but not at penalty to the existing contributor base.

If the only reason is to make it easier to open in a different IDE, we can perhaps have some basic build files outlining code structure for importing, that are compatible with our canonical ant build? We could perhaps even generate them.


> On 25 Nov 2022, at 09:35, Miklosovic, Stefan <St...@netapp.com> wrote:
> 
> For the record, I was testing that same combo Claude mentioned and it did not work out of the box but it is definitely possible to set up successfully. I do not remember the details.
> 
> To replay to Maxim, it all seems good to me, roughly, but I humbly think it all boils down to Maven/Gradle refactoring and on top of that we can do all else.
> 
> For example, there is (1) where the solution, besides fixing the tests, is to introduce an Ant task which would check this on build. That being said, how is that going to look like when we change Ant for something else? That stuff suddenly becomes obsolete.
> 
> This case maybe applies to other problems we want to solve as well. I do not want to do something tailored for one build system just to rewrite it all or to spend significant amount of time on that again when we switch the build system.
> 
> For that reason I think changing Ant for something else should be top priority (as I understand that it the hot topic for community for very long time) and then everything else should follow. We should spend time on things mentioned only in case they do not collide with any build system at all.
> 
> (1) https://issues.apache.org/jira/browse/CASSANDRA-17964
> 
> Stefan
> 
> ________________________________________
> From: Claude Warren, Jr via dev <de...@cassandra.apache.org>
> Sent: Friday, November 25, 2022 10:16
> To: dev@cassandra.apache.org
> Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis
> 
> NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.
> 
> 
> 
> +1 for the concept as a whole.  I am certain I could find nits to pick if I looked deeply.
> 
> @mck -- I did have a problem with Cassandra + Eclipse + Java11 (Classpath).  I gave up and am spending time trying to learn IntelliJ.  I also mentioned it in one of the discussion areas.
> 
> Claude
> 
> On Thu, Nov 24, 2022 at 8:55 PM Mick Semb Wever <mc...@apache.org>> wrote:
> Thank you for a solid write up Maxim. And welcome to Cassandra, it's
> very positive to see you here.
> 
> I whole-heartedly agree with nearly everything you write. Some input
> and questions inline.
> 
> 
>> 
>> As you may know, the infrastructure team has disabled public sign-up
>> to ASF JIRA (the GitHub issues are recommended instead).
>> 
> 
> 
> I suspect (based on chatter in general, but not on dev@ AFAIK) is to
> avoid GH issues and stick with jira. The sign-up hurdle we will
> document on the website: CASSANDRA-18064
> 
> 
> 
>> == 1. Make the checkstyle config a single point of truth for the
>> source code style. ==
>> 
>> The checkstyle is already used and included in the Cassandra project
>> build lifecycle (ant command line, Jenkins, CircleCI). There is no
>> need to maintain code style configurations for different types of IDEs
>> (e.g. IntelliJ inspections configuration) since the checkstyle.xml
>> file can be directly imported to IDE used by a developer. This is fair
>> for Intellij Idea, NetBeans, and Eclipse.
> 
> 
> Big +1
> 
> 
>> So, I propose to focus on the checks themselves and checking pull
>> requests with automation scripts, rather than maintaining these
>> integrations. The benefits here are avoiding all issues with
>> maintaining configurations for different IDEs. Another good advantage
>> of this approach would be the ability to add new checkstyle rules
>> without touching IDE configuration - and such tickets will be LFH and
>> easy to commit.
>> 
>> The actions points here are:
>> 
>> - create an umbrella JIRA ticket for all checkstyle issues e.g. [8]
>> (or label checkstyle);
>> - add checkstyle to GitHub pull requests using GitHub actions (execute
>> ant command);
> 
> 
> Instead of custom GHA scripting, please use our existing
> cassandra-artifact.sh (which should already include all such checks).
> 
> Something like https://github.com/apache/cassandra/compare/cassandra-3.11...thelastpickle:cassandra:mck/github-actions/3.11
> 
> 
> 
>> == 3. Enable pushing backwards build results for both Jenkins and
>> CircleCI to GitHub pull requests. ==
>> 
>> The goal here is to have a "green checkbox" for those GitHub pull
>> requests that have corresponding Jenkins/CircleCI runs. According to
>> the following links, it is completely possible to have those.
>> 
>> https://github.com/jenkinsci/github-branch-source-plugin
>> https://circleci.com/docs/enable-checks/
>> 
>> Moreover, the GitHub Branch Source Plugin is already enabled for the
>> Cassandra project [16]. The same seems should work the same way for
>> CirleCI, but I have faced the infrastructure team comment [17] that
>> describes admin permissions are required for that, so the question is
>> still open here. I could dig a bit more once we've agreed on it.
>> 
>> The actions points here are:
>> - enable Jenkins integration for GitHub pull requests;
>> - enable CircleCI integration for GitHub pull requests;
> 
> 
> Some folk use CircleCI, some use ci-cassandra. The green checkbox idea
> is great, so long as it's optional. We don't want PRs triggering the
> runs either (there are other mechanisms for triggering for now).
> 
> 
>> The actions points here are:
>> 
>> - initiate a wide survey for user and dev lists, to get to know about
>> the usages;
>> - remove those configurations that are not used anymore;
>> - force migration from Ant to Gradle/Maven;
> 
> 
> Let's leave this out for now. There's too many unknowns here. If
> there's an IDE configuration that's broken, no one has reported it for
> ages, and no one is around to fix it, then I say we should raise the
> discussion to remove it.
> 
> The Gradle/Maven migration is a hot one, contribute to that discussion
> but let's not tangle this work up with it, IMHO.
> 
> Totally agree that IDE project files should be as light weight as possible.
> 
> 
>> Summarizing everything proposed above I think it is possible to
>> simplify adding small contributions easier to the codebase as well as
>> save a bunch of committer's time.
>> 
>> So,
>> WDYT about the things described above?
>> Should I create a CEP for this?
> 
> 
> I see no need for a CEP here. An epic and tickets will work.
> Again, thanks for the input Maxim!

Re: [DISCUSSION] Cassandra's code style and source code analysis

["Miklosovic, Stefan" <St...@netapp.com>]

For the record, I was testing that same combo Claude mentioned and it did not work out of the box but it is definitely possible to set up successfully. I do not remember the details.

To replay to Maxim, it all seems good to me, roughly, but I humbly think it all boils down to Maven/Gradle refactoring and on top of that we can do all else.

For example, there is (1) where the solution, besides fixing the tests, is to introduce an Ant task which would check this on build. That being said, how is that going to look like when we change Ant for something else? That stuff suddenly becomes obsolete.

This case maybe applies to other problems we want to solve as well. I do not want to do something tailored for one build system just to rewrite it all or to spend significant amount of time on that again when we switch the build system.

For that reason I think changing Ant for something else should be top priority (as I understand that it the hot topic for community for very long time) and then everything else should follow. We should spend time on things mentioned only in case they do not collide with any build system at all.

(1) https://issues.apache.org/jira/browse/CASSANDRA-17964

Stefan

________________________________________
From: Claude Warren, Jr via dev <de...@cassandra.apache.org>
Sent: Friday, November 25, 2022 10:16
To: dev@cassandra.apache.org
Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis

NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.



+1 for the concept as a whole.  I am certain I could find nits to pick if I looked deeply.

@mck -- I did have a problem with Cassandra + Eclipse + Java11 (Classpath).  I gave up and am spending time trying to learn IntelliJ.  I also mentioned it in one of the discussion areas.

Claude

On Thu, Nov 24, 2022 at 8:55 PM Mick Semb Wever <mc...@apache.org>> wrote:
 Thank you for a solid write up Maxim. And welcome to Cassandra, it's
very positive to see you here.

I whole-heartedly agree with nearly everything you write. Some input
and questions inline.


>
> As you may know, the infrastructure team has disabled public sign-up
> to ASF JIRA (the GitHub issues are recommended instead).
>


I suspect (based on chatter in general, but not on dev@ AFAIK) is to
avoid GH issues and stick with jira. The sign-up hurdle we will
document on the website: CASSANDRA-18064



> == 1. Make the checkstyle config a single point of truth for the
> source code style. ==
>
> The checkstyle is already used and included in the Cassandra project
> build lifecycle (ant command line, Jenkins, CircleCI). There is no
> need to maintain code style configurations for different types of IDEs
> (e.g. IntelliJ inspections configuration) since the checkstyle.xml
> file can be directly imported to IDE used by a developer. This is fair
> for Intellij Idea, NetBeans, and Eclipse.


Big +1


> So, I propose to focus on the checks themselves and checking pull
> requests with automation scripts, rather than maintaining these
> integrations. The benefits here are avoiding all issues with
> maintaining configurations for different IDEs. Another good advantage
> of this approach would be the ability to add new checkstyle rules
> without touching IDE configuration - and such tickets will be LFH and
> easy to commit.
>
> The actions points here are:
>
> - create an umbrella JIRA ticket for all checkstyle issues e.g. [8]
> (or label checkstyle);
> - add checkstyle to GitHub pull requests using GitHub actions (execute
> ant command);


Instead of custom GHA scripting, please use our existing
cassandra-artifact.sh (which should already include all such checks).

Something like https://github.com/apache/cassandra/compare/cassandra-3.11...thelastpickle:cassandra:mck/github-actions/3.11



> == 3. Enable pushing backwards build results for both Jenkins and
> CircleCI to GitHub pull requests. ==
>
> The goal here is to have a "green checkbox" for those GitHub pull
> requests that have corresponding Jenkins/CircleCI runs. According to
> the following links, it is completely possible to have those.
>
> https://github.com/jenkinsci/github-branch-source-plugin
> https://circleci.com/docs/enable-checks/
>
> Moreover, the GitHub Branch Source Plugin is already enabled for the
> Cassandra project [16]. The same seems should work the same way for
> CirleCI, but I have faced the infrastructure team comment [17] that
> describes admin permissions are required for that, so the question is
> still open here. I could dig a bit more once we've agreed on it.
>
> The actions points here are:
> - enable Jenkins integration for GitHub pull requests;
> - enable CircleCI integration for GitHub pull requests;


Some folk use CircleCI, some use ci-cassandra. The green checkbox idea
is great, so long as it's optional. We don't want PRs triggering the
runs either (there are other mechanisms for triggering for now).


> The actions points here are:
>
> - initiate a wide survey for user and dev lists, to get to know about
> the usages;
> - remove those configurations that are not used anymore;
> - force migration from Ant to Gradle/Maven;


Let's leave this out for now. There's too many unknowns here. If
there's an IDE configuration that's broken, no one has reported it for
ages, and no one is around to fix it, then I say we should raise the
discussion to remove it.

The Gradle/Maven migration is a hot one, contribute to that discussion
but let's not tangle this work up with it, IMHO.

Totally agree that IDE project files should be as light weight as possible.


> Summarizing everything proposed above I think it is possible to
> simplify adding small contributions easier to the codebase as well as
> save a bunch of committer's time.
>
> So,
> WDYT about the things described above?
> Should I create a CEP for this?


I see no need for a CEP here. An epic and tickets will work.
Again, thanks for the input Maxim!

Re: [DISCUSSION] Cassandra's code style and source code analysis

["Claude Warren, Jr via dev" <de...@cassandra.apache.org>]

+1 for the concept as a whole.  I am certain I could find nits to pick if I
looked deeply.

@mck -- I did have a problem with Cassandra + Eclipse + Java11
(Classpath).  I gave up and am spending time trying to learn IntelliJ.  I
also mentioned it in one of the discussion areas.

Claude

On Thu, Nov 24, 2022 at 8:55 PM Mick Semb Wever <mc...@apache.org> wrote:

>  Thank you for a solid write up Maxim. And welcome to Cassandra, it's
> very positive to see you here.
>
> I whole-heartedly agree with nearly everything you write. Some input
> and questions inline.
>
>
> >
> > As you may know, the infrastructure team has disabled public sign-up
> > to ASF JIRA (the GitHub issues are recommended instead).
> >
>
>
> I suspect (based on chatter in general, but not on dev@ AFAIK) is to
> avoid GH issues and stick with jira. The sign-up hurdle we will
> document on the website: CASSANDRA-18064
>
>
>
> > == 1. Make the checkstyle config a single point of truth for the
> > source code style. ==
> >
> > The checkstyle is already used and included in the Cassandra project
> > build lifecycle (ant command line, Jenkins, CircleCI). There is no
> > need to maintain code style configurations for different types of IDEs
> > (e.g. IntelliJ inspections configuration) since the checkstyle.xml
> > file can be directly imported to IDE used by a developer. This is fair
> > for Intellij Idea, NetBeans, and Eclipse.
>
>
> Big +1
>
>
> > So, I propose to focus on the checks themselves and checking pull
> > requests with automation scripts, rather than maintaining these
> > integrations. The benefits here are avoiding all issues with
> > maintaining configurations for different IDEs. Another good advantage
> > of this approach would be the ability to add new checkstyle rules
> > without touching IDE configuration - and such tickets will be LFH and
> > easy to commit.
> >
> > The actions points here are:
> >
> > - create an umbrella JIRA ticket for all checkstyle issues e.g. [8]
> > (or label checkstyle);
> > - add checkstyle to GitHub pull requests using GitHub actions (execute
> > ant command);
>
>
> Instead of custom GHA scripting, please use our existing
> cassandra-artifact.sh (which should already include all such checks).
>
> Something like
> https://github.com/apache/cassandra/compare/cassandra-3.11...thelastpickle:cassandra:mck/github-actions/3.11
>
>
>
> > == 3. Enable pushing backwards build results for both Jenkins and
> > CircleCI to GitHub pull requests. ==
> >
> > The goal here is to have a "green checkbox" for those GitHub pull
> > requests that have corresponding Jenkins/CircleCI runs. According to
> > the following links, it is completely possible to have those.
> >
> > https://github.com/jenkinsci/github-branch-source-plugin
> > https://circleci.com/docs/enable-checks/
> >
> > Moreover, the GitHub Branch Source Plugin is already enabled for the
> > Cassandra project [16]. The same seems should work the same way for
> > CirleCI, but I have faced the infrastructure team comment [17] that
> > describes admin permissions are required for that, so the question is
> > still open here. I could dig a bit more once we've agreed on it.
> >
> > The actions points here are:
> > - enable Jenkins integration for GitHub pull requests;
> > - enable CircleCI integration for GitHub pull requests;
>
>
> Some folk use CircleCI, some use ci-cassandra. The green checkbox idea
> is great, so long as it's optional. We don't want PRs triggering the
> runs either (there are other mechanisms for triggering for now).
>
>
> > The actions points here are:
> >
> > - initiate a wide survey for user and dev lists, to get to know about
> > the usages;
> > - remove those configurations that are not used anymore;
> > - force migration from Ant to Gradle/Maven;
>
>
> Let's leave this out for now. There's too many unknowns here. If
> there's an IDE configuration that's broken, no one has reported it for
> ages, and no one is around to fix it, then I say we should raise the
> discussion to remove it.
>
> The Gradle/Maven migration is a hot one, contribute to that discussion
> but let's not tangle this work up with it, IMHO.
>
> Totally agree that IDE project files should be as light weight as possible.
>
>
> > Summarizing everything proposed above I think it is possible to
> > simplify adding small contributions easier to the codebase as well as
> > save a bunch of committer's time.
> >
> > So,
> > WDYT about the things described above?
> > Should I create a CEP for this?
>
>
> I see no need for a CEP here. An epic and tickets will work.
> Again, thanks for the input Maxim!
>

Re: [DISCUSSION] Cassandra's code style and source code analysis

[Mick Semb Wever <mc...@apache.org>]

 Thank you for a solid write up Maxim. And welcome to Cassandra, it's
very positive to see you here.

I whole-heartedly agree with nearly everything you write. Some input
and questions inline.


>
> As you may know, the infrastructure team has disabled public sign-up
> to ASF JIRA (the GitHub issues are recommended instead).
>


I suspect (based on chatter in general, but not on dev@ AFAIK) is to
avoid GH issues and stick with jira. The sign-up hurdle we will
document on the website: CASSANDRA-18064



> == 1. Make the checkstyle config a single point of truth for the
> source code style. ==
>
> The checkstyle is already used and included in the Cassandra project
> build lifecycle (ant command line, Jenkins, CircleCI). There is no
> need to maintain code style configurations for different types of IDEs
> (e.g. IntelliJ inspections configuration) since the checkstyle.xml
> file can be directly imported to IDE used by a developer. This is fair
> for Intellij Idea, NetBeans, and Eclipse.


Big +1


> So, I propose to focus on the checks themselves and checking pull
> requests with automation scripts, rather than maintaining these
> integrations. The benefits here are avoiding all issues with
> maintaining configurations for different IDEs. Another good advantage
> of this approach would be the ability to add new checkstyle rules
> without touching IDE configuration - and such tickets will be LFH and
> easy to commit.
>
> The actions points here are:
>
> - create an umbrella JIRA ticket for all checkstyle issues e.g. [8]
> (or label checkstyle);
> - add checkstyle to GitHub pull requests using GitHub actions (execute
> ant command);


Instead of custom GHA scripting, please use our existing
cassandra-artifact.sh (which should already include all such checks).

Something like https://github.com/apache/cassandra/compare/cassandra-3.11...thelastpickle:cassandra:mck/github-actions/3.11



> == 3. Enable pushing backwards build results for both Jenkins and
> CircleCI to GitHub pull requests. ==
>
> The goal here is to have a "green checkbox" for those GitHub pull
> requests that have corresponding Jenkins/CircleCI runs. According to
> the following links, it is completely possible to have those.
>
> https://github.com/jenkinsci/github-branch-source-plugin
> https://circleci.com/docs/enable-checks/
>
> Moreover, the GitHub Branch Source Plugin is already enabled for the
> Cassandra project [16]. The same seems should work the same way for
> CirleCI, but I have faced the infrastructure team comment [17] that
> describes admin permissions are required for that, so the question is
> still open here. I could dig a bit more once we've agreed on it.
>
> The actions points here are:
> - enable Jenkins integration for GitHub pull requests;
> - enable CircleCI integration for GitHub pull requests;


Some folk use CircleCI, some use ci-cassandra. The green checkbox idea
is great, so long as it's optional. We don't want PRs triggering the
runs either (there are other mechanisms for triggering for now).


> The actions points here are:
>
> - initiate a wide survey for user and dev lists, to get to know about
> the usages;
> - remove those configurations that are not used anymore;
> - force migration from Ant to Gradle/Maven;


Let's leave this out for now. There's too many unknowns here. If
there's an IDE configuration that's broken, no one has reported it for
ages, and no one is around to fix it, then I say we should raise the
discussion to remove it.

The Gradle/Maven migration is a hot one, contribute to that discussion
but let's not tangle this work up with it, IMHO.

Totally agree that IDE project files should be as light weight as possible.


> Summarizing everything proposed above I think it is possible to
> simplify adding small contributions easier to the codebase as well as
> save a bunch of committer's time.
>
> So,
> WDYT about the things described above?
> Should I create a CEP for this?


I see no need for a CEP here. An epic and tickets will work.
Again, thanks for the input Maxim!