You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cassandra.apache.org by Sumanth Pasupuleti <su...@gmail.com> on 2018/09/05 00:38:39 UTC
Re: Supporting multiple JDKs
C* 2.2
> I'm still left wondering why we want to use CI resources to find bugs
that the users will never encounter.
How would we be sure users will never encounter bugs unless we build
against that JDK? This is the reason I propose having a CircleCI build
against 1.7.
> If you take out "and optionally run UTs and Dtests against 1.7" from
workflow1 then I'm fine with it.
I don't think it hurts to have workflows that "can" do UTs and DTests
against 1.7. We can run them only when we make a release.
> The time it takes for tests to run is a headache, so to have to run
dtests four times over makes me grimace.
It takes only about 25min with default 4x parallelism to run unit tests in
CircleCI.
4.0
> Currently afaik we can't build the artifacts against only either JDK8 or
JDK11, hence the hybrid JDK setup.
We definitely can build against JDK 8 alone, however from the thread you
linked and from 9608, we wanted to do a stable release that uses JDK8, and
an experimental release, which uses JDK8 to build most files, and JDK11 to
build the Java 11 specific AtomicBTreePartitionBase file.
> In that thread it was mentioned the concerns about the cost of running
tests twice, and whether we should avoid running tests with JDK11 until
we're closer to formally supporting JDK11 at run-time.
My proposal is not to necessarily run UTs and DTests against JDK11 always
with every commit but to have workflows in place that can be used whenever
we deem necessary.
Thanks,
Sumanth
On Thu, Aug 30, 2018 at 12:34 AM Mick Semb Wever <mc...@apache.org> wrote:
>
> Hey Sumanth,
> could you clear a few things up for me…
>
> > C* 2.2
>
> > > I'm still a bit confused as to what's the benefit in compiling with
> > > jdk1.7 and then testing with jdk1.7 or jdk1.8
> >
> > I meant two separate workflows for each JDK i.e.
> > Workflow1: Build against jdk1.7, and optionally run UTs and Dtests
> against
> > 1.7
>
> I'm still left wondering why we want to use CI resources to find bugs that
> the users will never encounter.
> If you take out "and optionally run UTs and Dtests against 1.7" from
> workflow1 then I'm fine with it.
>
> The time it takes for tests to run is a headache, so to have to run dtests
> four times over makes me grimace.
>
>
> > C* 4.0
>
> I'm not quite clear on what the change you intend here is.
>
> Currently afaik we can't build the artefacts against only either JDK8 or
> JDK11, hence the hybrid jdk setup.
> I think building the artefacts should be part of the CI build step because
> patches are not always about java code.
>
> And that unit and dtests are run only against these 'release' built
> artefacts. Presuming the plan remains that the hybrid approach would be the
> 'release' process so long as jdk11 was GA before Cassandra-4.0.
>
>
> https://lists.apache.org/thread.html/45b3f12885f881d211f79368bdd5046e504e0149757cf19c8747bcb2@%3Cdev.cassandra.apache.org%3E
>
> In that thread it was mentioned the concerns about the cost of running
> tests twice, and whether we should avoid running tests with JDK11 until
> we're closer to formally supporting JDK11 at run-time.
>
> regards,
> Mick
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> For additional commands, e-mail: dev-help@cassandra.apache.org
>
>
Re: Supporting multiple JDKs
Posted by Sumanth Pasupuleti <su...@gmail.com>.
> I also can't remember any reports on regressions in 2.2 bug fix releases
specific to 1.7. So what's the actual problem we want to solve here?
Discussion for 2.2 came up from the ticket CASSANDRA-14563
<https://issues.apache.org/jira/browse/CASSANDRA-14563>, which I think came
up from an incident where a JDK1.7 incompatible commit was made into 2.2
branch. But I see the consensus on 2.2 that, there is no ROI on spending
time on 2.2 given that its reaching EOL, which can mean (CASSANDRA-14563
<https://issues.apache.org/jira/browse/CASSANDRA-14563> (in discussion
state) and CASSANDRA-14625
<https://issues.apache.org/jira/browse/CASSANDRA-14625> (in patch available
state) may be resolved as won't fix)
As for 4.0/4.x, I think there are a couple of takeaways from this thread,
and I am happy to pursue them.
1. Make CI jobs use "_build_multi_java" and adding optional workflow to run
tests against JAVA 11 - CASSANDRA-14609
<https://issues.apache.org/jira/browse/CASSANDRA-14609>
2. Add CI job to validate "ant artifacts" - CASSANDRA-14718
<https://issues.apache.org/jira/browse/CASSANDRA-14718>
3. Auto-update of documentation from CI builds - CASSANDRA-14719
<https://issues.apache.org/jira/browse/CASSANDRA-14719>
Thanks,
Sumanth
On Sat, Sep 8, 2018 at 4:59 AM Stefan Podkowinski <sp...@apache.org> wrote:
> I really don't see any benefit at all in having any additional Java 1.7
> specific build and testing changes for the 2.2 branch. The 2.2 version
> is reaching EOL and will only get critical patches until then anyways. I
> also can't remember any reports on regressions in 2.2 bug fix releases
> specific to 1.7. So what's the actual problem we want to solve here?
>
> As for 4.0, we're going to ship multi-release jars, which are targeted
> against Java 8, but also contain Java 11 classes that will only be used
> when executed under Java 11 (also currently just a single class). I can
> see two issues that need our attention with that:
> * We should make sure to use the "_build_multi_java" target for our CI
> jobs, so we're really testing the same build that we would ship. It's
> probably not going to make a real difference, but who knows..
> * It would also be nice to have the option to run tests on CI under
> Java 11, although we only provide "experimental" support for that Java
> version. Nice to have at this point, as there will be plenty of bugs in
> 4.0 to fix, until we should spend time looking more closely for more
> subtle Java 11 issues. But if someone wants to contribute any work to
> make this happen, I'd be glad to have that option running tests on Java
> 11, so don't get me wrong.
>
>
> On 07.09.18 04:10, Sumanth Pasupuleti wrote:
> >> And I would suggest to go further and crash the build with JDK1.7 so we
> > can take away the possibility for users to shoot their foot off this way.
> >
> > I like this suggestion. Either we should be on the side of NO support to
> > JDK 1.7, or if we say we support JDK1.7, I believe we should be building
> > against JDK1.7 to make sure we are compliant.
> > I have a quick clarifying question here - I believe origin of
> > CASSANDRA-14563 is from the introduction of an API in 2.2 that is
> > incompatible with 1.7, that has then been manually detected and fixed.
> Are
> > you suggesting, going further, we would not support 1.7?
> >
> >> Currently I'm unclear on how we would make a stable release using only
> > JDK8, maybe their are plans on the table i don't know about?
> >
> > From the current state of build.xml and from the past discussions, I do
> > believe as well, that we need both JDKs to make a 4.0 release using
> > ‘_build_multi_java’. Bonus would be that, the release would also be able
> to
> > run against Java11, but that would be an experimental release.
> >
> >> I'm not familiar with optional jobs or workflows in CircleCi, do you
> have
> > an example of what you mean at hand?
> >
> > By optional, I was referring to having workflow definitions in place, but
> > calls to those workflows commented out. Basically similar to what we have
> > today.
> > workflows:
> > version: 2
> > build_and_run_tests: *default_jobs
> > #build_and_run_tests: *with_dtest_jobs_only
> > #build_and_run_tests_java11: *with_dtest_jobs_java11
> > Jason created CASSANDRA-14609 for this purpose I believe.
> >
> >> Off-topic, but what are your thoughts on this? Can we add `ant
> > artifacts`, and the building of the docs, as a separate jobs into the
> > existing default CircleCI workflow? I think we should also be looking
> into
> > getting https://cassandra.apache.org/doc/latest/ automatically updated
> > after each successful trunk build, and have
> > https://cassandra.apache.org/doc/X.Y versions on the docs in place
> (which
> > are only updated after each patch release).
> >
> > I like all these ideas! I believe we should be able to add a workflow to
> > test out artifact generation. Will create a JIRA for this. Your
> suggestions
> > around auto-update of docs provides a way to keep our website docs
> > up-to-date. Not sure what it takes to do it though. Will be happy to
> > explore (as part of separate JIRAs).
> >
> > Thanks,
> > Sumanth
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> For additional commands, e-mail: dev-help@cassandra.apache.org
>
>
Re: Supporting multiple JDKs
Posted by Stefan Podkowinski <sp...@apache.org>.
I really don't see any benefit at all in having any additional Java 1.7
specific build and testing changes for the 2.2 branch. The 2.2 version
is reaching EOL and will only get critical patches until then anyways. I
also can't remember any reports on regressions in 2.2 bug fix releases
specific to 1.7. So what's the actual problem we want to solve here?
As for 4.0, we're going to ship multi-release jars, which are targeted
against Java 8, but also contain Java 11 classes that will only be used
when executed under Java 11 (also currently just a single class). I can
see two issues that need our attention with that:
* We should make sure to use the "_build_multi_java" target for our CI
jobs, so we're really testing the same build that we would ship. It's
probably not going to make a real difference, but who knows..
* It would also be nice to have the option to run tests on CI under
Java 11, although we only provide "experimental" support for that Java
version. Nice to have at this point, as there will be plenty of bugs in
4.0 to fix, until we should spend time looking more closely for more
subtle Java 11 issues. But if someone wants to contribute any work to
make this happen, I'd be glad to have that option running tests on Java
11, so don't get me wrong.
On 07.09.18 04:10, Sumanth Pasupuleti wrote:
>> And I would suggest to go further and crash the build with JDK1.7 so we
> can take away the possibility for users to shoot their foot off this way.
>
> I like this suggestion. Either we should be on the side of NO support to
> JDK 1.7, or if we say we support JDK1.7, I believe we should be building
> against JDK1.7 to make sure we are compliant.
> I have a quick clarifying question here - I believe origin of
> CASSANDRA-14563 is from the introduction of an API in 2.2 that is
> incompatible with 1.7, that has then been manually detected and fixed. Are
> you suggesting, going further, we would not support 1.7?
>
>> Currently I'm unclear on how we would make a stable release using only
> JDK8, maybe their are plans on the table i don't know about?
>
> From the current state of build.xml and from the past discussions, I do
> believe as well, that we need both JDKs to make a 4.0 release using
> ‘_build_multi_java’. Bonus would be that, the release would also be able to
> run against Java11, but that would be an experimental release.
>
>> I'm not familiar with optional jobs or workflows in CircleCi, do you have
> an example of what you mean at hand?
>
> By optional, I was referring to having workflow definitions in place, but
> calls to those workflows commented out. Basically similar to what we have
> today.
> workflows:
> version: 2
> build_and_run_tests: *default_jobs
> #build_and_run_tests: *with_dtest_jobs_only
> #build_and_run_tests_java11: *with_dtest_jobs_java11
> Jason created CASSANDRA-14609 for this purpose I believe.
>
>> Off-topic, but what are your thoughts on this? Can we add `ant
> artifacts`, and the building of the docs, as a separate jobs into the
> existing default CircleCI workflow? I think we should also be looking into
> getting https://cassandra.apache.org/doc/latest/ automatically updated
> after each successful trunk build, and have
> https://cassandra.apache.org/doc/X.Y versions on the docs in place (which
> are only updated after each patch release).
>
> I like all these ideas! I believe we should be able to add a workflow to
> test out artifact generation. Will create a JIRA for this. Your suggestions
> around auto-update of docs provides a way to keep our website docs
> up-to-date. Not sure what it takes to do it though. Will be happy to
> explore (as part of separate JIRAs).
>
> Thanks,
> Sumanth
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
For additional commands, e-mail: dev-help@cassandra.apache.org
Re: Supporting multiple JDKs
Posted by Sumanth Pasupuleti <su...@gmail.com>.
> And I would suggest to go further and crash the build with JDK1.7 so we
can take away the possibility for users to shoot their foot off this way.
I like this suggestion. Either we should be on the side of NO support to
JDK 1.7, or if we say we support JDK1.7, I believe we should be building
against JDK1.7 to make sure we are compliant.
I have a quick clarifying question here - I believe origin of
CASSANDRA-14563 is from the introduction of an API in 2.2 that is
incompatible with 1.7, that has then been manually detected and fixed. Are
you suggesting, going further, we would not support 1.7?
> Currently I'm unclear on how we would make a stable release using only
JDK8, maybe their are plans on the table i don't know about?
From the current state of build.xml and from the past discussions, I do
believe as well, that we need both JDKs to make a 4.0 release using
‘_build_multi_java’. Bonus would be that, the release would also be able to
run against Java11, but that would be an experimental release.
> I'm not familiar with optional jobs or workflows in CircleCi, do you have
an example of what you mean at hand?
By optional, I was referring to having workflow definitions in place, but
calls to those workflows commented out. Basically similar to what we have
today.
workflows:
version: 2
build_and_run_tests: *default_jobs
#build_and_run_tests: *with_dtest_jobs_only
#build_and_run_tests_java11: *with_dtest_jobs_java11
Jason created CASSANDRA-14609 for this purpose I believe.
> Off-topic, but what are your thoughts on this? Can we add `ant
artifacts`, and the building of the docs, as a separate jobs into the
existing default CircleCI workflow? I think we should also be looking into
getting https://cassandra.apache.org/doc/latest/ automatically updated
after each successful trunk build, and have
https://cassandra.apache.org/doc/X.Y versions on the docs in place (which
are only updated after each patch release).
I like all these ideas! I believe we should be able to add a workflow to
test out artifact generation. Will create a JIRA for this. Your suggestions
around auto-update of docs provides a way to keep our website docs
up-to-date. Not sure what it takes to do it though. Will be happy to
explore (as part of separate JIRAs).
Thanks,
Sumanth
On Wed, Sep 5, 2018 at 9:30 PM Mick Semb Wever <mc...@apache.org> wrote:
>
>
> > How would we be sure users will never encounter bugs unless we build
> > against that JDK?
>
>
> Apache Cassandra does not distribute JDK1.7 built releases.
>
> The only way a user could repeat such a bug is if they have built C*
> themselves.
>
> I don't think the project should be responsible for every possible build
> combination tom, dick and harry can do.
> That's my 2cents anyway.
>
> And I would suggest to go further and crash the build with JDK1.7 so we
> can take away the possibility for users to shoot their foot off this way.
>
>
> > > The time it takes for tests to run is a headache, so to have to run
> > dtests four times over makes me grimace.
> > It takes only about 25min with default 4x parallelism to run unit tests
> in
> > CircleCI.
>
> I referred to dtests, how would you do this on CircleCI?
> Today dtests take 5-9 hours on builds.apache.org, not including re-runs
> for offheap, large, novnode.
>
>
> > We definitely can build against JDK 8 alone, however from the thread you
> > linked and from 9608, we wanted to do a stable release that uses JDK8,
> and
> > an experimental release, which uses JDK8 to build most files, and JDK11
> to
> > build the Java 11 specific AtomicBTreePartitionBase file.
>
> Currently I'm unclear on how we would make a stable release using only
> JDK8, maybe their are plans on the table i don't know about?
>
> The current build.xml requires both JDKs to run `ant artifacts`.
> That is any release will have compiled in ant all but one class with
> `_build_multi_java` instead of `_build_java8_only`.
>
>
> > My proposal is not to necessarily run UTs and DTests against JDK11 always
> > with every commit but to have workflows in place that can be used
> whenever
> > we deem necessary.
>
>
> I'm not familiar with optional jobs or workflows in CircleCi, do you have
> an example of what you mean at hand?
> I like the idea of having a collection of CircleCi workflows, even if I'd
> rather see less JDKs supported at compile-time.
>
>
> > I think building the artefacts should be part of the CI build step
> because patches are not always about java code.
>
> Off-topic, but what are your thoughts on this? Can we add `ant artifacts`,
> and the building of the docs, as a separate jobs into the existing default
> CircleCI workflow? I think we should also be looking into getting
> https://cassandra.apache.org/doc/latest/ automatically updated after each
> successful trunk build, and have https://cassandra.apache.org/doc/X.Y
> versions on the docs in place (which are only updated after each patch
> release).
>
> regards,
> Mick
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> For additional commands, e-mail: dev-help@cassandra.apache.org
>
>
Re: Supporting multiple JDKs
Posted by Mick Semb Wever <mc...@apache.org>.
> How would we be sure users will never encounter bugs unless we build
> against that JDK?
Apache Cassandra does not distribute JDK1.7 built releases.
The only way a user could repeat such a bug is if they have built C* themselves.
I don't think the project should be responsible for every possible build combination tom, dick and harry can do.
That's my 2cents anyway.
And I would suggest to go further and crash the build with JDK1.7 so we can take away the possibility for users to shoot their foot off this way.
> > The time it takes for tests to run is a headache, so to have to run
> dtests four times over makes me grimace.
> It takes only about 25min with default 4x parallelism to run unit tests in
> CircleCI.
I referred to dtests, how would you do this on CircleCI?
Today dtests take 5-9 hours on builds.apache.org, not including re-runs for offheap, large, novnode.
> We definitely can build against JDK 8 alone, however from the thread you
> linked and from 9608, we wanted to do a stable release that uses JDK8, and
> an experimental release, which uses JDK8 to build most files, and JDK11 to
> build the Java 11 specific AtomicBTreePartitionBase file.
Currently I'm unclear on how we would make a stable release using only JDK8, maybe their are plans on the table i don't know about?
The current build.xml requires both JDKs to run `ant artifacts`.
That is any release will have compiled in ant all but one class with `_build_multi_java` instead of `_build_java8_only`.
> My proposal is not to necessarily run UTs and DTests against JDK11 always
> with every commit but to have workflows in place that can be used whenever
> we deem necessary.
I'm not familiar with optional jobs or workflows in CircleCi, do you have an example of what you mean at hand?
I like the idea of having a collection of CircleCi workflows, even if I'd rather see less JDKs supported at compile-time.
> I think building the artefacts should be part of the CI build step because patches are not always about java code.
Off-topic, but what are your thoughts on this? Can we add `ant artifacts`, and the building of the docs, as a separate jobs into the existing default CircleCI workflow? I think we should also be looking into getting https://cassandra.apache.org/doc/latest/ automatically updated after each successful trunk build, and have https://cassandra.apache.org/doc/X.Y versions on the docs in place (which are only updated after each patch release).
regards,
Mick
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
For additional commands, e-mail: dev-help@cassandra.apache.org