You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@cassandra.apache.org by Jeff Jirsa <jj...@gmail.com> on 2017/11/16 01:44:27 UTC

Flakey Dtests

In lieu of a weekly wrap-up, here's a pre-Thanksgiving call for help.

If you haven't been paying attention to JIRA, you likely didn't notice that
Josh went through and triage/categorized a bunch of issues by adding
components, and Michael took the time to open a bunch of JIRAs for failing
tests.

How many is a bunch? Something like 35 or so just for tests currently
failing on trunk. If you're a regular contributor, you already know that
dtests are flakey - it'd be great if a few of us can go through and fix a
few. Even incremental improvements are improvements. Here's an easy search
to find them:

https://issues.apache.org/jira/secure/IssueNavigator.jspa?reset=true&jqlQuery=project+%3D+CASSANDRA+AND+component+%3D+Testing+ORDER+BY+updated+DESC%2C+priority+DESC%2C+created+ASC&mode=hide

If you're a new contributor, fixing tests is often a good way to learn a
new part of the codebase. Many of these are dtests, which live in a
different repo ( https://github.com/apache/cassandra-dtest ) and are in
python, but have no fear, the repo has instructions for setting up and
running dtests(
https://github.com/apache/cassandra-dtest/blob/master/INSTALL.md )

Normal contribution workflow applies: self-assign the ticket if you want to
work on it, click on 'start progress' to indicate that you're working on
it, mark it 'patch available' when you've uploaded code to be reviewed (in
a github branch, or as a standalone patch file attached to the JIRA). If
you have questions, feel free to email the dev list (that's what it's here
for).

Many thanks will be given,
- Jeff

Re: Flakey Dtests

Posted by Josh McKenzie <jm...@apache.org>.

>
> I fixed one CDC uTest, please review:https://issues.apache.
> org/jira/browse/CASSANDRA-14066

Thanks for taking that Jay! I ended up buried on things last week and am
OOO this week so won't be able to get to these until next week at the
earliest.

Also, it just came up that we somehow neglected to merge the PR for the
12148 dtest changes on master, so I'm sure that's playing in to the
failures there (failed hand-off from one author to another on the PR led to
this unfortunately).

On Mon, Nov 27, 2017 at 5:55 PM, Jaydeep Chovatia <
chovatia.jaydeep@gmail.com> wrote:

> This is useful info, Thanks!
>
> Jaydeep
>
> On Mon, Nov 27, 2017 at 2:43 PM, Michael Kjellman <
> mkjellman@internalcircle.com> wrote:
>
> > Complicated question unfortunately — and something we’re actively working
> > on improving:
> >
> > Cassci is no longer being offered/run by Datastax and so we've need to
> > come up with a new solution, and what that ultimately is is still a WIP —
> > it’s loss was very huge obviously and a testament to the awesome resource
> > and effort that was put into providing it to the community for all those
> > years.
> >
> >  - Short Term/Current: Tests (both dtests and unit tests) are being run
> > via the ASF Jenkins (https://builds.apache.org) - but that solution
> isn’t
> > hugely helpful as it’s resource constrained.
> >  - Short-Medium Term: we hope to get a fully baked CircleCI solution to
> > get reliable fast test runs.
> >  - Long Term: Actively being discussed but I’m optimistic that we can get
> > something awesome for the project with some stable combination of
> CircleCI
> > + ASF Jenkins, and once we do I’m sure this will change any long term
> plans.
> >
> > For Unit Tests (a.k.a the Java ones in tree - https://github.com/apache/
> > cassandra/tree/trunk/test/unit/org/apache/cassandra):
> > Take a look at https://builds.apache.org/view/A-D/view/Cassandra/job/
> > Cassandra-trunk-test/… looks like the last successful job to finish was
> > #389. (https://builds.apache.org/view/A-D/view/Cassandra/job/
> > Cassandra-trunk-test/389/testReport/). There are currently a total of 6
> > tests  (all from CompressedInputStreamTest) failing on trunk via ASF
> > Jenkins. These specific test failures are environmental. The only *unit*
> > test on trunk that I currently know to be flaky is
> > org.apache.cassandra.cql3.ViewTest. testRegularColumnTimestampUpdates
> > (tracked as https://issues.apache.org/jira/browse/CASSANDRA-14054)
> >
> > For Distributed Tests (DTests) (a.k.a the Python ones -
> > https://github.com/apache/cassandra-dtest):
> > The situation is a great deal more complicated due to the length of time
> > and number of resources executing all of the dtests take (and executing
> the
> > tests across the various configurations)...
> >
> > There are 4 dtest jobs on ASF Jenkins for trunk:
> > https://builds.apache.org/view/A-D/view/Cassandra/job/
> > Cassandra-trunk-dtest/
> > https://builds.apache.org/view/A-D/view/Cassandra/job/
> > Cassandra-trunk-dtest-large/
> > https://builds.apache.org/view/A-D/view/Cassandra/job/
> > Cassandra-trunk-dtest-novnode/
> > https://builds.apache.org/view/A-D/view/Cassandra/job/
> > Cassandra-trunk-dtest-offheap/
> >
> > It looks like you’ll need to go back to run #353 (
> > https://builds.apache.org/view/A-D/view/Cassandra/job/
> > Cassandra-trunk-dtest/353/testReport/) to see the test results as the
> > last 2 jobs that were triggered failed to execute. Depending on the
> > environment variables set tests are executed or skipped — so you’ll see
> > different tests being run on the no-vnode job/off-heap job/regular dtest
> > job (or some tests might be run multiple times)
> >
> >
> > More recently we’ve been woking on getting CircleCI running. Some sample
> > runs from my personal fork can be seen at https://circleci.com/gh/
> > mkjellman/cassandra/tree/trunk_circle. I’m personally using a paid
> > account to get more CircleCI resources (with 100 containers we can
> actually
> > build the project, run all of the unit tests, and run all of the dtests
> in
> > roughly 28 minutes!). I’m actively working to determine out exactly can
> > (and cannot) be executed reliably, routinely, and easily by anyone with
> > just a simple free CircleCI account.
> >
> > I’m also working on getting scheduled CircleCI daily runs setup against
> > trunk/3.0 — more on both of those when we’ve got that story fully baked..
> > Hope this answers your question! There are quite a few dtests currently
> > failing and as Jeff mentioned I’ve created JIRAs for a lot of them
> already
> > so any help (no matter how trivial or annoying it might be or seem) to
> get
> > everything green again.
> >
> > best,
> > kjellman
> >
> >
> > On Nov 27, 2017, at 1:54 PM, Jaydeep Chovatia <
> chovatia.jaydeep@gmail.com<
> > mailto:chovatia.jaydeep@gmail.com>> wrote:
> >
> > Is there a way to check which tests are failing in trunk currently?
> > Previously this URL <http://cassci.datastax.com/> was giving such
> results
> > but is no longer working.
> >
> > Jaydeep
> >
> > On Wed, Nov 15, 2017 at 5:44 PM, Jeff Jirsa <jjirsa@gmail.com<mailto:
> jjirs
> > a@gmail.com>> wrote:
> >
> > In lieu of a weekly wrap-up, here's a pre-Thanksgiving call for help.
> >
> > If you haven't been paying attention to JIRA, you likely didn't notice
> that
> > Josh went through and triage/categorized a bunch of issues by adding
> > components, and Michael took the time to open a bunch of JIRAs for
> failing
> > tests.
> >
> > How many is a bunch? Something like 35 or so just for tests currently
> > failing on trunk.  If you're a regular contributor, you already know that
> > dtests are flakey - it'd be great if a few of us can go through and fix a
> > few. Even incremental improvements are improvements. Here's an easy
> search
> > to find them:
> >
> > https://issues.apache.org/jira/secure/IssueNavigator.
> > jspa?reset=true&jqlQuery=project+%3D+CASSANDRA+AND+
> > component+%3D+Testing+ORDER+BY+updated+DESC%2C+priority+
> > DESC%2C+created+ASC&mode=hide
> >
> > If you're a new contributor, fixing tests is often a good way to learn a
> > new part of the codebase. Many of these are dtests, which live in a
> > different repo ( https://github.com/apache/cassandra-dtest ) and are in
> > python, but have no fear, the repo has instructions for setting up and
> > running dtests(
> > https://github.com/apache/cassandra-dtest/blob/master/INSTALL.md )
> >
> > Normal contribution workflow applies: self-assign the ticket if you want
> to
> > work on it, click on 'start progress' to indicate that you're working on
> > it, mark it 'patch available' when you've uploaded code to be reviewed
> (in
> > a github branch, or as a standalone patch file attached to the JIRA). If
> > you have questions, feel free to email the dev list (that's what it's
> here
> > for).
> >
> > Many thanks will be given,
> > - Jeff
> >
> >
> >
>

Re: Flakey Dtests

Posted by Jaydeep Chovatia <ch...@gmail.com>.

This is useful info, Thanks!

Jaydeep

On Mon, Nov 27, 2017 at 2:43 PM, Michael Kjellman <
mkjellman@internalcircle.com> wrote:

> Complicated question unfortunately — and something we’re actively working
> on improving:
>
> Cassci is no longer being offered/run by Datastax and so we've need to
> come up with a new solution, and what that ultimately is is still a WIP —
> it’s loss was very huge obviously and a testament to the awesome resource
> and effort that was put into providing it to the community for all those
> years.
>
>  - Short Term/Current: Tests (both dtests and unit tests) are being run
> via the ASF Jenkins (https://builds.apache.org) - but that solution isn’t
> hugely helpful as it’s resource constrained.
>  - Short-Medium Term: we hope to get a fully baked CircleCI solution to
> get reliable fast test runs.
>  - Long Term: Actively being discussed but I’m optimistic that we can get
> something awesome for the project with some stable combination of CircleCI
> + ASF Jenkins, and once we do I’m sure this will change any long term plans.
>
> For Unit Tests (a.k.a the Java ones in tree - https://github.com/apache/
> cassandra/tree/trunk/test/unit/org/apache/cassandra):
> Take a look at https://builds.apache.org/view/A-D/view/Cassandra/job/
> Cassandra-trunk-test/… looks like the last successful job to finish was
> #389. (https://builds.apache.org/view/A-D/view/Cassandra/job/
> Cassandra-trunk-test/389/testReport/). There are currently a total of 6
> tests  (all from CompressedInputStreamTest) failing on trunk via ASF
> Jenkins. These specific test failures are environmental. The only *unit*
> test on trunk that I currently know to be flaky is
> org.apache.cassandra.cql3.ViewTest. testRegularColumnTimestampUpdates
> (tracked as https://issues.apache.org/jira/browse/CASSANDRA-14054)
>
> For Distributed Tests (DTests) (a.k.a the Python ones -
> https://github.com/apache/cassandra-dtest):
> The situation is a great deal more complicated due to the length of time
> and number of resources executing all of the dtests take (and executing the
> tests across the various configurations)...
>
> There are 4 dtest jobs on ASF Jenkins for trunk:
> https://builds.apache.org/view/A-D/view/Cassandra/job/
> Cassandra-trunk-dtest/
> https://builds.apache.org/view/A-D/view/Cassandra/job/
> Cassandra-trunk-dtest-large/
> https://builds.apache.org/view/A-D/view/Cassandra/job/
> Cassandra-trunk-dtest-novnode/
> https://builds.apache.org/view/A-D/view/Cassandra/job/
> Cassandra-trunk-dtest-offheap/
>
> It looks like you’ll need to go back to run #353 (
> https://builds.apache.org/view/A-D/view/Cassandra/job/
> Cassandra-trunk-dtest/353/testReport/) to see the test results as the
> last 2 jobs that were triggered failed to execute. Depending on the
> environment variables set tests are executed or skipped — so you’ll see
> different tests being run on the no-vnode job/off-heap job/regular dtest
> job (or some tests might be run multiple times)
>
>
> More recently we’ve been woking on getting CircleCI running. Some sample
> runs from my personal fork can be seen at https://circleci.com/gh/
> mkjellman/cassandra/tree/trunk_circle. I’m personally using a paid
> account to get more CircleCI resources (with 100 containers we can actually
> build the project, run all of the unit tests, and run all of the dtests in
> roughly 28 minutes!). I’m actively working to determine out exactly can
> (and cannot) be executed reliably, routinely, and easily by anyone with
> just a simple free CircleCI account.
>
> I’m also working on getting scheduled CircleCI daily runs setup against
> trunk/3.0 — more on both of those when we’ve got that story fully baked..
> Hope this answers your question! There are quite a few dtests currently
> failing and as Jeff mentioned I’ve created JIRAs for a lot of them already
> so any help (no matter how trivial or annoying it might be or seem) to get
> everything green again.
>
> best,
> kjellman
>
>
> On Nov 27, 2017, at 1:54 PM, Jaydeep Chovatia <chovatia.jaydeep@gmail.com<
> mailto:chovatia.jaydeep@gmail.com>> wrote:
>
> Is there a way to check which tests are failing in trunk currently?
> Previously this URL <http://cassci.datastax.com/> was giving such results
> but is no longer working.
>
> Jaydeep
>
> On Wed, Nov 15, 2017 at 5:44 PM, Jeff Jirsa <jjirsa@gmail.com<mailto:jjirs
> a@gmail.com>> wrote:
>
> In lieu of a weekly wrap-up, here's a pre-Thanksgiving call for help.
>
> If you haven't been paying attention to JIRA, you likely didn't notice that
> Josh went through and triage/categorized a bunch of issues by adding
> components, and Michael took the time to open a bunch of JIRAs for failing
> tests.
>
> How many is a bunch? Something like 35 or so just for tests currently
> failing on trunk.  If you're a regular contributor, you already know that
> dtests are flakey - it'd be great if a few of us can go through and fix a
> few. Even incremental improvements are improvements. Here's an easy search
> to find them:
>
> https://issues.apache.org/jira/secure/IssueNavigator.
> jspa?reset=true&jqlQuery=project+%3D+CASSANDRA+AND+
> component+%3D+Testing+ORDER+BY+updated+DESC%2C+priority+
> DESC%2C+created+ASC&mode=hide
>
> If you're a new contributor, fixing tests is often a good way to learn a
> new part of the codebase. Many of these are dtests, which live in a
> different repo ( https://github.com/apache/cassandra-dtest ) and are in
> python, but have no fear, the repo has instructions for setting up and
> running dtests(
> https://github.com/apache/cassandra-dtest/blob/master/INSTALL.md )
>
> Normal contribution workflow applies: self-assign the ticket if you want to
> work on it, click on 'start progress' to indicate that you're working on
> it, mark it 'patch available' when you've uploaded code to be reviewed (in
> a github branch, or as a standalone patch file attached to the JIRA). If
> you have questions, feel free to email the dev list (that's what it's here
> for).
>
> Many thanks will be given,
> - Jeff
>
>
>

Re: Flakey Dtests

Posted by Michael Kjellman <mk...@internalcircle.com>.

and just to make it super clear how awesome this is: currently the dtests when executed via ASF Jenkins (if they actually run successfully at all) take roughly 15+ hours to execute. Being able to run *everything* reliably and stably in 28 minutes is obviously many many orders of magnitude better.

best,
kjellman

On Nov 27, 2017, at 2:43 PM, Michael Kjellman <mk...@internalcircle.com>> wrote:

(with 100 containers we can actually build the project, run all of the unit tests, and run all of the dtests in roughly 28 minutes!).

Re: Flakey Dtests

Posted by Michael Kjellman <mk...@internalcircle.com>.

Complicated question unfortunately — and something we’re actively working on improving:

Cassci is no longer being offered/run by Datastax and so we've need to come up with a new solution, and what that ultimately is is still a WIP — it’s loss was very huge obviously and a testament to the awesome resource and effort that was put into providing it to the community for all those years.

- Short Term/Current: Tests (both dtests and unit tests) are being run via the ASF Jenkins (https://builds.apache.org) - but that solution isn’t hugely helpful as it’s resource constrained.
- Short-Medium Term: we hope to get a fully baked CircleCI solution to get reliable fast test runs.
- Long Term: Actively being discussed but I’m optimistic that we can get something awesome for the project with some stable combination of CircleCI + ASF Jenkins, and once we do I’m sure this will change any long term plans.

For Unit Tests (a.k.a the Java ones in tree - https://github.com/apache/cassandra/tree/trunk/test/unit/org/apache/cassandra):
Take a look at https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-trunk-test/… looks like the last successful job to finish was #389. (https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-trunk-test/389/testReport/). There are currently a total of 6 tests (all from CompressedInputStreamTest) failing on trunk via ASF Jenkins. These specific test failures are environmental. The only *unit* test on trunk that I currently know to be flaky is org.apache.cassandra.cql3.ViewTest. testRegularColumnTimestampUpdates (tracked as https://issues.apache.org/jira/browse/CASSANDRA-14054)

For Distributed Tests (DTests) (a.k.a the Python ones - https://github.com/apache/cassandra-dtest):
The situation is a great deal more complicated due to the length of time and number of resources executing all of the dtests take (and executing the tests across the various configurations)...

There are 4 dtest jobs on ASF Jenkins for trunk:
https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-trunk-dtest/
https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-trunk-dtest-large/
https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-trunk-dtest-novnode/
https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-trunk-dtest-offheap/

It looks like you’ll need to go back to run #353 (https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-trunk-dtest/353/testReport/) to see the test results as the last 2 jobs that were triggered failed to execute. Depending on the environment variables set tests are executed or skipped — so you’ll see different tests being run on the no-vnode job/off-heap job/regular dtest job (or some tests might be run multiple times)

More recently we’ve been woking on getting CircleCI running. Some sample runs from my personal fork can be seen at https://circleci.com/gh/mkjellman/cassandra/tree/trunk_circle. I’m personally using a paid account to get more CircleCI resources (with 100 containers we can actually build the project, run all of the unit tests, and run all of the dtests in roughly 28 minutes!). I’m actively working to determine out exactly can (and cannot) be executed reliably, routinely, and easily by anyone with just a simple free CircleCI account.

I’m also working on getting scheduled CircleCI daily runs setup against trunk/3.0 — more on both of those when we’ve got that story fully baked.. Hope this answers your question! There are quite a few dtests currently failing and as Jeff mentioned I’ve created JIRAs for a lot of them already so any help (no matter how trivial or annoying it might be or seem) to get everything green again.

best,
kjellman

On Nov 27, 2017, at 1:54 PM, Jaydeep Chovatia <ch...@gmail.com>> wrote:

Is there a way to check which tests are failing in trunk currently?
Previously this URL <http://cassci.datastax.com/> was giving such results
but is no longer working.

Jaydeep

On Wed, Nov 15, 2017 at 5:44 PM, Jeff Jirsa <jj...@gmail.com>> wrote:

In lieu of a weekly wrap-up, here's a pre-Thanksgiving call for help.

https://issues.apache.org/jira/secure/IssueNavigator.
jspa?reset=true&jqlQuery=project+%3D+CASSANDRA+AND+
component+%3D+Testing+ORDER+BY+updated+DESC%2C+priority+
DESC%2C+created+ASC&mode=hide

Many thanks will be given,
- Jeff

Re: Flakey Dtests

Posted by Jaydeep Chovatia <ch...@gmail.com>.

Is there a way to check which tests are failing in trunk currently?
Previously this URL <http://cassci.datastax.com/> was giving such results
but is no longer working.

Jaydeep

On Wed, Nov 15, 2017 at 5:44 PM, Jeff Jirsa <jj...@gmail.com> wrote:

> In lieu of a weekly wrap-up, here's a pre-Thanksgiving call for help.
>
> If you haven't been paying attention to JIRA, you likely didn't notice that
> Josh went through and triage/categorized a bunch of issues by adding
> components, and Michael took the time to open a bunch of JIRAs for failing
> tests.
>
> How many is a bunch? Something like 35 or so just for tests currently
> failing on trunk.  If you're a regular contributor, you already know that
> dtests are flakey - it'd be great if a few of us can go through and fix a
> few. Even incremental improvements are improvements. Here's an easy search
> to find them:
>
> https://issues.apache.org/jira/secure/IssueNavigator.
> jspa?reset=true&jqlQuery=project+%3D+CASSANDRA+AND+
> component+%3D+Testing+ORDER+BY+updated+DESC%2C+priority+
> DESC%2C+created+ASC&mode=hide
>
> If you're a new contributor, fixing tests is often a good way to learn a
> new part of the codebase. Many of these are dtests, which live in a
> different repo ( https://github.com/apache/cassandra-dtest ) and are in
> python, but have no fear, the repo has instructions for setting up and
> running dtests(
> https://github.com/apache/cassandra-dtest/blob/master/INSTALL.md )
>
> Normal contribution workflow applies: self-assign the ticket if you want to
> work on it, click on 'start progress' to indicate that you're working on
> it, mark it 'patch available' when you've uploaded code to be reviewed (in
> a github branch, or as a standalone patch file attached to the JIRA). If
> you have questions, feel free to email the dev list (that's what it's here
> for).
>
> Many thanks will be given,
> - Jeff
>

Re: Flakey Dtests

Posted by Philip Thompson <ph...@datastax.com>.

Hi Michael,

I release ccm and publish to pypi. Releases are cut on request, but since
that commit is from December 2016, I can tell you at least ten releases
have gone out since then. The release apache/cassandra-dtest uses is 3.1.0,
which was cut on November 3, 2017.



On Fri, Nov 17, 2017 at 9:28 AM, Michael Kjellman <
mkjellman@internalcircle.com> wrote:

> I’m guessing this was part of https://issues.apache.org/
> jira/browse/CASSANDRA-11115?
>
> I see Sylvain left a comment about something that sounds pretty similar…
> was this actually resolved? looks like it was merged as
> https://github.com/pcmanus/ccm/commit/1c0bf62e0b21fc78ee09026882953a
> 5436ccf0f0? when do ccm releases get published to pypy?
>
>
> On Nov 17, 2017, at 12:18 AM, Michael Kjellman <
> mkjellman@internalcircle.com<ma...@internalcircle.com>> wrote:
>
> I see a ton of upgrade tests right now failing for:
>
>
> Unexpected error in node1 log, error:
> ERROR [main] 2017-11-17 07:57:54,477 CassandraDaemon.java:672 - Exception
> encountered during startup: Invalid yaml. Please remove properties
> [rpc_port] from your cassandra.yaml
>
> I do see that rpc_port is in 3.0 and it seems to have been yanked from
> trunk.. So it seems like a legitimate failure.. I’m not sure I fully
> understand how the yaml upgrade path works for upgrade test dtests. I’ve
> taken a look at upgrade_tests/upgrade_manifest.py and
> upgrade_tests/README.md… can anyone shed any light on how this is supposed
> to work? Was handling rpc_port in the upgrade dtests just missed when this
> was removed for whatever reason from trunk?
>
> thanks…
>
>
> best,
> kjellman
>
> On Nov 16, 2017, at 9:09 PM, Michael Kjellman <
> mkjellman@internalcircle.com<ma...@internalcircle.com><mailto:
> mkjellman@internalcircle.com>> wrote:
>
> Quick update re: dtests and off-heap memtables:
>
> I’ve filed CASSANDRA-14056 (Many dtests fail with ConfigurationException:
> offheap_objects are not available in 3.0 when OFFHEAP_MEMTABLES=“true”)
>
> Looks like we’re gonna need to do some work to test this configuration and
> right now it’s pretty broken...
>
> Do we have any volunteers to fix the broken Materialized Views and CDC
> DTests?
>
> best,
> kjellman
>
>
> On Nov 15, 2017, at 5:59 PM, Michael Kjellman <
> mkjellman@internalcircle.com<ma...@internalcircle.com><mailto:
> mkjellman@internalcircle.com>> wrote:
>
> yes - true- some are flaky, but almost all of the ones i filed fail 100%
> (💯) of the time. i look forward to triaging just the remaining flaky ones
> (hopefully - without powers combined - by the end of this month!!)
>
> appreciate everyone’s help - no matter how small... i already personally
> did a few “fun” random-python-class-is-missing-return-after-method stuff.
>
> we’ve wanted this for a while and now is our time to actually execute and
> make good on our previous dev list promises.
>
> best,
> kjellman
>
> On Nov 15, 2017, at 5:45 PM, Jeff Jirsa <jjirsa@gmail.com<mailto:jjirs
> a@gmail.com><ma...@gmail.com>> wrote:
>
> In lieu of a weekly wrap-up, here's a pre-Thanksgiving call for help.
>
> If you haven't been paying attention to JIRA, you likely didn't notice that
> Josh went through and triage/categorized a bunch of issues by adding
> components, and Michael took the time to open a bunch of JIRAs for failing
> tests.
>
> How many is a bunch? Something like 35 or so just for tests currently
> failing on trunk.  If you're a regular contributor, you already know that
> dtests are flakey - it'd be great if a few of us can go through and fix a
> few. Even incremental improvements are improvements. Here's an easy search
> to find them:
>
> https://issues.apache.org/jira/secure/IssueNavigator.
> jspa?reset=true&jqlQuery=project+%3D+CASSANDRA+AND+
> component+%3D+Testing+ORDER+BY+updated+DESC%2C+priority+
> DESC%2C+created+ASC&mode=hide
>
> If you're a new contributor, fixing tests is often a good way to learn a
> new part of the codebase. Many of these are dtests, which live in a
> different repo ( https://github.com/apache/cassandra-dtest ) and are in
> python, but have no fear, the repo has instructions for setting up and
> running dtests(
> https://github.com/apache/cassandra-dtest/blob/master/INSTALL.md )
>
> Normal contribution workflow applies: self-assign the ticket if you want to
> work on it, click on 'start progress' to indicate that you're working on
> it, mark it 'patch available' when you've uploaded code to be reviewed (in
> a github branch, or as a standalone patch file attached to the JIRA). If
> you have questions, feel free to email the dev list (that's what it's here
> for).
>
> Many thanks will be given,
> - Jeff
>
>
>
>

Re: Flakey Dtests

Posted by Michael Kjellman <mk...@internalcircle.com>.

I’m guessing this was part of https://issues.apache.org/jira/browse/CASSANDRA-11115?

I see Sylvain left a comment about something that sounds pretty similar… was this actually resolved? looks like it was merged as https://github.com/pcmanus/ccm/commit/1c0bf62e0b21fc78ee09026882953a5436ccf0f0? when do ccm releases get published to pypy?

On Nov 17, 2017, at 12:18 AM, Michael Kjellman <mk...@internalcircle.com>> wrote:

I see a ton of upgrade tests right now failing for:

Unexpected error in node1 log, error:
ERROR [main] 2017-11-17 07:57:54,477 CassandraDaemon.java:672 - Exception encountered during startup: Invalid yaml. Please remove properties [rpc_port] from your cassandra.yaml

I do see that rpc_port is in 3.0 and it seems to have been yanked from trunk.. So it seems like a legitimate failure.. I’m not sure I fully understand how the yaml upgrade path works for upgrade test dtests. I’ve taken a look at upgrade_tests/upgrade_manifest.py and upgrade_tests/README.md… can anyone shed any light on how this is supposed to work? Was handling rpc_port in the upgrade dtests just missed when this was removed for whatever reason from trunk?

thanks…

best,
kjellman

On Nov 16, 2017, at 9:09 PM, Michael Kjellman <mk...@internalcircle.com>> wrote:

Quick update re: dtests and off-heap memtables:

I’ve filed CASSANDRA-14056 (Many dtests fail with ConfigurationException: offheap_objects are not available in 3.0 when OFFHEAP_MEMTABLES=“true”)

Looks like we’re gonna need to do some work to test this configuration and right now it’s pretty broken...

Do we have any volunteers to fix the broken Materialized Views and CDC DTests?

best,
kjellman

On Nov 15, 2017, at 5:59 PM, Michael Kjellman <mk...@internalcircle.com>> wrote:

yes - true- some are flaky, but almost all of the ones i filed fail 100% (💯) of the time. i look forward to triaging just the remaining flaky ones (hopefully - without powers combined - by the end of this month!!)

appreciate everyone’s help - no matter how small... i already personally did a few “fun” random-python-class-is-missing-return-after-method stuff.

we’ve wanted this for a while and now is our time to actually execute and make good on our previous dev list promises.

best,
kjellman

On Nov 15, 2017, at 5:45 PM, Jeff Jirsa <jj...@gmail.com>> wrote:

In lieu of a weekly wrap-up, here's a pre-Thanksgiving call for help.

https://issues.apache.org/jira/secure/IssueNavigator.jspa?reset=true&jqlQuery=project+%3D+CASSANDRA+AND+component+%3D+Testing+ORDER+BY+updated+DESC%2C+priority+DESC%2C+created+ASC&mode=hide

Many thanks will be given,
- Jeff

Re: Flakey Dtests

Posted by Michael Kjellman <mk...@internalcircle.com>.

I see a ton of upgrade tests right now failing for:

thanks…

best,
kjellman

On Nov 16, 2017, at 9:09 PM, Michael Kjellman <mk...@internalcircle.com>> wrote:

Quick update re: dtests and off-heap memtables:

I’ve filed CASSANDRA-14056 (Many dtests fail with ConfigurationException: offheap_objects are not available in 3.0 when OFFHEAP_MEMTABLES=“true”)

Looks like we’re gonna need to do some work to test this configuration and right now it’s pretty broken...

Do we have any volunteers to fix the broken Materialized Views and CDC DTests?

best,
kjellman

On Nov 15, 2017, at 5:59 PM, Michael Kjellman <mk...@internalcircle.com>> wrote:

appreciate everyone’s help - no matter how small... i already personally did a few “fun” random-python-class-is-missing-return-after-method stuff.

we’ve wanted this for a while and now is our time to actually execute and make good on our previous dev list promises.

best,
kjellman

On Nov 15, 2017, at 5:45 PM, Jeff Jirsa <jj...@gmail.com>> wrote:

In lieu of a weekly wrap-up, here's a pre-Thanksgiving call for help.

https://issues.apache.org/jira/secure/IssueNavigator.jspa?reset=true&jqlQuery=project+%3D+CASSANDRA+AND+component+%3D+Testing+ORDER+BY+updated+DESC%2C+priority+DESC%2C+created+ASC&mode=hide

Many thanks will be given,
- Jeff

Re: Flakey Dtests

Posted by Michael Kjellman <mk...@internalcircle.com>.

do you know why this is the case? shouldn’t -all test...all?

> On Nov 27, 2017, at 7:39 PM, Michael Shuler <mi...@pbandjelly.org> wrote:
> 
> The `test-cdc` target is not a dependent of `test-all`, so it was set up
> as a separate job in Jenkins:
> https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-3.11-test-cdc/
> https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-trunk-test-cdc/
> 
> -- 
> Michael
> 
>> On 11/27/2017 03:45 PM, Michael Kjellman wrote:
>> Hey Jay:
>> 
>> Thanks!! I just took a quick look at the JIRA and noticed that there is a “test-cdc” ant target? So, does that mean CDC get’s no testing with ant test? Do you know any of the history around this?
>> 
>>> On Nov 27, 2017, at 9:44 AM, Jay Zhuang <ja...@yahoo.com.INVALID> wrote:
>>> 
>>> I fixed one CDC uTest, please review:https://issues.apache.org/jira/browse/CASSANDRA-14066
>>> 
>>> 
>>>   On Friday, November 17, 2017 6:34 AM, Josh McKenzie <jm...@apache.org> wrote:
>>> 
>>> 
>>>> 
>>>> Do we have any volunteers to fix the broken Materialized Views and CDC
>>>> DTests?
>>> 
>>> I'll try to take a look at the CDC tests next week; looks like one of the
>>> base unit tests is failing as well.
>>> 
>>> On Fri, Nov 17, 2017 at 12:09 AM, Michael Kjellman <
>>> mkjellman@internalcircle.com> wrote:
>>> 
>>>> Quick update re: dtests and off-heap memtables:
>>>> 
>>>> I’ve filed CASSANDRA-14056 (Many dtests fail with ConfigurationException:
>>>> offheap_objects are not available in 3.0 when OFFHEAP_MEMTABLES=“true”)
>>>> 
>>>> Looks like we’re gonna need to do some work to test this configuration and
>>>> right now it’s pretty broken...
>>>> 
>>>> Do we have any volunteers to fix the broken Materialized Views and CDC
>>>> DTests?
>>>> 
>>>> best,
>>>> kjellman
>>>> 
>>>> 
>>>>> On Nov 15, 2017, at 5:59 PM, Michael Kjellman <
>>>> mkjellman@internalcircle.com> wrote:
>>>>> 
>>>>> yes - true- some are flaky, but almost all of the ones i filed fail 100%
>>>> (💯) of the time. i look forward to triaging just the remaining flaky ones
>>>> (hopefully - without powers combined - by the end of this month!!)
>>>>> 
>>>>> appreciate everyone’s help - no matter how small... i already personally
>>>> did a few “fun” random-python-class-is-missing-return-after-method stuff.
>>>>> 
>>>>> we’ve wanted this for a while and now is our time to actually execute
>>>> and make good on our previous dev list promises.
>>>>> 
>>>>> best,
>>>>> kjellman
>>>>> 
>>>>>> On Nov 15, 2017, at 5:45 PM, Jeff Jirsa <jj...@gmail.com> wrote:
>>>>>> 
>>>>>> In lieu of a weekly wrap-up, here's a pre-Thanksgiving call for help.
>>>>>> 
>>>>>> If you haven't been paying attention to JIRA, you likely didn't notice
>>>> that
>>>>>> Josh went through and triage/categorized a bunch of issues by adding
>>>>>> components, and Michael took the time to open a bunch of JIRAs for
>>>> failing
>>>>>> tests.
>>>>>> 
>>>>>> How many is a bunch? Something like 35 or so just for tests currently
>>>>>> failing on trunk.  If you're a regular contributor, you already know
>>>> that
>>>>>> dtests are flakey - it'd be great if a few of us can go through and fix
>>>> a
>>>>>> few. Even incremental improvements are improvements. Here's an easy
>>>> search
>>>>>> to find them:
>>>>>> 
>>>>>> https://issues.apache.org/jira/secure/IssueNavigator.
>>>> jspa?reset=true&jqlQuery=project+%3D+CASSANDRA+AND+
>>>> component+%3D+Testing+ORDER+BY+updated+DESC%2C+priority+
>>>> DESC%2C+created+ASC&mode=hide
>>>>>> 
>>>>>> If you're a new contributor, fixing tests is often a good way to learn a
>>>>>> new part of the codebase. Many of these are dtests, which live in a
>>>>>> different repo ( https://github.com/apache/cassandra-dtest ) and are in
>>>>>> python, but have no fear, the repo has instructions for setting up and
>>>>>> running dtests(
>>>>>> https://github.com/apache/cassandra-dtest/blob/master/INSTALL.md )
>>>>>> 
>>>>>> Normal contribution workflow applies: self-assign the ticket if you
>>>> want to
>>>>>> work on it, click on 'start progress' to indicate that you're working on
>>>>>> it, mark it 'patch available' when you've uploaded code to be reviewed
>>>> (in
>>>>>> a github branch, or as a standalone patch file attached to the JIRA). If
>>>>>> you have questions, feel free to email the dev list (that's what it's
>>>> here
>>>>>> for).
>>>>>> 
>>>>>> Many thanks will be given,
>>>>>> - Jeff
>>>> 
>>>> 
>>> 
>> 
>> 
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
>> For additional commands, e-mail: dev-help@cassandra.apache.org
>> 
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> For additional commands, e-mail: dev-help@cassandra.apache.org
>

Re: Flakey Dtests

Posted by Michael Shuler <mi...@pbandjelly.org>.

The `test-cdc` target is not a dependent of `test-all`, so it was set up
as a separate job in Jenkins:
https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-3.11-test-cdc/
https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-trunk-test-cdc/

-- 
Michael

On 11/27/2017 03:45 PM, Michael Kjellman wrote:
> Hey Jay:
> 
> Thanks!! I just took a quick look at the JIRA and noticed that there is a “test-cdc” ant target? So, does that mean CDC get’s no testing with ant test? Do you know any of the history around this?
> 
>> On Nov 27, 2017, at 9:44 AM, Jay Zhuang <ja...@yahoo.com.INVALID> wrote:
>>
>> I fixed one CDC uTest, please review:https://issues.apache.org/jira/browse/CASSANDRA-14066
>>
>>
>>    On Friday, November 17, 2017 6:34 AM, Josh McKenzie <jm...@apache.org> wrote:
>>
>>
>>>
>>> Do we have any volunteers to fix the broken Materialized Views and CDC
>>> DTests?
>>
>> I'll try to take a look at the CDC tests next week; looks like one of the
>> base unit tests is failing as well.
>>
>> On Fri, Nov 17, 2017 at 12:09 AM, Michael Kjellman <
>> mkjellman@internalcircle.com> wrote:
>>
>>> Quick update re: dtests and off-heap memtables:
>>>
>>> I’ve filed CASSANDRA-14056 (Many dtests fail with ConfigurationException:
>>> offheap_objects are not available in 3.0 when OFFHEAP_MEMTABLES=“true”)
>>>
>>> Looks like we’re gonna need to do some work to test this configuration and
>>> right now it’s pretty broken...
>>>
>>> Do we have any volunteers to fix the broken Materialized Views and CDC
>>> DTests?
>>>
>>> best,
>>> kjellman
>>>
>>>
>>>> On Nov 15, 2017, at 5:59 PM, Michael Kjellman <
>>> mkjellman@internalcircle.com> wrote:
>>>>
>>>> yes - true- some are flaky, but almost all of the ones i filed fail 100%
>>> (💯) of the time. i look forward to triaging just the remaining flaky ones
>>> (hopefully - without powers combined - by the end of this month!!)
>>>>
>>>> appreciate everyone’s help - no matter how small... i already personally
>>> did a few “fun” random-python-class-is-missing-return-after-method stuff.
>>>>
>>>> we’ve wanted this for a while and now is our time to actually execute
>>> and make good on our previous dev list promises.
>>>>
>>>> best,
>>>> kjellman
>>>>
>>>>> On Nov 15, 2017, at 5:45 PM, Jeff Jirsa <jj...@gmail.com> wrote:
>>>>>
>>>>> In lieu of a weekly wrap-up, here's a pre-Thanksgiving call for help.
>>>>>
>>>>> If you haven't been paying attention to JIRA, you likely didn't notice
>>> that
>>>>> Josh went through and triage/categorized a bunch of issues by adding
>>>>> components, and Michael took the time to open a bunch of JIRAs for
>>> failing
>>>>> tests.
>>>>>
>>>>> How many is a bunch? Something like 35 or so just for tests currently
>>>>> failing on trunk.  If you're a regular contributor, you already know
>>> that
>>>>> dtests are flakey - it'd be great if a few of us can go through and fix
>>> a
>>>>> few. Even incremental improvements are improvements. Here's an easy
>>> search
>>>>> to find them:
>>>>>
>>>>> https://issues.apache.org/jira/secure/IssueNavigator.
>>> jspa?reset=true&jqlQuery=project+%3D+CASSANDRA+AND+
>>> component+%3D+Testing+ORDER+BY+updated+DESC%2C+priority+
>>> DESC%2C+created+ASC&mode=hide
>>>>>
>>>>> If you're a new contributor, fixing tests is often a good way to learn a
>>>>> new part of the codebase. Many of these are dtests, which live in a
>>>>> different repo ( https://github.com/apache/cassandra-dtest ) and are in
>>>>> python, but have no fear, the repo has instructions for setting up and
>>>>> running dtests(
>>>>> https://github.com/apache/cassandra-dtest/blob/master/INSTALL.md )
>>>>>
>>>>> Normal contribution workflow applies: self-assign the ticket if you
>>> want to
>>>>> work on it, click on 'start progress' to indicate that you're working on
>>>>> it, mark it 'patch available' when you've uploaded code to be reviewed
>>> (in
>>>>> a github branch, or as a standalone patch file attached to the JIRA). If
>>>>> you have questions, feel free to email the dev list (that's what it's
>>> here
>>>>> for).
>>>>>
>>>>> Many thanks will be given,
>>>>> - Jeff
>>>
>>>
>>
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> For additional commands, e-mail: dev-help@cassandra.apache.org
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
For additional commands, e-mail: dev-help@cassandra.apache.org

Re: Flakey Dtests

Posted by Michael Kjellman <mk...@internalcircle.com>.

Hey Jay:

Thanks!! I just took a quick look at the JIRA and noticed that there is a “test-cdc” ant target? So, does that mean CDC get’s no testing with ant test? Do you know any of the history around this?

> On Nov 27, 2017, at 9:44 AM, Jay Zhuang <ja...@yahoo.com.INVALID> wrote:
> 
> I fixed one CDC uTest, please review:https://issues.apache.org/jira/browse/CASSANDRA-14066
> 
> 
>    On Friday, November 17, 2017 6:34 AM, Josh McKenzie <jm...@apache.org> wrote:
> 
> 
>> 
>> Do we have any volunteers to fix the broken Materialized Views and CDC
>> DTests?
> 
> I'll try to take a look at the CDC tests next week; looks like one of the
> base unit tests is failing as well.
> 
> On Fri, Nov 17, 2017 at 12:09 AM, Michael Kjellman <
> mkjellman@internalcircle.com> wrote:
> 
>> Quick update re: dtests and off-heap memtables:
>> 
>> I’ve filed CASSANDRA-14056 (Many dtests fail with ConfigurationException:
>> offheap_objects are not available in 3.0 when OFFHEAP_MEMTABLES=“true”)
>> 
>> Looks like we’re gonna need to do some work to test this configuration and
>> right now it’s pretty broken...
>> 
>> Do we have any volunteers to fix the broken Materialized Views and CDC
>> DTests?
>> 
>> best,
>> kjellman
>> 
>> 
>>> On Nov 15, 2017, at 5:59 PM, Michael Kjellman <
>> mkjellman@internalcircle.com> wrote:
>>> 
>>> yes - true- some are flaky, but almost all of the ones i filed fail 100%
>> (💯) of the time. i look forward to triaging just the remaining flaky ones
>> (hopefully - without powers combined - by the end of this month!!)
>>> 
>>> appreciate everyone’s help - no matter how small... i already personally
>> did a few “fun” random-python-class-is-missing-return-after-method stuff.
>>> 
>>> we’ve wanted this for a while and now is our time to actually execute
>> and make good on our previous dev list promises.
>>> 
>>> best,
>>> kjellman
>>> 
>>>> On Nov 15, 2017, at 5:45 PM, Jeff Jirsa <jj...@gmail.com> wrote:
>>>> 
>>>> In lieu of a weekly wrap-up, here's a pre-Thanksgiving call for help.
>>>> 
>>>> If you haven't been paying attention to JIRA, you likely didn't notice
>> that
>>>> Josh went through and triage/categorized a bunch of issues by adding
>>>> components, and Michael took the time to open a bunch of JIRAs for
>> failing
>>>> tests.
>>>> 
>>>> How many is a bunch? Something like 35 or so just for tests currently
>>>> failing on trunk.  If you're a regular contributor, you already know
>> that
>>>> dtests are flakey - it'd be great if a few of us can go through and fix
>> a
>>>> few. Even incremental improvements are improvements. Here's an easy
>> search
>>>> to find them:
>>>> 
>>>> https://issues.apache.org/jira/secure/IssueNavigator.
>> jspa?reset=true&jqlQuery=project+%3D+CASSANDRA+AND+
>> component+%3D+Testing+ORDER+BY+updated+DESC%2C+priority+
>> DESC%2C+created+ASC&mode=hide
>>>> 
>>>> If you're a new contributor, fixing tests is often a good way to learn a
>>>> new part of the codebase. Many of these are dtests, which live in a
>>>> different repo ( https://github.com/apache/cassandra-dtest ) and are in
>>>> python, but have no fear, the repo has instructions for setting up and
>>>> running dtests(
>>>> https://github.com/apache/cassandra-dtest/blob/master/INSTALL.md )
>>>> 
>>>> Normal contribution workflow applies: self-assign the ticket if you
>> want to
>>>> work on it, click on 'start progress' to indicate that you're working on
>>>> it, mark it 'patch available' when you've uploaded code to be reviewed
>> (in
>>>> a github branch, or as a standalone patch file attached to the JIRA). If
>>>> you have questions, feel free to email the dev list (that's what it's
>> here
>>>> for).
>>>> 
>>>> Many thanks will be given,
>>>> - Jeff
>> 
>> 
>

Re: Flakey Dtests

Posted by Jay Zhuang <ja...@yahoo.com.INVALID>.

I fixed one CDC uTest, please review:https://issues.apache.org/jira/browse/CASSANDRA-14066


    On Friday, November 17, 2017 6:34 AM, Josh McKenzie <jm...@apache.org> wrote:
 

 >
> Do we have any volunteers to fix the broken Materialized Views and CDC
> DTests?

I'll try to take a look at the CDC tests next week; looks like one of the
base unit tests is failing as well.

On Fri, Nov 17, 2017 at 12:09 AM, Michael Kjellman <
mkjellman@internalcircle.com> wrote:

> Quick update re: dtests and off-heap memtables:
>
> I’ve filed CASSANDRA-14056 (Many dtests fail with ConfigurationException:
> offheap_objects are not available in 3.0 when OFFHEAP_MEMTABLES=“true”)
>
> Looks like we’re gonna need to do some work to test this configuration and
> right now it’s pretty broken...
>
> Do we have any volunteers to fix the broken Materialized Views and CDC
> DTests?
>
> best,
> kjellman
>
>
> > On Nov 15, 2017, at 5:59 PM, Michael Kjellman <
> mkjellman@internalcircle.com> wrote:
> >
> > yes - true- some are flaky, but almost all of the ones i filed fail 100%
> (💯) of the time. i look forward to triaging just the remaining flaky ones
> (hopefully - without powers combined - by the end of this month!!)
> >
> > appreciate everyone’s help - no matter how small... i already personally
> did a few “fun” random-python-class-is-missing-return-after-method stuff.
> >
> > we’ve wanted this for a while and now is our time to actually execute
> and make good on our previous dev list promises.
> >
> > best,
> > kjellman
> >
> >> On Nov 15, 2017, at 5:45 PM, Jeff Jirsa <jj...@gmail.com> wrote:
> >>
> >> In lieu of a weekly wrap-up, here's a pre-Thanksgiving call for help.
> >>
> >> If you haven't been paying attention to JIRA, you likely didn't notice
> that
> >> Josh went through and triage/categorized a bunch of issues by adding
> >> components, and Michael took the time to open a bunch of JIRAs for
> failing
> >> tests.
> >>
> >> How many is a bunch? Something like 35 or so just for tests currently
> >> failing on trunk.  If you're a regular contributor, you already know
> that
> >> dtests are flakey - it'd be great if a few of us can go through and fix
> a
> >> few. Even incremental improvements are improvements. Here's an easy
> search
> >> to find them:
> >>
> >> https://issues.apache.org/jira/secure/IssueNavigator.
> jspa?reset=true&jqlQuery=project+%3D+CASSANDRA+AND+
> component+%3D+Testing+ORDER+BY+updated+DESC%2C+priority+
> DESC%2C+created+ASC&mode=hide
> >>
> >> If you're a new contributor, fixing tests is often a good way to learn a
> >> new part of the codebase. Many of these are dtests, which live in a
> >> different repo ( https://github.com/apache/cassandra-dtest ) and are in
> >> python, but have no fear, the repo has instructions for setting up and
> >> running dtests(
> >> https://github.com/apache/cassandra-dtest/blob/master/INSTALL.md )
> >>
> >> Normal contribution workflow applies: self-assign the ticket if you
> want to
> >> work on it, click on 'start progress' to indicate that you're working on
> >> it, mark it 'patch available' when you've uploaded code to be reviewed
> (in
> >> a github branch, or as a standalone patch file attached to the JIRA). If
> >> you have questions, feel free to email the dev list (that's what it's
> here
> >> for).
> >>
> >> Many thanks will be given,
> >> - Jeff
>
>

Re: Flakey Dtests

Posted by kurt greaves <ku...@instaclustr.com>.

Upgrade tests are probably broken because we haven't been running them
since the move to ASF jenkins (I believe). I'll start having a look at some
MV test failures.

On 17 November 2017 at 14:34, Josh McKenzie <jm...@apache.org> wrote:

> >
> > Do we have any volunteers to fix the broken Materialized Views and CDC
> > DTests?
>
> I'll try to take a look at the CDC tests next week; looks like one of the
> base unit tests is failing as well.
>
> On Fri, Nov 17, 2017 at 12:09 AM, Michael Kjellman <
> mkjellman@internalcircle.com> wrote:
>
> > Quick update re: dtests and off-heap memtables:
> >
> > I’ve filed CASSANDRA-14056 (Many dtests fail with ConfigurationException:
> > offheap_objects are not available in 3.0 when OFFHEAP_MEMTABLES=“true”)
> >
> > Looks like we’re gonna need to do some work to test this configuration
> and
> > right now it’s pretty broken...
> >
> > Do we have any volunteers to fix the broken Materialized Views and CDC
> > DTests?
> >
> > best,
> > kjellman
> >
> >
> > > On Nov 15, 2017, at 5:59 PM, Michael Kjellman <
> > mkjellman@internalcircle.com> wrote:
> > >
> > > yes - true- some are flaky, but almost all of the ones i filed fail
> 100%
> > (💯) of the time. i look forward to triaging just the remaining flaky
> ones
> > (hopefully - without powers combined - by the end of this month!!)
> > >
> > > appreciate everyone’s help - no matter how small... i already
> personally
> > did a few “fun” random-python-class-is-missing-return-after-method
> stuff.
> > >
> > > we’ve wanted this for a while and now is our time to actually execute
> > and make good on our previous dev list promises.
> > >
> > > best,
> > > kjellman
> > >
> > >> On Nov 15, 2017, at 5:45 PM, Jeff Jirsa <jj...@gmail.com> wrote:
> > >>
> > >> In lieu of a weekly wrap-up, here's a pre-Thanksgiving call for help.
> > >>
> > >> If you haven't been paying attention to JIRA, you likely didn't notice
> > that
> > >> Josh went through and triage/categorized a bunch of issues by adding
> > >> components, and Michael took the time to open a bunch of JIRAs for
> > failing
> > >> tests.
> > >>
> > >> How many is a bunch? Something like 35 or so just for tests currently
> > >> failing on trunk.  If you're a regular contributor, you already know
> > that
> > >> dtests are flakey - it'd be great if a few of us can go through and
> fix
> > a
> > >> few. Even incremental improvements are improvements. Here's an easy
> > search
> > >> to find them:
> > >>
> > >> https://issues.apache.org/jira/secure/IssueNavigator.
> > jspa?reset=true&jqlQuery=project+%3D+CASSANDRA+AND+
> > component+%3D+Testing+ORDER+BY+updated+DESC%2C+priority+
> > DESC%2C+created+ASC&mode=hide
> > >>
> > >> If you're a new contributor, fixing tests is often a good way to
> learn a
> > >> new part of the codebase. Many of these are dtests, which live in a
> > >> different repo ( https://github.com/apache/cassandra-dtest ) and are
> in
> > >> python, but have no fear, the repo has instructions for setting up and
> > >> running dtests(
> > >> https://github.com/apache/cassandra-dtest/blob/master/INSTALL.md )
> > >>
> > >> Normal contribution workflow applies: self-assign the ticket if you
> > want to
> > >> work on it, click on 'start progress' to indicate that you're working
> on
> > >> it, mark it 'patch available' when you've uploaded code to be reviewed
> > (in
> > >> a github branch, or as a standalone patch file attached to the JIRA).
> If
> > >> you have questions, feel free to email the dev list (that's what it's
> > here
> > >> for).
> > >>
> > >> Many thanks will be given,
> > >> - Jeff
> >
> >
>

Re: Flakey Dtests

Posted by Josh McKenzie <jm...@apache.org>.

>
> Do we have any volunteers to fix the broken Materialized Views and CDC
> DTests?

I'll try to take a look at the CDC tests next week; looks like one of the
base unit tests is failing as well.

On Fri, Nov 17, 2017 at 12:09 AM, Michael Kjellman <
mkjellman@internalcircle.com> wrote:

> Quick update re: dtests and off-heap memtables:
>
> I’ve filed CASSANDRA-14056 (Many dtests fail with ConfigurationException:
> offheap_objects are not available in 3.0 when OFFHEAP_MEMTABLES=“true”)
>
> Looks like we’re gonna need to do some work to test this configuration and
> right now it’s pretty broken...
>
> Do we have any volunteers to fix the broken Materialized Views and CDC
> DTests?
>
> best,
> kjellman
>
>
> > On Nov 15, 2017, at 5:59 PM, Michael Kjellman <
> mkjellman@internalcircle.com> wrote:
> >
> > yes - true- some are flaky, but almost all of the ones i filed fail 100%
> (💯) of the time. i look forward to triaging just the remaining flaky ones
> (hopefully - without powers combined - by the end of this month!!)
> >
> > appreciate everyone’s help - no matter how small... i already personally
> did a few “fun” random-python-class-is-missing-return-after-method stuff.
> >
> > we’ve wanted this for a while and now is our time to actually execute
> and make good on our previous dev list promises.
> >
> > best,
> > kjellman
> >
> >> On Nov 15, 2017, at 5:45 PM, Jeff Jirsa <jj...@gmail.com> wrote:
> >>
> >> In lieu of a weekly wrap-up, here's a pre-Thanksgiving call for help.
> >>
> >> If you haven't been paying attention to JIRA, you likely didn't notice
> that
> >> Josh went through and triage/categorized a bunch of issues by adding
> >> components, and Michael took the time to open a bunch of JIRAs for
> failing
> >> tests.
> >>
> >> How many is a bunch? Something like 35 or so just for tests currently
> >> failing on trunk.  If you're a regular contributor, you already know
> that
> >> dtests are flakey - it'd be great if a few of us can go through and fix
> a
> >> few. Even incremental improvements are improvements. Here's an easy
> search
> >> to find them:
> >>
> >> https://issues.apache.org/jira/secure/IssueNavigator.
> jspa?reset=true&jqlQuery=project+%3D+CASSANDRA+AND+
> component+%3D+Testing+ORDER+BY+updated+DESC%2C+priority+
> DESC%2C+created+ASC&mode=hide
> >>
> >> If you're a new contributor, fixing tests is often a good way to learn a
> >> new part of the codebase. Many of these are dtests, which live in a
> >> different repo ( https://github.com/apache/cassandra-dtest ) and are in
> >> python, but have no fear, the repo has instructions for setting up and
> >> running dtests(
> >> https://github.com/apache/cassandra-dtest/blob/master/INSTALL.md )
> >>
> >> Normal contribution workflow applies: self-assign the ticket if you
> want to
> >> work on it, click on 'start progress' to indicate that you're working on
> >> it, mark it 'patch available' when you've uploaded code to be reviewed
> (in
> >> a github branch, or as a standalone patch file attached to the JIRA). If
> >> you have questions, feel free to email the dev list (that's what it's
> here
> >> for).
> >>
> >> Many thanks will be given,
> >> - Jeff
>
>

Re: Flakey Dtests

Posted by Michael Kjellman <mk...@internalcircle.com>.

Quick update re: dtests and off-heap memtables:

I’ve filed CASSANDRA-14056 (Many dtests fail with ConfigurationException: offheap_objects are not available in 3.0 when OFFHEAP_MEMTABLES=“true”)

Looks like we’re gonna need to do some work to test this configuration and right now it’s pretty broken...

Do we have any volunteers to fix the broken Materialized Views and CDC DTests?

best,
kjellman


> On Nov 15, 2017, at 5:59 PM, Michael Kjellman <mk...@internalcircle.com> wrote:
> 
> yes - true- some are flaky, but almost all of the ones i filed fail 100% (💯) of the time. i look forward to triaging just the remaining flaky ones (hopefully - without powers combined - by the end of this month!!)
> 
> appreciate everyone’s help - no matter how small... i already personally did a few “fun” random-python-class-is-missing-return-after-method stuff. 
> 
> we’ve wanted this for a while and now is our time to actually execute and make good on our previous dev list promises. 
> 
> best,
> kjellman
> 
>> On Nov 15, 2017, at 5:45 PM, Jeff Jirsa <jj...@gmail.com> wrote:
>> 
>> In lieu of a weekly wrap-up, here's a pre-Thanksgiving call for help.
>> 
>> If you haven't been paying attention to JIRA, you likely didn't notice that
>> Josh went through and triage/categorized a bunch of issues by adding
>> components, and Michael took the time to open a bunch of JIRAs for failing
>> tests.
>> 
>> How many is a bunch? Something like 35 or so just for tests currently
>> failing on trunk.  If you're a regular contributor, you already know that
>> dtests are flakey - it'd be great if a few of us can go through and fix a
>> few. Even incremental improvements are improvements. Here's an easy search
>> to find them:
>> 
>> https://issues.apache.org/jira/secure/IssueNavigator.jspa?reset=true&jqlQuery=project+%3D+CASSANDRA+AND+component+%3D+Testing+ORDER+BY+updated+DESC%2C+priority+DESC%2C+created+ASC&mode=hide
>> 
>> If you're a new contributor, fixing tests is often a good way to learn a
>> new part of the codebase. Many of these are dtests, which live in a
>> different repo ( https://github.com/apache/cassandra-dtest ) and are in
>> python, but have no fear, the repo has instructions for setting up and
>> running dtests(
>> https://github.com/apache/cassandra-dtest/blob/master/INSTALL.md )
>> 
>> Normal contribution workflow applies: self-assign the ticket if you want to
>> work on it, click on 'start progress' to indicate that you're working on
>> it, mark it 'patch available' when you've uploaded code to be reviewed (in
>> a github branch, or as a standalone patch file attached to the JIRA). If
>> you have questions, feel free to email the dev list (that's what it's here
>> for).
>> 
>> Many thanks will be given,
>> - Jeff

Re: Flakey Dtests

Posted by Michael Kjellman <mk...@internalcircle.com>.

yes - true- some are flaky, but almost all of the ones i filed fail 100% (💯) of the time. i look forward to triaging just the remaining flaky ones (hopefully - without powers combined - by the end of this month!!)

appreciate everyone’s help - no matter how small... i already personally did a few “fun” random-python-class-is-missing-return-after-method stuff. 

we’ve wanted this for a while and now is our time to actually execute and make good on our previous dev list promises. 

best,
kjellman

> On Nov 15, 2017, at 5:45 PM, Jeff Jirsa <jj...@gmail.com> wrote:
> 
> In lieu of a weekly wrap-up, here's a pre-Thanksgiving call for help.
> 
> If you haven't been paying attention to JIRA, you likely didn't notice that
> Josh went through and triage/categorized a bunch of issues by adding
> components, and Michael took the time to open a bunch of JIRAs for failing
> tests.
> 
> How many is a bunch? Something like 35 or so just for tests currently
> failing on trunk.  If you're a regular contributor, you already know that
> dtests are flakey - it'd be great if a few of us can go through and fix a
> few. Even incremental improvements are improvements. Here's an easy search
> to find them:
> 
> https://issues.apache.org/jira/secure/IssueNavigator.jspa?reset=true&jqlQuery=project+%3D+CASSANDRA+AND+component+%3D+Testing+ORDER+BY+updated+DESC%2C+priority+DESC%2C+created+ASC&mode=hide
> 
> If you're a new contributor, fixing tests is often a good way to learn a
> new part of the codebase. Many of these are dtests, which live in a
> different repo ( https://github.com/apache/cassandra-dtest ) and are in
> python, but have no fear, the repo has instructions for setting up and
> running dtests(
> https://github.com/apache/cassandra-dtest/blob/master/INSTALL.md )
> 
> Normal contribution workflow applies: self-assign the ticket if you want to
> work on it, click on 'start progress' to indicate that you're working on
> it, mark it 'patch available' when you've uploaded code to be reviewed (in
> a github branch, or as a standalone patch file attached to the JIRA). If
> you have questions, feel free to email the dev list (that's what it's here
> for).
> 
> Many thanks will be given,
> - Jeff

Re: Flakey Dtests

Posted by Jeff Jirsa <jj...@gmail.com>.

Happily! Assigned. 

-- 
Jeff Jirsa


> On Nov 22, 2017, at 6:04 AM, sam sriramadhesikan <sa...@oracle.com> wrote:
> 
> Hi Jeff,
> 
> I am new to the Cassandra project. I would like to take on one of the dtest tickets as my first contribution. Could you or Michael assign -
> 
> https://issues.apache.org/jira/browse/CASSANDRA-14050 <https://issues.apache.org/jira/browse/CASSANDRA-14050>
> 
> - to me? My ASF Jira ID is ‘sasrira’, and I don’t see an Assign action / button to self-assign.
> 
> Thanks,
> 
> Sam
> 
> 
>> On Nov 15, 2017, at 8:44 PM, Jeff Jirsa <jj...@gmail.com> wrote:
>> 
>> In lieu of a weekly wrap-up, here's a pre-Thanksgiving call for help.
>> 
>> If you haven't been paying attention to JIRA, you likely didn't notice that
>> Josh went through and triage/categorized a bunch of issues by adding
>> components, and Michael took the time to open a bunch of JIRAs for failing
>> tests.
>> 
>> How many is a bunch? Something like 35 or so just for tests currently
>> failing on trunk.  If you're a regular contributor, you already know that
>> dtests are flakey - it'd be great if a few of us can go through and fix a
>> few. Even incremental improvements are improvements. Here's an easy search
>> to find them:
>> 
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.apache.org_jira_secure_IssueNavigator.jspa-3Freset-3Dtrue-26jqlQuery-3Dproject-2B-253D-2BCASSANDRA-2BAND-2Bcomponent-2B-253D-2BTesting-2BORDER-2BBY-2Bupdated-2BDESC-252C-2Bpriority-2BDESC-252C-2Bcreated-2BASC-26mode-3Dhide&d=DwIBaQ&c=RoP1YumCXCgaWHvlZYR8PZh8Bv7qIrMUB65eapI_JnE&r=AJfDCyM3u5-XbEgygLPEjU2yutePmNCLC8Dp7T2lLug&m=SmgwbaoCFiFaXHHGAOrFYLZOKtmtPgubix3dX8rrEAU&s=tJg9omGEdQlCIRv_TUZoLqcv5E6iQ--t596WYExrx3I&e=
>> 
>> If you're a new contributor, fixing tests is often a good way to learn a
>> new part of the codebase. Many of these are dtests, which live in a
>> different repo ( https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_cassandra-2Ddtest&d=DwIBaQ&c=RoP1YumCXCgaWHvlZYR8PZh8Bv7qIrMUB65eapI_JnE&r=AJfDCyM3u5-XbEgygLPEjU2yutePmNCLC8Dp7T2lLug&m=SmgwbaoCFiFaXHHGAOrFYLZOKtmtPgubix3dX8rrEAU&s=xP4RA6TkxUM-EMoZBxtR5esHkAXpqitQv2yJPqbKs-k&e= ) and are in
>> python, but have no fear, the repo has instructions for setting up and
>> running dtests(
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_cassandra-2Ddtest_blob_master_INSTALL.md&d=DwIBaQ&c=RoP1YumCXCgaWHvlZYR8PZh8Bv7qIrMUB65eapI_JnE&r=AJfDCyM3u5-XbEgygLPEjU2yutePmNCLC8Dp7T2lLug&m=SmgwbaoCFiFaXHHGAOrFYLZOKtmtPgubix3dX8rrEAU&s=8iDBkJzCYOwJi7eJlsKXbIt_RV4ziixhIVkkEi40p7U&e= )
>> 
>> Normal contribution workflow applies: self-assign the ticket if you want to
>> work on it, click on 'start progress' to indicate that you're working on
>> it, mark it 'patch available' when you've uploaded code to be reviewed (in
>> a github branch, or as a standalone patch file attached to the JIRA). If
>> you have questions, feel free to email the dev list (that's what it's here
>> for).
>> 
>> Many thanks will be given,
>> - Jeff
> 

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
For additional commands, e-mail: dev-help@cassandra.apache.org

Re: Flakey Dtests

Posted by sam sriramadhesikan <sa...@oracle.com>.

Hi Jeff,

I am new to the Cassandra project. I would like to take on one of the dtest tickets as my first contribution. Could you or Michael assign -

https://issues.apache.org/jira/browse/CASSANDRA-14050 <https://issues.apache.org/jira/browse/CASSANDRA-14050>

- to me? My ASF Jira ID is ‘sasrira’, and I don’t see an Assign action / button to self-assign.

Thanks,

Sam


> On Nov 15, 2017, at 8:44 PM, Jeff Jirsa <jj...@gmail.com> wrote:
> 
> In lieu of a weekly wrap-up, here's a pre-Thanksgiving call for help.
> 
> If you haven't been paying attention to JIRA, you likely didn't notice that
> Josh went through and triage/categorized a bunch of issues by adding
> components, and Michael took the time to open a bunch of JIRAs for failing
> tests.
> 
> How many is a bunch? Something like 35 or so just for tests currently
> failing on trunk.  If you're a regular contributor, you already know that
> dtests are flakey - it'd be great if a few of us can go through and fix a
> few. Even incremental improvements are improvements. Here's an easy search
> to find them:
> 
> https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.apache.org_jira_secure_IssueNavigator.jspa-3Freset-3Dtrue-26jqlQuery-3Dproject-2B-253D-2BCASSANDRA-2BAND-2Bcomponent-2B-253D-2BTesting-2BORDER-2BBY-2Bupdated-2BDESC-252C-2Bpriority-2BDESC-252C-2Bcreated-2BASC-26mode-3Dhide&d=DwIBaQ&c=RoP1YumCXCgaWHvlZYR8PZh8Bv7qIrMUB65eapI_JnE&r=AJfDCyM3u5-XbEgygLPEjU2yutePmNCLC8Dp7T2lLug&m=SmgwbaoCFiFaXHHGAOrFYLZOKtmtPgubix3dX8rrEAU&s=tJg9omGEdQlCIRv_TUZoLqcv5E6iQ--t596WYExrx3I&e=
> 
> If you're a new contributor, fixing tests is often a good way to learn a
> new part of the codebase. Many of these are dtests, which live in a
> different repo ( https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_cassandra-2Ddtest&d=DwIBaQ&c=RoP1YumCXCgaWHvlZYR8PZh8Bv7qIrMUB65eapI_JnE&r=AJfDCyM3u5-XbEgygLPEjU2yutePmNCLC8Dp7T2lLug&m=SmgwbaoCFiFaXHHGAOrFYLZOKtmtPgubix3dX8rrEAU&s=xP4RA6TkxUM-EMoZBxtR5esHkAXpqitQv2yJPqbKs-k&e= ) and are in
> python, but have no fear, the repo has instructions for setting up and
> running dtests(
> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_cassandra-2Ddtest_blob_master_INSTALL.md&d=DwIBaQ&c=RoP1YumCXCgaWHvlZYR8PZh8Bv7qIrMUB65eapI_JnE&r=AJfDCyM3u5-XbEgygLPEjU2yutePmNCLC8Dp7T2lLug&m=SmgwbaoCFiFaXHHGAOrFYLZOKtmtPgubix3dX8rrEAU&s=8iDBkJzCYOwJi7eJlsKXbIt_RV4ziixhIVkkEi40p7U&e= )
> 
> Normal contribution workflow applies: self-assign the ticket if you want to
> work on it, click on 'start progress' to indicate that you're working on
> it, mark it 'patch available' when you've uploaded code to be reviewed (in
> a github branch, or as a standalone patch file attached to the JIRA). If
> you have questions, feel free to email the dev list (that's what it's here
> for).
> 
> Many thanks will be given,
> - Jeff