You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flink.apache.org by "Matthias J. Sax" <mj...@apache.org> on 2015/10/09 13:47:12 UTC
[DISCUSS] Collecting Instable Tests
Hi,
right now we collect instable tests via labeling them with "test-stability".
I think it would be a better approach, to use an Umbrella Issue in JIRA
and collect them as Sub-Tasks. Storm does the same thing:
https://issues.apache.org/jira/browse/STORM-915
This would have the advantage, that we get a simpler overview what tests
got already reported (and/or fixed). We can also list the tests by
component which makes it easier to find (the JIRA "search issues"
feature is cumbersome to use because the UI is quite slow)
I would keep current approach and label the sub-tasks with
"test-stability" and mark them as "critical".
What do you think about it?
-Matthias
Re: [DISCUSS] Collecting Instable Tests
Posted by Maximilian Michels <mx...@apache.org>.
In terms of usability, you might have a point Matthias. In terms of
JIRA workflow, I'd say that it makes more sense to tag and filter. An
issue with subtasks should be resolvable. Test instability, on the
other hand, requires a continuous effort and therefore not really
applicable to the concept of an issue. An "Umbrella" issue is not a
standard issue type and seems more like a hack to me.
On Fri, Oct 9, 2015 at 2:56 PM, Márton Balassi <ba...@gmail.com> wrote:
> I like Fabian's approach, you can also share these filters on JIRA.
>
> On Fri, Oct 9, 2015 at 2:55 PM, Fabian Hueske <fh...@gmail.com> wrote:
>
>> Sorry, that was not the correct link.
>> You can create a bookmark for this one:
>>
>>
>> https://issues.apache.org/jira/issues/?jql=project%20%3D%20FLINK%20AND%20resolution%20%3D%20Unresolved%20AND%20labels%20%3D%20test-stability%20ORDER%20BY%20priority%20DESC
>>
>> and directly ask for the list without searching.
>>
>> 2015-10-09 14:48 GMT+02:00 Matthias J. Sax <mj...@apache.org>:
>>
>> > Yes, but this filtering is super slow (or is it just my browser?)
>> >
>> > I personally don't like the search feature in JIRA and an umbrella would
>> > give a easier to access overview.
>> >
>> > But if it's just me, never mind ;)
>> >
>> >
>> > On 10/09/2015 02:39 PM, Fabian Hueske wrote:
>> > > What would be the benefit over the current approach?
>> > > You can just fiter JIRA for issues with a specific label:
>> > >
>> > >
>> >
>> https://issues.apache.org/jira/browse/FLINK-2771?jql=project%20%3D%20FLINK%20AND%20resolution%20%3D%20Unresolved%20AND%20labels%20%3D%20test-stability%20ORDER%20BY%20priority%20DESC
>> > >
>> > > Not sure if the gain (if there is any) justifies the overhead of adding
>> > > issues to an umbrella issue.
>> > >
>> > >
>> > > 2015-10-09 13:47 GMT+02:00 Matthias J. Sax <mj...@apache.org>:
>> > >
>> > >> Hi,
>> > >>
>> > >> right now we collect instable tests via labeling them with
>> > >> "test-stability".
>> > >>
>> > >> I think it would be a better approach, to use an Umbrella Issue in
>> JIRA
>> > >> and collect them as Sub-Tasks. Storm does the same thing:
>> > >> https://issues.apache.org/jira/browse/STORM-915
>> > >>
>> > >> This would have the advantage, that we get a simpler overview what
>> tests
>> > >> got already reported (and/or fixed). We can also list the tests by
>> > >> component which makes it easier to find (the JIRA "search issues"
>> > >> feature is cumbersome to use because the UI is quite slow)
>> > >>
>> > >> I would keep current approach and label the sub-tasks with
>> > >> "test-stability" and mark them as "critical".
>> > >>
>> > >> What do you think about it?
>> > >>
>> > >>
>> > >> -Matthias
>> > >>
>> > >>
>> > >
>> >
>> >
>>
Re: [DISCUSS] Collecting Instable Tests
Posted by Márton Balassi <ba...@gmail.com>.
I like Fabian's approach, you can also share these filters on JIRA.
On Fri, Oct 9, 2015 at 2:55 PM, Fabian Hueske <fh...@gmail.com> wrote:
> Sorry, that was not the correct link.
> You can create a bookmark for this one:
>
>
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20FLINK%20AND%20resolution%20%3D%20Unresolved%20AND%20labels%20%3D%20test-stability%20ORDER%20BY%20priority%20DESC
>
> and directly ask for the list without searching.
>
> 2015-10-09 14:48 GMT+02:00 Matthias J. Sax <mj...@apache.org>:
>
> > Yes, but this filtering is super slow (or is it just my browser?)
> >
> > I personally don't like the search feature in JIRA and an umbrella would
> > give a easier to access overview.
> >
> > But if it's just me, never mind ;)
> >
> >
> > On 10/09/2015 02:39 PM, Fabian Hueske wrote:
> > > What would be the benefit over the current approach?
> > > You can just fiter JIRA for issues with a specific label:
> > >
> > >
> >
> https://issues.apache.org/jira/browse/FLINK-2771?jql=project%20%3D%20FLINK%20AND%20resolution%20%3D%20Unresolved%20AND%20labels%20%3D%20test-stability%20ORDER%20BY%20priority%20DESC
> > >
> > > Not sure if the gain (if there is any) justifies the overhead of adding
> > > issues to an umbrella issue.
> > >
> > >
> > > 2015-10-09 13:47 GMT+02:00 Matthias J. Sax <mj...@apache.org>:
> > >
> > >> Hi,
> > >>
> > >> right now we collect instable tests via labeling them with
> > >> "test-stability".
> > >>
> > >> I think it would be a better approach, to use an Umbrella Issue in
> JIRA
> > >> and collect them as Sub-Tasks. Storm does the same thing:
> > >> https://issues.apache.org/jira/browse/STORM-915
> > >>
> > >> This would have the advantage, that we get a simpler overview what
> tests
> > >> got already reported (and/or fixed). We can also list the tests by
> > >> component which makes it easier to find (the JIRA "search issues"
> > >> feature is cumbersome to use because the UI is quite slow)
> > >>
> > >> I would keep current approach and label the sub-tasks with
> > >> "test-stability" and mark them as "critical".
> > >>
> > >> What do you think about it?
> > >>
> > >>
> > >> -Matthias
> > >>
> > >>
> > >
> >
> >
>
Re: [DISCUSS] Collecting Instable Tests
Posted by Fabian Hueske <fh...@gmail.com>.
Sorry, that was not the correct link.
You can create a bookmark for this one:
https://issues.apache.org/jira/issues/?jql=project%20%3D%20FLINK%20AND%20resolution%20%3D%20Unresolved%20AND%20labels%20%3D%20test-stability%20ORDER%20BY%20priority%20DESC
and directly ask for the list without searching.
2015-10-09 14:48 GMT+02:00 Matthias J. Sax <mj...@apache.org>:
> Yes, but this filtering is super slow (or is it just my browser?)
>
> I personally don't like the search feature in JIRA and an umbrella would
> give a easier to access overview.
>
> But if it's just me, never mind ;)
>
>
> On 10/09/2015 02:39 PM, Fabian Hueske wrote:
> > What would be the benefit over the current approach?
> > You can just fiter JIRA for issues with a specific label:
> >
> >
> https://issues.apache.org/jira/browse/FLINK-2771?jql=project%20%3D%20FLINK%20AND%20resolution%20%3D%20Unresolved%20AND%20labels%20%3D%20test-stability%20ORDER%20BY%20priority%20DESC
> >
> > Not sure if the gain (if there is any) justifies the overhead of adding
> > issues to an umbrella issue.
> >
> >
> > 2015-10-09 13:47 GMT+02:00 Matthias J. Sax <mj...@apache.org>:
> >
> >> Hi,
> >>
> >> right now we collect instable tests via labeling them with
> >> "test-stability".
> >>
> >> I think it would be a better approach, to use an Umbrella Issue in JIRA
> >> and collect them as Sub-Tasks. Storm does the same thing:
> >> https://issues.apache.org/jira/browse/STORM-915
> >>
> >> This would have the advantage, that we get a simpler overview what tests
> >> got already reported (and/or fixed). We can also list the tests by
> >> component which makes it easier to find (the JIRA "search issues"
> >> feature is cumbersome to use because the UI is quite slow)
> >>
> >> I would keep current approach and label the sub-tasks with
> >> "test-stability" and mark them as "critical".
> >>
> >> What do you think about it?
> >>
> >>
> >> -Matthias
> >>
> >>
> >
>
>
Re: [DISCUSS] Collecting Instable Tests
Posted by "Matthias J. Sax" <mj...@apache.org>.
Yes, but this filtering is super slow (or is it just my browser?)
I personally don't like the search feature in JIRA and an umbrella would
give a easier to access overview.
But if it's just me, never mind ;)
On 10/09/2015 02:39 PM, Fabian Hueske wrote:
> What would be the benefit over the current approach?
> You can just fiter JIRA for issues with a specific label:
>
> https://issues.apache.org/jira/browse/FLINK-2771?jql=project%20%3D%20FLINK%20AND%20resolution%20%3D%20Unresolved%20AND%20labels%20%3D%20test-stability%20ORDER%20BY%20priority%20DESC
>
> Not sure if the gain (if there is any) justifies the overhead of adding
> issues to an umbrella issue.
>
>
> 2015-10-09 13:47 GMT+02:00 Matthias J. Sax <mj...@apache.org>:
>
>> Hi,
>>
>> right now we collect instable tests via labeling them with
>> "test-stability".
>>
>> I think it would be a better approach, to use an Umbrella Issue in JIRA
>> and collect them as Sub-Tasks. Storm does the same thing:
>> https://issues.apache.org/jira/browse/STORM-915
>>
>> This would have the advantage, that we get a simpler overview what tests
>> got already reported (and/or fixed). We can also list the tests by
>> component which makes it easier to find (the JIRA "search issues"
>> feature is cumbersome to use because the UI is quite slow)
>>
>> I would keep current approach and label the sub-tasks with
>> "test-stability" and mark them as "critical".
>>
>> What do you think about it?
>>
>>
>> -Matthias
>>
>>
>
Re: [DISCUSS] Collecting Instable Tests
Posted by Fabian Hueske <fh...@gmail.com>.
What would be the benefit over the current approach?
You can just fiter JIRA for issues with a specific label:
https://issues.apache.org/jira/browse/FLINK-2771?jql=project%20%3D%20FLINK%20AND%20resolution%20%3D%20Unresolved%20AND%20labels%20%3D%20test-stability%20ORDER%20BY%20priority%20DESC
Not sure if the gain (if there is any) justifies the overhead of adding
issues to an umbrella issue.
2015-10-09 13:47 GMT+02:00 Matthias J. Sax <mj...@apache.org>:
> Hi,
>
> right now we collect instable tests via labeling them with
> "test-stability".
>
> I think it would be a better approach, to use an Umbrella Issue in JIRA
> and collect them as Sub-Tasks. Storm does the same thing:
> https://issues.apache.org/jira/browse/STORM-915
>
> This would have the advantage, that we get a simpler overview what tests
> got already reported (and/or fixed). We can also list the tests by
> component which makes it easier to find (the JIRA "search issues"
> feature is cumbersome to use because the UI is quite slow)
>
> I would keep current approach and label the sub-tasks with
> "test-stability" and mark them as "critical".
>
> What do you think about it?
>
>
> -Matthias
>
>