[DISCUSS] filling affected versions on JIRA issue

[Jungtaek Lim <ka...@gmail.com>]

Hi devs,

I know we're busy with making Spark 3.0 be out, but I think the topic is
good to discuss at any time and actually be better to be resolved sooner
than later.

In the page "Contributing to Spark", we describe the guide of "affects
version" as "For Bugs, assign at least one version that is known to exhibit
the problem or need the change".

For me, that sentence clearly describes minimal requirement of affects
version via:

* For the type of bug, assign one valid version
* For other types, there's no requirement

but I'm seeing the requests more than the requirement which makes me think
there might be different understanding of the sentence. Maybe there's more,
but to summarize on such requests:

1) add affects version as same as master branch for improvement/new feature
2) check with older versions to fill up affects version for bug

I don't see any point on doing 1). It might give some context if we don't
update the affect version (so that it can say which version was considered
when filing JIRA issue) but we also update the affect version when we bump
the master branch, which is no longer informational as the version should
have been always the same as master branch.

I agree it's ideal to do 2) but I think the reason the guide doesn't
enforce is that it requires pretty much efforts to check with old versions
(sometimes even more than origin work).

Suppose the happy case we have UT to verify the bugfix which fails without
the patch and passes with the patch. To check with older versions we have
to checkout the tag, and apply the UT, and "rebuild", and run UT to verify
which is pretty much time-consuming. What if there's a conflict indeed?
That's still a happy case, and in worse case (there's no such UT) we should
do E2E manual verification which I would give up.

There should have some balance/threshold, and the balance should be the
thing the community has a consensus.

Would like to hear everyone's voice on this.

Thanks,
Jungtaek Lim (HeartSaVioR)

Re: [DISCUSS] filling affected versions on JIRA issue

[Jungtaek Lim <ka...@gmail.com>]

On Fri, Apr 3, 2020 at 12:31 AM Sean Owen <sr...@gmail.com> wrote:

> On Wed, Apr 1, 2020 at 10:28 PM Jungtaek Lim
> <ka...@gmail.com> wrote:
> > The definition of "latest version" would matter, especially there's a
> time we prepare minor+ version release.
> >
> > For example, lots of people (even including committers) filed an
> "improvement" issue with setting fix version to 3.0, which is NOT incorrect
> in point of "release", but incorrect in point of the version of "master
> branch". If we say it as "latest" version, maybe they should not even be
> set to 3.0. Looks like it still confuses someone; we need to make clear
> which version it should if we really want to require it, and should be
> documented.
>
> OK shall we simply say, tag it with whatever version you were using
> when you found the bug? If the reporter or anyone else knows it
> affects other versions, sure, add that. Point being, I don't think we
> should ask people to investigate which N versions it affects, unless
> it's particularly vital to the nature of the issue.
>

The paragraph is aimed for "new feature" / "improvement", not for "bug". I
guess we have consensus for "bug".


> For improvements, it matters less, and simply saying 'the latest
> version' is a fine default.
>

I meant 'the latest version' is confusing one. If someone is not tightly
involved into Spark development then they would consider the latest version
as 'the latest released version'. Even for someone who works on Spark
development it gives confusion when we cut a branch for minor+ version,
master branch goes up to (unreleased version + 1), which some of us may
think the latest version as the minor+ version we will target to release.

It would avoid the confusion if we decide the definition of 'the latest
version', and define the standard way to get 'the latest version'. and
document into contribution guide page. Otherwise we would have different
understanding and try to guide with different version, or even try to
correct others.


> Is there another desired standard out there that we're debating
> against? so far this sounds like existing practice.
>

Existing practice is not documented and gives confusion - 3.1.0 vs 3.0.0
for now. Though I believe leaving it empty or N/A if the field is required
should be even better.


> > Also I'm not in favor of bumping affect version in existing improvement
> issues when bumping up the minor+ version. As I said, I'm not sure we get
> some benefits from there. Even more, once batch updates are executed, lots
> of notifications happen in issue@ and these issues bump to the top in
> mail inbox, whereas technically they have no actual update. I'd rather say
> we should do opposite, don't update it to leave some context which version
> it was considered.
>
> What is this referring to - there are some batch updates of affected
> version? could be fine but for what reason?
> You can disable sending an email for bulk updates in JIRA, if that's the
> issue.
>

If you're subscribing to issue@ then you've recognized there were bulk
updates after cutting the branch for 3.0. (That's not controllable by
myself unless I unsubscribe.) I didn't do bulk update by myself - as I said
I'm not in favor of updating version.

And bulk mail update is just a side-effect and not the main issue, of
course. The main point is that the benefits/values: personally I don't see
any value from doing that. If we are on the same page then why we do that,
and if someone objects then doesn't it the thing we need to discuss?

> I'm assuming that we should require the affect version even for non-bug
> issue, but yes if possible I'd in favor of leave it empty. In any way let's
> document it explicitly.
>
> Agree. I think it's required in JIRA just because we can't require it
> for Bugs but not Improvements though?
>

There's "N/A" which could simply play as a dummy marker which would avoid
confusion at any time. I'm not expert on JIRA automation (especially ASF
JIRA) but we might be able to ask ASF INFRA to do it automatically for the
some types. Even it can't be automated, still clearer to set the version.
WDYT?

Re: [DISCUSS] filling affected versions on JIRA issue

[Sean Owen <sr...@gmail.com>]

On Wed, Apr 1, 2020 at 10:28 PM Jungtaek Lim
<ka...@gmail.com> wrote:
> The definition of "latest version" would matter, especially there's a time we prepare minor+ version release.
>
> For example, lots of people (even including committers) filed an "improvement" issue with setting fix version to 3.0, which is NOT incorrect in point of "release", but incorrect in point of the version of "master branch". If we say it as "latest" version, maybe they should not even be set to 3.0. Looks like it still confuses someone; we need to make clear which version it should if we really want to require it, and should be documented.

OK shall we simply say, tag it with whatever version you were using
when you found the bug? If the reporter or anyone else knows it
affects other versions, sure, add that. Point being, I don't think we
should ask people to investigate which N versions it affects, unless
it's particularly vital to the nature of the issue.

For improvements, it matters less, and simply saying 'the latest
version' is a fine default.

Is there another desired standard out there that we're debating
against? so far this sounds like existing practice.


> Also I'm not in favor of bumping affect version in existing improvement issues when bumping up the minor+ version. As I said, I'm not sure we get some benefits from there. Even more, once batch updates are executed, lots of notifications happen in issue@ and these issues bump to the top in mail inbox, whereas technically they have no actual update. I'd rather say we should do opposite, don't update it to leave some context which version it was considered.

What is this referring to - there are some batch updates of affected
version? could be fine but for what reason?
You can disable sending an email for bulk updates in JIRA, if that's the issue.


> I'm assuming that we should require the affect version even for non-bug issue, but yes if possible I'd in favor of leave it empty. In any way let's document it explicitly.

Agree. I think it's required in JIRA just because we can't require it
for Bugs but not Improvements though?

---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org

Re: [DISCUSS] filling affected versions on JIRA issue

[Jungtaek Lim <ka...@gmail.com>]

I didn't point out actual case "intentionally", because I want to avoid
unnecessary debate and make sure we don't decide with bias. Note that the
context would include people.

I have been seen these requests consistently (at least consistently for 1,
but I feel I also saw 2 more than couple of time as well), so let's just
treat it as "trends" and discuss it in general for this mail thread. We
know there's some exceptional case and in such case we'd rather discuss it
there.

> It's not wrong to say it affects the latest version, at least.
> And I believe we require it in JIRA because we can't require an
> Affects Version for one type of issue but not another. So, just asking
> people to default to 'latest version' there is no burden.

The definition of "latest version" would matter, especially there's a time
we prepare minor+ version release.

For example, lots of people (even including committers) filed an
"improvement" issue with setting fix version to 3.0, which is NOT incorrect
in point of "release", but incorrect in point of the version of "master
branch". If we say it as "latest" version, maybe they should not even be
set to 3.0. Looks like it still confuses someone; we need to make clear
which version it should if we really want to require it, and should be
documented.

Also I'm not in favor of bumping affect version in existing improvement
issues when bumping up the minor+ version. As I said, I'm not sure we get
some benefits from there. Even more, once batch updates are executed, lots
of notifications happen in issue@ and these issues bump to the top in mail
inbox, whereas technically they have no actual update. I'd rather say we
should do opposite, don't update it to leave some context which version it
was considered.

I'm assuming that we should require the affect version even for non-bug
issue, but yes if possible I'd in favor of leave it empty. In any way let's
document it explicitly.

For bugs I guess we are on the same page - there're some details but in
general we don't require to check the old versions. I'd be OK with checking
against "latest" active version branches to evaluate the possibility of
backport, but that's all. Even for security / correctness issues we may
want to define lower versions to check - do we want to check these issues
with EOL version lines? If yes, even 1.x?


On Thu, Apr 2, 2020 at 11:43 AM Nicholas Chammas <ni...@gmail.com>
wrote:

> Probably the discussion here about Improvement Jira tickets and the
> "Affects Version" field:
> https://github.com/apache/spark/pull/27534#issuecomment-588416416
>
> On Wed, Apr 1, 2020 at 9:59 PM Hyukjin Kwon <gu...@gmail.com> wrote:
>
>> > 2) check with older versions to fill up affects version for bug
>> I don't agree with this in general. To me usually it's "For the type of
>> bug, assign one valid version" instead.
>>
>> > The only place where I can see some amount of investigation being
>> required would be for security issues or correctness issues.
>> Yes, I agree.
>>
>> Yes, was there a particular case or context that motivated this thread?
>>
>> 2020년 4월 2일 (목) 오전 10:24, Mridul Muralidharan <mr...@gmail.com>님이 작성:
>>
>>>
>>> I agree with what Sean detailed.
>>> The only place where I can see some amount of investigation being
>>> required would be for security issues or correctness issues.
>>> Knowing the affected versions, particularly if an earlier supported
>>> version does not have the bug, will help users understand the
>>> broken/insecure versions.
>>>
>>> Regards,
>>> Mridul
>>>
>>>
>>> On Wed, Apr 1, 2020 at 6:12 PM Sean Owen <sr...@gmail.com> wrote:
>>>
>>>> I think we discussed this briefly on a PR.
>>>>
>>>> It's not as clear what it means for an Improvement to 'affect a
>>>> version'. Certainly, an improvement to a feature introduced in 1.2.3
>>>> can't affect anything earlier, and implicitly affects everything
>>>> after. It's not wrong to say it affects the latest version, at least.
>>>> And I believe we require it in JIRA because we can't require an
>>>> Affects Version for one type of issue but not another. So, just asking
>>>> people to default to 'latest version' there is no burden.
>>>>
>>>> I would not ask someone to figure out all and earliest versions that
>>>> an Improvement applies to; it just isn't that useful. We aren't
>>>> generally going to back-port improvements anyway.
>>>>
>>>> Even for bugs, we don't really need to know that a bug in master
>>>> affects 2.4.5, 2.4.4, 2.4.3, ... 2.3.6, 2.3.5, etc. It doesn't hurt to
>>>> at least say it affects the latest 2.4.x, 2.3.x releases, if known,
>>>> because it's possible it should be back-ported. Again even where this
>>>> is significantly more useful, I'm not in favor of telling people they
>>>> must test the bug report vs previous releases.
>>>>
>>>> So, if you're asserting that the current guidance is OK, I generally
>>>> agree.
>>>> Is there a particular context where this was questioned? maybe we
>>>> should examine the particulars of that situation. As in all things,
>>>> context matters.
>>>>
>>>> Sean
>>>>
>>>> On Wed, Apr 1, 2020 at 7:34 PM Jungtaek Lim
>>>> <ka...@gmail.com> wrote:
>>>> >
>>>> > Hi devs,
>>>> >
>>>> > I know we're busy with making Spark 3.0 be out, but I think the topic
>>>> is good to discuss at any time and actually be better to be resolved sooner
>>>> than later.
>>>> >
>>>> > In the page "Contributing to Spark", we describe the guide of
>>>> "affects version" as "For Bugs, assign at least one version that is known
>>>> to exhibit the problem or need the change".
>>>> >
>>>> > For me, that sentence clearly describes minimal requirement of
>>>> affects version via:
>>>> >
>>>> > * For the type of bug, assign one valid version
>>>> > * For other types, there's no requirement
>>>> >
>>>> > but I'm seeing the requests more than the requirement which makes me
>>>> think there might be different understanding of the sentence. Maybe there's
>>>> more, but to summarize on such requests:
>>>> >
>>>> > 1) add affects version as same as master branch for improvement/new
>>>> feature
>>>> > 2) check with older versions to fill up affects version for bug
>>>> >
>>>> > I don't see any point on doing 1). It might give some context if we
>>>> don't update the affect version (so that it can say which version was
>>>> considered when filing JIRA issue) but we also update the affect version
>>>> when we bump the master branch, which is no longer informational as the
>>>> version should have been always the same as master branch.
>>>> >
>>>> > I agree it's ideal to do 2) but I think the reason the guide doesn't
>>>> enforce is that it requires pretty much efforts to check with old versions
>>>> (sometimes even more than origin work).
>>>> >
>>>> > Suppose the happy case we have UT to verify the bugfix which fails
>>>> without the patch and passes with the patch. To check with older versions
>>>> we have to checkout the tag, and apply the UT, and "rebuild", and run UT to
>>>> verify which is pretty much time-consuming. What if there's a conflict
>>>> indeed? That's still a happy case, and in worse case (there's no such UT)
>>>> we should do E2E manual verification which I would give up.
>>>> >
>>>> > There should have some balance/threshold, and the balance should be
>>>> the thing the community has a consensus.
>>>> >
>>>> > Would like to hear everyone's voice on this.
>>>> >
>>>> > Thanks,
>>>> > Jungtaek Lim (HeartSaVioR)
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>>>>
>>>>

Re: [DISCUSS] filling affected versions on JIRA issue

[Nicholas Chammas <ni...@gmail.com>]

Probably the discussion here about Improvement Jira tickets and the
"Affects Version" field:
https://github.com/apache/spark/pull/27534#issuecomment-588416416

On Wed, Apr 1, 2020 at 9:59 PM Hyukjin Kwon <gu...@gmail.com> wrote:

> > 2) check with older versions to fill up affects version for bug
> I don't agree with this in general. To me usually it's "For the type of
> bug, assign one valid version" instead.
>
> > The only place where I can see some amount of investigation being
> required would be for security issues or correctness issues.
> Yes, I agree.
>
> Yes, was there a particular case or context that motivated this thread?
>
> 2020년 4월 2일 (목) 오전 10:24, Mridul Muralidharan <mr...@gmail.com>님이 작성:
>
>>
>> I agree with what Sean detailed.
>> The only place where I can see some amount of investigation being
>> required would be for security issues or correctness issues.
>> Knowing the affected versions, particularly if an earlier supported
>> version does not have the bug, will help users understand the
>> broken/insecure versions.
>>
>> Regards,
>> Mridul
>>
>>
>> On Wed, Apr 1, 2020 at 6:12 PM Sean Owen <sr...@gmail.com> wrote:
>>
>>> I think we discussed this briefly on a PR.
>>>
>>> It's not as clear what it means for an Improvement to 'affect a
>>> version'. Certainly, an improvement to a feature introduced in 1.2.3
>>> can't affect anything earlier, and implicitly affects everything
>>> after. It's not wrong to say it affects the latest version, at least.
>>> And I believe we require it in JIRA because we can't require an
>>> Affects Version for one type of issue but not another. So, just asking
>>> people to default to 'latest version' there is no burden.
>>>
>>> I would not ask someone to figure out all and earliest versions that
>>> an Improvement applies to; it just isn't that useful. We aren't
>>> generally going to back-port improvements anyway.
>>>
>>> Even for bugs, we don't really need to know that a bug in master
>>> affects 2.4.5, 2.4.4, 2.4.3, ... 2.3.6, 2.3.5, etc. It doesn't hurt to
>>> at least say it affects the latest 2.4.x, 2.3.x releases, if known,
>>> because it's possible it should be back-ported. Again even where this
>>> is significantly more useful, I'm not in favor of telling people they
>>> must test the bug report vs previous releases.
>>>
>>> So, if you're asserting that the current guidance is OK, I generally
>>> agree.
>>> Is there a particular context where this was questioned? maybe we
>>> should examine the particulars of that situation. As in all things,
>>> context matters.
>>>
>>> Sean
>>>
>>> On Wed, Apr 1, 2020 at 7:34 PM Jungtaek Lim
>>> <ka...@gmail.com> wrote:
>>> >
>>> > Hi devs,
>>> >
>>> > I know we're busy with making Spark 3.0 be out, but I think the topic
>>> is good to discuss at any time and actually be better to be resolved sooner
>>> than later.
>>> >
>>> > In the page "Contributing to Spark", we describe the guide of "affects
>>> version" as "For Bugs, assign at least one version that is known to exhibit
>>> the problem or need the change".
>>> >
>>> > For me, that sentence clearly describes minimal requirement of affects
>>> version via:
>>> >
>>> > * For the type of bug, assign one valid version
>>> > * For other types, there's no requirement
>>> >
>>> > but I'm seeing the requests more than the requirement which makes me
>>> think there might be different understanding of the sentence. Maybe there's
>>> more, but to summarize on such requests:
>>> >
>>> > 1) add affects version as same as master branch for improvement/new
>>> feature
>>> > 2) check with older versions to fill up affects version for bug
>>> >
>>> > I don't see any point on doing 1). It might give some context if we
>>> don't update the affect version (so that it can say which version was
>>> considered when filing JIRA issue) but we also update the affect version
>>> when we bump the master branch, which is no longer informational as the
>>> version should have been always the same as master branch.
>>> >
>>> > I agree it's ideal to do 2) but I think the reason the guide doesn't
>>> enforce is that it requires pretty much efforts to check with old versions
>>> (sometimes even more than origin work).
>>> >
>>> > Suppose the happy case we have UT to verify the bugfix which fails
>>> without the patch and passes with the patch. To check with older versions
>>> we have to checkout the tag, and apply the UT, and "rebuild", and run UT to
>>> verify which is pretty much time-consuming. What if there's a conflict
>>> indeed? That's still a happy case, and in worse case (there's no such UT)
>>> we should do E2E manual verification which I would give up.
>>> >
>>> > There should have some balance/threshold, and the balance should be
>>> the thing the community has a consensus.
>>> >
>>> > Would like to hear everyone's voice on this.
>>> >
>>> > Thanks,
>>> > Jungtaek Lim (HeartSaVioR)
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>>>
>>>

Re: [DISCUSS] filling affected versions on JIRA issue

[Hyukjin Kwon <gu...@gmail.com>]

> 2) check with older versions to fill up affects version for bug
I don't agree with this in general. To me usually it's "For the type of
bug, assign one valid version" instead.

> The only place where I can see some amount of investigation being
required would be for security issues or correctness issues.
Yes, I agree.

Yes, was there a particular case or context that motivated this thread?

2020년 4월 2일 (목) 오전 10:24, Mridul Muralidharan <mr...@gmail.com>님이 작성:

>
> I agree with what Sean detailed.
> The only place where I can see some amount of investigation being required
> would be for security issues or correctness issues.
> Knowing the affected versions, particularly if an earlier supported
> version does not have the bug, will help users understand the
> broken/insecure versions.
>
> Regards,
> Mridul
>
>
> On Wed, Apr 1, 2020 at 6:12 PM Sean Owen <sr...@gmail.com> wrote:
>
>> I think we discussed this briefly on a PR.
>>
>> It's not as clear what it means for an Improvement to 'affect a
>> version'. Certainly, an improvement to a feature introduced in 1.2.3
>> can't affect anything earlier, and implicitly affects everything
>> after. It's not wrong to say it affects the latest version, at least.
>> And I believe we require it in JIRA because we can't require an
>> Affects Version for one type of issue but not another. So, just asking
>> people to default to 'latest version' there is no burden.
>>
>> I would not ask someone to figure out all and earliest versions that
>> an Improvement applies to; it just isn't that useful. We aren't
>> generally going to back-port improvements anyway.
>>
>> Even for bugs, we don't really need to know that a bug in master
>> affects 2.4.5, 2.4.4, 2.4.3, ... 2.3.6, 2.3.5, etc. It doesn't hurt to
>> at least say it affects the latest 2.4.x, 2.3.x releases, if known,
>> because it's possible it should be back-ported. Again even where this
>> is significantly more useful, I'm not in favor of telling people they
>> must test the bug report vs previous releases.
>>
>> So, if you're asserting that the current guidance is OK, I generally
>> agree.
>> Is there a particular context where this was questioned? maybe we
>> should examine the particulars of that situation. As in all things,
>> context matters.
>>
>> Sean
>>
>> On Wed, Apr 1, 2020 at 7:34 PM Jungtaek Lim
>> <ka...@gmail.com> wrote:
>> >
>> > Hi devs,
>> >
>> > I know we're busy with making Spark 3.0 be out, but I think the topic
>> is good to discuss at any time and actually be better to be resolved sooner
>> than later.
>> >
>> > In the page "Contributing to Spark", we describe the guide of "affects
>> version" as "For Bugs, assign at least one version that is known to exhibit
>> the problem or need the change".
>> >
>> > For me, that sentence clearly describes minimal requirement of affects
>> version via:
>> >
>> > * For the type of bug, assign one valid version
>> > * For other types, there's no requirement
>> >
>> > but I'm seeing the requests more than the requirement which makes me
>> think there might be different understanding of the sentence. Maybe there's
>> more, but to summarize on such requests:
>> >
>> > 1) add affects version as same as master branch for improvement/new
>> feature
>> > 2) check with older versions to fill up affects version for bug
>> >
>> > I don't see any point on doing 1). It might give some context if we
>> don't update the affect version (so that it can say which version was
>> considered when filing JIRA issue) but we also update the affect version
>> when we bump the master branch, which is no longer informational as the
>> version should have been always the same as master branch.
>> >
>> > I agree it's ideal to do 2) but I think the reason the guide doesn't
>> enforce is that it requires pretty much efforts to check with old versions
>> (sometimes even more than origin work).
>> >
>> > Suppose the happy case we have UT to verify the bugfix which fails
>> without the patch and passes with the patch. To check with older versions
>> we have to checkout the tag, and apply the UT, and "rebuild", and run UT to
>> verify which is pretty much time-consuming. What if there's a conflict
>> indeed? That's still a happy case, and in worse case (there's no such UT)
>> we should do E2E manual verification which I would give up.
>> >
>> > There should have some balance/threshold, and the balance should be the
>> thing the community has a consensus.
>> >
>> > Would like to hear everyone's voice on this.
>> >
>> > Thanks,
>> > Jungtaek Lim (HeartSaVioR)
>>
>> ---------------------------------------------------------------------
>> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>>
>>

Re: [DISCUSS] filling affected versions on JIRA issue

[Mridul Muralidharan <mr...@gmail.com>]

I agree with what Sean detailed.
The only place where I can see some amount of investigation being required
would be for security issues or correctness issues.
Knowing the affected versions, particularly if an earlier supported version
does not have the bug, will help users understand the broken/insecure
versions.

Regards,
Mridul


On Wed, Apr 1, 2020 at 6:12 PM Sean Owen <sr...@gmail.com> wrote:

> I think we discussed this briefly on a PR.
>
> It's not as clear what it means for an Improvement to 'affect a
> version'. Certainly, an improvement to a feature introduced in 1.2.3
> can't affect anything earlier, and implicitly affects everything
> after. It's not wrong to say it affects the latest version, at least.
> And I believe we require it in JIRA because we can't require an
> Affects Version for one type of issue but not another. So, just asking
> people to default to 'latest version' there is no burden.
>
> I would not ask someone to figure out all and earliest versions that
> an Improvement applies to; it just isn't that useful. We aren't
> generally going to back-port improvements anyway.
>
> Even for bugs, we don't really need to know that a bug in master
> affects 2.4.5, 2.4.4, 2.4.3, ... 2.3.6, 2.3.5, etc. It doesn't hurt to
> at least say it affects the latest 2.4.x, 2.3.x releases, if known,
> because it's possible it should be back-ported. Again even where this
> is significantly more useful, I'm not in favor of telling people they
> must test the bug report vs previous releases.
>
> So, if you're asserting that the current guidance is OK, I generally agree.
> Is there a particular context where this was questioned? maybe we
> should examine the particulars of that situation. As in all things,
> context matters.
>
> Sean
>
> On Wed, Apr 1, 2020 at 7:34 PM Jungtaek Lim
> <ka...@gmail.com> wrote:
> >
> > Hi devs,
> >
> > I know we're busy with making Spark 3.0 be out, but I think the topic is
> good to discuss at any time and actually be better to be resolved sooner
> than later.
> >
> > In the page "Contributing to Spark", we describe the guide of "affects
> version" as "For Bugs, assign at least one version that is known to exhibit
> the problem or need the change".
> >
> > For me, that sentence clearly describes minimal requirement of affects
> version via:
> >
> > * For the type of bug, assign one valid version
> > * For other types, there's no requirement
> >
> > but I'm seeing the requests more than the requirement which makes me
> think there might be different understanding of the sentence. Maybe there's
> more, but to summarize on such requests:
> >
> > 1) add affects version as same as master branch for improvement/new
> feature
> > 2) check with older versions to fill up affects version for bug
> >
> > I don't see any point on doing 1). It might give some context if we
> don't update the affect version (so that it can say which version was
> considered when filing JIRA issue) but we also update the affect version
> when we bump the master branch, which is no longer informational as the
> version should have been always the same as master branch.
> >
> > I agree it's ideal to do 2) but I think the reason the guide doesn't
> enforce is that it requires pretty much efforts to check with old versions
> (sometimes even more than origin work).
> >
> > Suppose the happy case we have UT to verify the bugfix which fails
> without the patch and passes with the patch. To check with older versions
> we have to checkout the tag, and apply the UT, and "rebuild", and run UT to
> verify which is pretty much time-consuming. What if there's a conflict
> indeed? That's still a happy case, and in worse case (there's no such UT)
> we should do E2E manual verification which I would give up.
> >
> > There should have some balance/threshold, and the balance should be the
> thing the community has a consensus.
> >
> > Would like to hear everyone's voice on this.
> >
> > Thanks,
> > Jungtaek Lim (HeartSaVioR)
>
> ---------------------------------------------------------------------
> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>
>

Re: [DISCUSS] filling affected versions on JIRA issue

[Sean Owen <sr...@gmail.com>]

I think we discussed this briefly on a PR.

It's not as clear what it means for an Improvement to 'affect a
version'. Certainly, an improvement to a feature introduced in 1.2.3
can't affect anything earlier, and implicitly affects everything
after. It's not wrong to say it affects the latest version, at least.
And I believe we require it in JIRA because we can't require an
Affects Version for one type of issue but not another. So, just asking
people to default to 'latest version' there is no burden.

I would not ask someone to figure out all and earliest versions that
an Improvement applies to; it just isn't that useful. We aren't
generally going to back-port improvements anyway.

Even for bugs, we don't really need to know that a bug in master
affects 2.4.5, 2.4.4, 2.4.3, ... 2.3.6, 2.3.5, etc. It doesn't hurt to
at least say it affects the latest 2.4.x, 2.3.x releases, if known,
because it's possible it should be back-ported. Again even where this
is significantly more useful, I'm not in favor of telling people they
must test the bug report vs previous releases.

So, if you're asserting that the current guidance is OK, I generally agree.
Is there a particular context where this was questioned? maybe we
should examine the particulars of that situation. As in all things,
context matters.

Sean

On Wed, Apr 1, 2020 at 7:34 PM Jungtaek Lim
<ka...@gmail.com> wrote:
>
> Hi devs,
>
> I know we're busy with making Spark 3.0 be out, but I think the topic is good to discuss at any time and actually be better to be resolved sooner than later.
>
> In the page "Contributing to Spark", we describe the guide of "affects version" as "For Bugs, assign at least one version that is known to exhibit the problem or need the change".
>
> For me, that sentence clearly describes minimal requirement of affects version via:
>
> * For the type of bug, assign one valid version
> * For other types, there's no requirement
>
> but I'm seeing the requests more than the requirement which makes me think there might be different understanding of the sentence. Maybe there's more, but to summarize on such requests:
>
> 1) add affects version as same as master branch for improvement/new feature
> 2) check with older versions to fill up affects version for bug
>
> I don't see any point on doing 1). It might give some context if we don't update the affect version (so that it can say which version was considered when filing JIRA issue) but we also update the affect version when we bump the master branch, which is no longer informational as the version should have been always the same as master branch.
>
> I agree it's ideal to do 2) but I think the reason the guide doesn't enforce is that it requires pretty much efforts to check with old versions (sometimes even more than origin work).
>
> Suppose the happy case we have UT to verify the bugfix which fails without the patch and passes with the patch. To check with older versions we have to checkout the tag, and apply the UT, and "rebuild", and run UT to verify which is pretty much time-consuming. What if there's a conflict indeed? That's still a happy case, and in worse case (there's no such UT) we should do E2E manual verification which I would give up.
>
> There should have some balance/threshold, and the balance should be the thing the community has a consensus.
>
> Would like to hear everyone's voice on this.
>
> Thanks,
> Jungtaek Lim (HeartSaVioR)

---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org