You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flink.apache.org by Robert Metzger <rm...@apache.org> on 2020/06/12 13:44:00 UTC
Re: [DISCUSS] Semantics of our JIRA fields
I agree with you Till -- changing the definition of the priorities should
be a separate discussion.
Piotrek, do you agree with my "affects version" explanation? I would like
to bring this discussion to a conclusion.
On Tue, May 26, 2020 at 4:51 PM Till Rohrmann <tr...@apache.org> wrote:
> If we change the meaning of the priority levels, then I would suggest to
> have a dedicated discussion for it. This would also be more visible than
> compared to being hidden in some lengthy discussion thread. I think the
> proposed definitions of priority levels differ slightly from how the
> community worked in the past.
>
> Cheers,
> Till
>
> On Tue, May 26, 2020 at 4:30 PM Robert Metzger <rm...@apache.org>
> wrote:
>
> > Hi,
> >
> > 1. I'm okay with updating the definition of the priorities for the reason
> > you've mentioned.
> >
> > 2. "Affects version"
> >
> > The reason why like to mark affects version on unreleased versions is to
> > clearly indicate which branch is affected by a bug. Given the current
> Flink
> > release status, if there's a bug only in "release-1.11", but not in
> > "master", there is no way of figuring that out, if we only allow released
> > versions for "affects version" (In my proposal, you would set "affects
> > version" to '1.11.0', '1.12.0' to indicate that).
> >
> > What we could do is introduce "1.12-SNAPSHOT" as version to mark issues
> on
> > unreleased versions. (But then people might accidentally set the "fix
> > version" to a "-SNAPSHOT" version.)
> >
> > I'm still in favor of my proposal. I have never heard a report from a
> > confused user about our Jira fields (I guess they usually check bugs for
> > released versions only)
> >
> >
> > On Tue, May 26, 2020 at 12:39 PM Piotr Nowojski <pi...@ververica.com>
> > wrote:
> >
> > > Hi,
> > >
> > > Sorry for a bit late response. I have two concerns:
> > >
> > > 1. Priority
> > >
> > > I would propose to stretch priorities that we are using to
> differentiate
> > > between things that must be fixed for given release:
> > >
> > > BLOCKER - drop anything you are doing, this issue must be fixed right
> now
> > > CRITICAL - release can not happen without fixing it, but can be fixed a
> > > bit later (for example without context switching and dropping whatever
> > I’m
> > > doing right now)
> > > MAJOR - default, nice to have
> > > Anything below - meh
> > >
> > > We were already using this semantic for tracking test instabilities
> > during
> > > the 1.11 release cycle. Good examples:
> > >
> > > BLOCKER - master branch not compiling, very frequent test failures (for
> > > example almost every build affected), …
> > > CRITICAL - performance regression/bug that we introduced in some
> feature,
> > > but which is not affecting other developers as much
> > > MAJOR - freshly discovered test instability with unknown
> impact/frequency
> > > (could be happening once a year),
> > >
> > > 2. Affects version
> > >
> > > If bug is only on the master branch, does it affect an unreleased
> > version?
> > >
> > > So far I was assuming that it doesn’t - unreleased bugs would have
> empty
> > > “affects version” field. My reasoning was that this field should be
> used
> > > for Flink users, to check which RELEASED Flink versions are affected by
> > > some bug, that user is searching for. Otherwise it might be a bit
> > confusing
> > > if there are lots of bugs with both affects version and fix version set
> > to
> > > the same value.
> > >
> > > Piotrek
> > >
> > > > On 25 May 2020, at 16:40, Robert Metzger <rm...@apache.org>
> wrote:
> > > >
> > > > Hi all,
> > > > thanks a lot for the feedback. The majority of responses are very
> > > positive
> > > > to my proposal.
> > > >
> > > > I have put my proposal into our wiki:
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=154995514
> > > >
> > > > Regarding the comments so far:
> > > > @Jark: I clarified this in the wiki.
> > > >
> > > > @Israel: I have not considered build changing all 3000 resolved
> tickets
> > > to
> > > > closed yet, but after consideration I don't think it is necessary. If
> > > > others in the community would like to change them, please speak up in
> > > this
> > > > thread :)
> > > >
> > > > @Flavio: I agree that we can not ask new or infrequent users to fully
> > > > adhere to these definitions. I added a note in the Wiki.
> > > > Using the resolved state for indicating "PR available" is problematic
> > > > because there are plenty of cases where PRs are stale (and this
> ticket
> > > > would then appear as a "resolved"). The Apache tools are adding a
> link
> > to
> > > > the PR, and some contributors are setting the ticket to "In
> Progress".
> > I
> > > > don't see a problem that we need to solve here.
> > > >
> > > > @Yun: Thank you for your comment. I added an example clarifying how I
> > > would
> > > > handle such a case. It is slightly different from your proposal: You
> > > > suggested using x.y.0 versions, I used "the next supported,
> unreleased
> > > > version", because that's how we've done it so far (and I don't want
> to
> > > > change things, I just want to document how the majority of the core
> > > > contributors are using JIRA).
> > > >
> > > > Here are all the changes (in green, blue are just formatting
> changes) I
> > > > made compared to my initial proposal:
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/pages/diffpagesbyversion.action?pageId=154995514&selectedPageVersions=4&selectedPageVersions=1
> > > >
> > > >
> > > >
> > > > On Mon, May 25, 2020 at 2:28 PM Congxian Qiu <qcx978132955@gmail.com
> >
> > > wrote:
> > > >
> > > >> @chesnay@apache.org <ch...@apache.org> Thanks for the
> confirmation
> > > >>
> > > >> Best,
> > > >> Congxian
> > > >>
> > > >>
> > > >> Zhu Zhu <re...@gmail.com> 于2020年5月25日周一 下午4:13写道：
> > > >>
> > > >>> This is very helpful!
> > > >>> +1
> > > >>>
> > > >>> Thanks,
> > > >>> Zhu Zhu
> > > >>>
> > > >>> Yang Wang <da...@gmail.com> 于2020年5月25日周一 下午4:04写道：
> > > >>>
> > > >>>> +1 from this useful proposal.
> > > >>>>
> > > >>>> This makes me clearer about "Resolve" and "Close" since I used to
> be
> > > >>>> confused by this two button.
> > > >>>>
> > > >>>> Best,
> > > >>>> Yang
> > > >>>>
> > > >>>> Jingsong Li <ji...@gmail.com> 于2020年5月25日周一 下午3:10写道：
> > > >>>>
> > > >>>>> +1 for the proposal.
> > > >>>>> It makes me clearer.
> > > >>>>>
> > > >>>>> Best,
> > > >>>>> Jingsong Lee
> > > >>>>>
> > > >>>>> On Mon, May 25, 2020 at 2:51 PM Zhijiang <
> > wangzhijiang999@aliyun.com
> > > >>>>> .invalid>
> > > >>>>> wrote:
> > > >>>>>
> > > >>>>>> Thanks for launching this discussion and giving so detailed
> infos,
> > > >>>>>> Robert! +1 on my side for the proposal.
> > > >>>>>>
> > > >>>>>> For "Affects Version", I previously thought it was only for the
> > > >>> already
> > > >>>>>> released versions, so it can give a reminder that the fix should
> > > >> also
> > > >>>>> pick
> > > >>>>>> into the related released branches for future minor versions.
> > > >>>>>> I saw that Jark had somehow similar concerns for this field in
> > > >> below
> > > >>>>>> replies. Either way makes sense for me as long as we give a
> > > >>> determined
> > > >>>>>> rule in Wiki.
> > > >>>>>>
> > > >>>>>> Re Flavio' s comments, I agree that the Jira reporter can leave
> > > >> most
> > > >>> of
> > > >>>>>> the fields empty if not confirmed of them, then the respective
> > > >>>> component
> > > >>>>>> maintainer or committer can update them accordingly later.
> > > >>>>>> But the state of Jira should not be marked as "resolved" when
> the
> > > >> PR
> > > >>> is
> > > >>>>>> detected, that is not fitting into the resolved semantic I
> guess.
> > > >> If
> > > >>>>>> possible, the Jira can be updated as "in progress" automatically
> > if
> > > >>>>>> the respective PR is ready, then it will save some time to stat
> > > >>>> progress
> > > >>>>>> of related issues during release process.
> > > >>>>>>
> > > >>>>>> Best,
> > > >>>>>> Zhijiang
> > > >>>>>>
> ------------------------------------------------------------------
> > > >>>>>> From:Congxian Qiu <qc...@gmail.com>
> > > >>>>>> Send Time:2020年5月25日(星期一) 13:57
> > > >>>>>> To:dev@flink.apache.org <de...@flink.apache.org>
> > > >>>>>> Subject:Re: [DISCUSS] Semantics of our JIRA fields
> > > >>>>>>
> > > >>>>>> Hi
> > > >>>>>>
> > > >>>>>> Currently, when I'm going to create an issue for
> Project-website.
> > > >> I'm
> > > >>>> not
> > > >>>>>> very sure what the "Affect Version/s" should be set. The problem
> > is
> > > >>>> that
> > > >>>>>> the current Dockfile[1] in flink-web is broken because of the
> EOL
> > > >> of
> > > >>>>> Ubuntu
> > > >>>>>> 18.10[2], the project-web does not affect any release of Flink,
> it
> > > >>> does
> > > >>>>>> affect the process to build website, so what's the version
> should
> > I
> > > >>> set
> > > >>>>> to?
> > > >>>>>>
> > > >>>>>> [1]
> > > >>>>>>
> > > >>>>>>
> > > >>>>>
> > > >>>>
> > > >>>
> > > >>
> > >
> >
> https://github.com/apache/flink-web/blob/bc66f0f0f463ab62a22e81df7d7efd301b76a6b4/docker/Dockerfile#L17
> > > >>>>>> [2] https://wiki.ubuntu.com/Releases
> > > >>>>>>
> > > >>>>>> Best,
> > > >>>>>> Congxian
> > > >>>>>>
> > > >>>>>>
> > > >>>>>> Flavio Pompermaier <po...@okkam.it> 于2020年5月24日周日
> 下午1:23写道：
> > > >>>>>>
> > > >>>>>>> In my experience it's quite complicated for a normal reporter
> to
> > > >> be
> > > >>>>> able
> > > >>>>>> to
> > > >>>>>>> fill all the fields correctly (especially for new users).
> > > >>>>>>> Usually you just wanto to report a problem, remember to add a
> new
> > > >>>>> feature
> > > >>>>>>> or improve code/documentation but you can't really give a
> > > >> priority,
> > > >>>>>> assign
> > > >>>>>>> the correct label or decide which releases will benefit of the
> > > >>>> fix/new
> > > >>>>>>> feature. This is something that only core developers could
> decide
> > > >>>>> (IMHO).
> > > >>>>>>>
> > > >>>>>>> My experiece says that it's better if normal users could just
> > > >> open
> > > >>>>>> tickets
> > > >>>>>>> with some default (or mark ticket as new) and leave them in
> such
> > > >> a
> > > >>>>> state
> > > >>>>>>> until an experienced user, one that can assign tickets, have
> the
> > > >>> time
> > > >>>>> to
> > > >>>>>>> read it and immediately reject the ticket or accept it and
> > > >> properly
> > > >>>>>> assign
> > > >>>>>>> priorities, fix version, etc.
> > > >>>>>>>
> > > >>>>>>> With respect to resolve/close I think that a good practice
> could
> > > >> be
> > > >>>> to
> > > >>>>>> mark
> > > >>>>>>> automatically a ticket as resolved once that a PR is detected
> for
> > > >>>> that
> > > >>>>>>> ticket, while marking it as closed should be done by the
> commiter
> > > >>> who
> > > >>>>>> merge
> > > >>>>>>> the PR.
> > > >>>>>>>
> > > >>>>>>> Probably this process would slightly increase the work of a
> > > >>> committer
> > > >>>>> but
> > > >>>>>>> this would make things a lot cleaner and will bring the benefit
> > > >> of
> > > >>>>>> having a
> > > >>>>>>> reliable and semantically sound JIRA state.
> > > >>>>>>>
> > > >>>>>>> Cheers,
> > > >>>>>>> Flavio
> > > >>>>>>>
> > > >>>>>>> Il Dom 24 Mag 2020, 05:05 Israel Ekpo <is...@gmail.com>
> ha
> > > >>>>> scritto:
> > > >>>>>>>
> > > >>>>>>>> +1 for the proposal
> > > >>>>>>>>
> > > >>>>>>>> This will bring some consistency to the process
> > > >>>>>>>>
> > > >>>>>>>> Regarding Closed vs Resolved, should we go back and switch all
> > > >>> the
> > > >>>>>>> Resolved
> > > >>>>>>>> issues to Closed so that is is not confusing to new people to
> > > >> the
> > > >>>>>> project
> > > >>>>>>>> in the future that may not have seen this discussion?
> > > >>>>>>>>
> > > >>>>>>>> I would recommend changing it to Closed just to be consistent
> > > >>> since
> > > >>>>>> that
> > > >>>>>>> is
> > > >>>>>>>> the majority and the new process will be using Closed going
> > > >>> forward
> > > >>>>>>>>
> > > >>>>>>>> Those are my thoughts for now
> > > >>>>>>>>
> > > >>>>>>>> On Sat, May 23, 2020 at 7:48 AM Congxian Qiu <
> > > >>>> qcx978132955@gmail.com
> > > >>>>>>
> > > >>>>>>>> wrote:
> > > >>>>>>>>
> > > >>>>>>>>> +1 for the proposal. It's good to have a unified description
> > > >>> and
> > > >>>>>> write
> > > >>>>>>> it
> > > >>>>>>>>> down in the wiki, so that every contributor has the same
> > > >>>>>> understanding
> > > >>>>>>> of
> > > >>>>>>>>> all the fields.
> > > >>>>>>>>>
> > > >>>>>>>>> Best,
> > > >>>>>>>>> Congxian
> > > >>>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>> Till Rohrmann <tr...@apache.org> 于2020年5月23日周六
> > > >> 上午12:04写道：
> > > >>>>>>>>>
> > > >>>>>>>>>> Thanks for drafting this proposal Robert. +1 for the
> > > >>> proposal.
> > > >>>>>>>>>>
> > > >>>>>>>>>> Cheers,
> > > >>>>>>>>>> Till
> > > >>>>>>>>>>
> > > >>>>>>>>>> On Fri, May 22, 2020 at 5:39 PM Leonard Xu <
> > > >>> xbjtdcq@gmail.com>
> > > >>>>>>> wrote:
> > > >>>>>>>>>>
> > > >>>>>>>>>>> Thanks bringing up this topic @Robert, +1 to the
> > > >> proposal.
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> It clarifies the JIRA fields well and should be a rule to
> > > >>>>> follow.
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> Best,
> > > >>>>>>>>>>> Leonard Xu
> > > >>>>>>>>>>>> 在 2020年5月22日，20:24，Aljoscha Krettek <
> > > >> aljoscha@apache.org
> > > >>>>
> > > >>>>> 写道：
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>> +1 That's also how I think of the semantics of the
> > > >>> fields.
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>> Aljoscha
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>> On 22.05.20 08:07, Robert Metzger wrote:
> > > >>>>>>>>>>>>> Hi all,
> > > >>>>>>>>>>>>> I have the feeling that the semantics of some of our
> > > >>> JIRA
> > > >>>>>> fields
> > > >>>>>>>>>> (mostly
> > > >>>>>>>>>>>>> "affects versions", "fix versions" and resolve /
> > > >> close)
> > > >>>> are
> > > >>>>>> not
> > > >>>>>>>>>> defined
> > > >>>>>>>>>>> in
> > > >>>>>>>>>>>>> the same way by all the core Flink contributors, which
> > > >>>> leads
> > > >>>>>> to
> > > >>>>>>>>> cases
> > > >>>>>>>>>>> where
> > > >>>>>>>>>>>>> I spend quite some time on filling the fields
> > > >> correctly
> > > >>>> (at
> > > >>>>>>> least
> > > >>>>>>>>>> what I
> > > >>>>>>>>>>>>> consider correctly), and then others changing them
> > > >> again
> > > >>>> to
> > > >>>>>>> match
> > > >>>>>>>>>> their
> > > >>>>>>>>>>>>> semantics.
> > > >>>>>>>>>>>>> In an effort to increase our efficiency, and since I'm
> > > >>>>>> creating
> > > >>>>>>> a
> > > >>>>>>>>> lot
> > > >>>>>>>>>> of
> > > >>>>>>>>>>>>> (test instability-related) tickets these days, I would
> > > >>>> like
> > > >>>>> to
> > > >>>>>>>>> discuss
> > > >>>>>>>>>>> the
> > > >>>>>>>>>>>>> semantics, come to a conclusion and document this in
> > > >> our
> > > >>>>> Wiki.
> > > >>>>>>>>>>>>> *Proposal:*
> > > >>>>>>>>>>>>> *Priority:*
> > > >>>>>>>>>>>>> "Blocker": needs to be resolved before a release
> > > >>> (matched
> > > >>>>>> based
> > > >>>>>>> on
> > > >>>>>>>>> fix
> > > >>>>>>>>>>>>> versions)
> > > >>>>>>>>>>>>> "Critical": strongly considered before a release
> > > >>>>>>>>>>>>> other priorities have no practical meaning in Flink.
> > > >>>>>>>>>>>>> *Component/s:*
> > > >>>>>>>>>>>>> Primary component relevant for this feature / fix.
> > > >>>>>>>>>>>>> For test-related issues, add the component the test
> > > >>>> belongs
> > > >>>>> to
> > > >>>>>>>> (for
> > > >>>>>>>>>>> example
> > > >>>>>>>>>>>>> "Connectors / Kafka" for Kafka test failures) +
> > > >> "Test".
> > > >>>>>>>>>>>>> The same applies for documentation tickets. For
> > > >> example,
> > > >>>> if
> > > >>>>>>>> there's
> > > >>>>>>>>>>>>> something wrong with the DataStream API, add it to the
> > > >>>> "API
> > > >>>>> /
> > > >>>>>>>>>>> DataStream"
> > > >>>>>>>>>>>>> and "Documentation" components.
> > > >>>>>>>>>>>>> *Affects Version/s:*Only for Bug / Task-type tickets:
> > > >> We
> > > >>>>> list
> > > >>>>>>> all
> > > >>>>>>>>>>> currently
> > > >>>>>>>>>>>>> supported and unreleased Flink versions known to be
> > > >>>> affected
> > > >>>>>> by
> > > >>>>>>>>> this.
> > > >>>>>>>>>>>>> Example: If I see a test failure that happens on
> > > >>> "master"
> > > >>>>> and
> > > >>>>>>>>>>>>> "release-1.11", I set "affects version" to "1.12.0"
> > > >> and
> > > >>>>>>> "1.11.0".
> > > >>>>>>>>>>>>> *Fix Version/s:*
> > > >>>>>>>>>>>>> For closed/resolved tickets, this field lists the
> > > >>> released
> > > >>>>>> Flink
> > > >>>>>>>>>>> versions
> > > >>>>>>>>>>>>> that contain a fix or feature for the first time.
> > > >>>>>>>>>>>>> For open tickets, it indicates that a fix / feature
> > > >>> should
> > > >>>>> be
> > > >>>>>>>>>> contained
> > > >>>>>>>>>>> in
> > > >>>>>>>>>>>>> the listed versions. Only blocker issues can block a
> > > >>>>> release,
> > > >>>>>>> all
> > > >>>>>>>>>> other
> > > >>>>>>>>>>>>> tickets which have "fix version/s" set at the time of
> > > >> a
> > > >>>>>> release
> > > >>>>>>>> and
> > > >>>>>>>>>> are
> > > >>>>>>>>>>>>> unresolved will be moved to the next version.
> > > >>>>>>>>>>>>> *Assignee:*
> > > >>>>>>>>>>>>> Person currently working on the ticket. Assigned after
> > > >>>>>>> conclusion
> > > >>>>>>>> on
> > > >>>>>>>>>>>>> approach by a committer.
> > > >>>>>>>>>>>>> Often, fixes are obvious and committers self-assign
> > > >> w/o
> > > >>>>>>>> discussion.
> > > >>>>>>>>>>>>> *Resolve / Close:*
> > > >>>>>>>>>>>>> You can either Resolve or Close a ticket once it is
> > > >> done
> > > >>>>>> (fixed,
> > > >>>>>>>>>>> rejected,
> > > >>>>>>>>>>>>> invalid, ...).
> > > >>>>>>>>>>>>> As a rule, we Close tickets instead of Resolving them
> > > >>> when
> > > >>>>>> they
> > > >>>>>>>> are
> > > >>>>>>>>>>> done.
> > > >>>>>>>>>>>>> Background: There are semantic differences for Resolve
> > > >>> and
> > > >>>>>> Close
> > > >>>>>>>>>>>>> (implementor vs reporter considers it done), but I
> > > >> don't
> > > >>>> see
> > > >>>>>> how
> > > >>>>>>>>> they
> > > >>>>>>>>>>>>> practically apply to the Flink project. Looking at the
> > > >>>>>> numbers,
> > > >>>>>>>>> Flink
> > > >>>>>>>>>>> has
> > > >>>>>>>>>>>>> 11066 closed tickets, and 3372 resolved tickets
> > > >> (that's
> > > >>>> why
> > > >>>>> I
> > > >>>>>>>>> propose
> > > >>>>>>>>>> to
> > > >>>>>>>>>>>>> close instead of resolve)
> > > >>>>>>>>>>>>> *Labels:*
> > > >>>>>>>>>>>>> "test-stability" for all test instabilities
> > > >>>>>>>>>>>>> "starter" for tickets suitable for new contributors
> > > >>>>>>>>>>>>> *Release Note:*
> > > >>>>>>>>>>>>> Small notes that will be included into the release
> > > >> notes
> > > >>>>>>> published
> > > >>>>>>>>>> with
> > > >>>>>>>>>>> the
> > > >>>>>>>>>>>>> release.
> > > >>>>>>>>>>>>> *All other fields are not used not used on a regular
> > > >>>> basis.*
> > > >>>>>>>>>>>>> Please +1 my proposal if you want it to be published
> > > >> in
> > > >>>> our
> > > >>>>>> Wiki
> > > >>>>>>>>> like
> > > >>>>>>>>>>> that
> > > >>>>>>>>>>>>> or let me know if I got something wrong here.
> > > >>>>>>>>>>>>> Best,
> > > >>>>>>>>>>>>> Robert
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>
> > > >>>>>>
> > > >>>>>>
> > > >>>>>
> > > >>>>> --
> > > >>>>> Best, Jingsong Lee
> > > >>>>>
> > > >>>>
> > > >>>
> > > >>
> > >
> > >
> >
>
Re: [DISCUSS] Semantics of our JIRA fields
Posted by Robert Metzger <rm...@apache.org>.
Since there were no further comments on this discussion, I removed the
"draft" label from the Wiki page and I consider the Jira semantics proposal
agreed upon.
On Mon, Jun 15, 2020 at 9:49 AM Piotr Nowojski <pi...@ververica.com> wrote:
>
> > On 12 Jun 2020, at 15:44, Robert Metzger <rm...@apache.org> wrote:
> >
> > Piotrek, do you agree with my "affects version" explanation? I would like
> > to bring this discussion to a conclusion.
> >
>
> +0 for this semantic from my side.
>
> > On Tue, May 26, 2020 at 4:51 PM Till Rohrmann <tr...@apache.org>
> wrote:
> >
> >> If we change the meaning of the priority levels, then I would suggest to
> >> have a dedicated discussion for it. This would also be more visible than
> >> compared to being hidden in some lengthy discussion thread. I think the
> >> proposed definitions of priority levels differ slightly from how the
> >> community worked in the past.
> >>
> >> Cheers,
> >> Till
> >>
> >> On Tue, May 26, 2020 at 4:30 PM Robert Metzger <rm...@apache.org>
> >> wrote:
> >>
> >>> Hi,
> >>>
> >>> 1. I'm okay with updating the definition of the priorities for the
> reason
> >>> you've mentioned.
> >>>
> >>> 2. "Affects version"
> >>>
> >>> The reason why like to mark affects version on unreleased versions is
> to
> >>> clearly indicate which branch is affected by a bug. Given the current
> >> Flink
> >>> release status, if there's a bug only in "release-1.11", but not in
> >>> "master", there is no way of figuring that out, if we only allow
> released
> >>> versions for "affects version" (In my proposal, you would set "affects
> >>> version" to '1.11.0', '1.12.0' to indicate that).
> >>>
> >>> What we could do is introduce "1.12-SNAPSHOT" as version to mark issues
> >> on
> >>> unreleased versions. (But then people might accidentally set the "fix
> >>> version" to a "-SNAPSHOT" version.)
> >>>
> >>> I'm still in favor of my proposal. I have never heard a report from a
> >>> confused user about our Jira fields (I guess they usually check bugs
> for
> >>> released versions only)
> >>>
> >>>
> >>> On Tue, May 26, 2020 at 12:39 PM Piotr Nowojski <pi...@ververica.com>
> >>> wrote:
> >>>
> >>>> Hi,
> >>>>
> >>>> Sorry for a bit late response. I have two concerns:
> >>>>
> >>>> 1. Priority
> >>>>
> >>>> I would propose to stretch priorities that we are using to
> >> differentiate
> >>>> between things that must be fixed for given release:
> >>>>
> >>>> BLOCKER - drop anything you are doing, this issue must be fixed right
> >> now
> >>>> CRITICAL - release can not happen without fixing it, but can be fixed
> a
> >>>> bit later (for example without context switching and dropping whatever
> >>> I’m
> >>>> doing right now)
> >>>> MAJOR - default, nice to have
> >>>> Anything below - meh
> >>>>
> >>>> We were already using this semantic for tracking test instabilities
> >>> during
> >>>> the 1.11 release cycle. Good examples:
> >>>>
> >>>> BLOCKER - master branch not compiling, very frequent test failures
> (for
> >>>> example almost every build affected), …
> >>>> CRITICAL - performance regression/bug that we introduced in some
> >> feature,
> >>>> but which is not affecting other developers as much
> >>>> MAJOR - freshly discovered test instability with unknown
> >> impact/frequency
> >>>> (could be happening once a year),
> >>>>
> >>>> 2. Affects version
> >>>>
> >>>> If bug is only on the master branch, does it affect an unreleased
> >>> version?
> >>>>
> >>>> So far I was assuming that it doesn’t - unreleased bugs would have
> >> empty
> >>>> “affects version” field. My reasoning was that this field should be
> >> used
> >>>> for Flink users, to check which RELEASED Flink versions are affected
> by
> >>>> some bug, that user is searching for. Otherwise it might be a bit
> >>> confusing
> >>>> if there are lots of bugs with both affects version and fix version
> set
> >>> to
> >>>> the same value.
> >>>>
> >>>> Piotrek
> >>>>
> >>>>> On 25 May 2020, at 16:40, Robert Metzger <rm...@apache.org>
> >> wrote:
> >>>>>
> >>>>> Hi all,
> >>>>> thanks a lot for the feedback. The majority of responses are very
> >>>> positive
> >>>>> to my proposal.
> >>>>>
> >>>>> I have put my proposal into our wiki:
> >>>>>
> >>>>
> >>>
> >>
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=154995514
> >>>>>
> >>>>> Regarding the comments so far:
> >>>>> @Jark: I clarified this in the wiki.
> >>>>>
> >>>>> @Israel: I have not considered build changing all 3000 resolved
> >> tickets
> >>>> to
> >>>>> closed yet, but after consideration I don't think it is necessary. If
> >>>>> others in the community would like to change them, please speak up in
> >>>> this
> >>>>> thread :)
> >>>>>
> >>>>> @Flavio: I agree that we can not ask new or infrequent users to fully
> >>>>> adhere to these definitions. I added a note in the Wiki.
> >>>>> Using the resolved state for indicating "PR available" is problematic
> >>>>> because there are plenty of cases where PRs are stale (and this
> >> ticket
> >>>>> would then appear as a "resolved"). The Apache tools are adding a
> >> link
> >>> to
> >>>>> the PR, and some contributors are setting the ticket to "In
> >> Progress".
> >>> I
> >>>>> don't see a problem that we need to solve here.
> >>>>>
> >>>>> @Yun: Thank you for your comment. I added an example clarifying how I
> >>>> would
> >>>>> handle such a case. It is slightly different from your proposal: You
> >>>>> suggested using x.y.0 versions, I used "the next supported,
> >> unreleased
> >>>>> version", because that's how we've done it so far (and I don't want
> >> to
> >>>>> change things, I just want to document how the majority of the core
> >>>>> contributors are using JIRA).
> >>>>>
> >>>>> Here are all the changes (in green, blue are just formatting
> >> changes) I
> >>>>> made compared to my initial proposal:
> >>>>>
> >>>>
> >>>
> >>
> https://cwiki.apache.org/confluence/pages/diffpagesbyversion.action?pageId=154995514&selectedPageVersions=4&selectedPageVersions=1
> >>>>>
> >>>>>
> >>>>>
> >>>>> On Mon, May 25, 2020 at 2:28 PM Congxian Qiu <qcx978132955@gmail.com
> >>>
> >>>> wrote:
> >>>>>
> >>>>>> @chesnay@apache.org <ch...@apache.org> Thanks for the
> >> confirmation
> >>>>>>
> >>>>>> Best,
> >>>>>> Congxian
> >>>>>>
> >>>>>>
> >>>>>> Zhu Zhu <re...@gmail.com> 于2020年5月25日周一 下午4:13写道：
> >>>>>>
> >>>>>>> This is very helpful!
> >>>>>>> +1
> >>>>>>>
> >>>>>>> Thanks,
> >>>>>>> Zhu Zhu
> >>>>>>>
> >>>>>>> Yang Wang <da...@gmail.com> 于2020年5月25日周一 下午4:04写道：
> >>>>>>>
> >>>>>>>> +1 from this useful proposal.
> >>>>>>>>
> >>>>>>>> This makes me clearer about "Resolve" and "Close" since I used to
> >> be
> >>>>>>>> confused by this two button.
> >>>>>>>>
> >>>>>>>> Best,
> >>>>>>>> Yang
> >>>>>>>>
> >>>>>>>> Jingsong Li <ji...@gmail.com> 于2020年5月25日周一 下午3:10写道：
> >>>>>>>>
> >>>>>>>>> +1 for the proposal.
> >>>>>>>>> It makes me clearer.
> >>>>>>>>>
> >>>>>>>>> Best,
> >>>>>>>>> Jingsong Lee
> >>>>>>>>>
> >>>>>>>>> On Mon, May 25, 2020 at 2:51 PM Zhijiang <
> >>> wangzhijiang999@aliyun.com
> >>>>>>>>> .invalid>
> >>>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> Thanks for launching this discussion and giving so detailed
> >> infos,
> >>>>>>>>>> Robert! +1 on my side for the proposal.
> >>>>>>>>>>
> >>>>>>>>>> For "Affects Version", I previously thought it was only for the
> >>>>>>> already
> >>>>>>>>>> released versions, so it can give a reminder that the fix should
> >>>>>> also
> >>>>>>>>> pick
> >>>>>>>>>> into the related released branches for future minor versions.
> >>>>>>>>>> I saw that Jark had somehow similar concerns for this field in
> >>>>>> below
> >>>>>>>>>> replies. Either way makes sense for me as long as we give a
> >>>>>>> determined
> >>>>>>>>>> rule in Wiki.
> >>>>>>>>>>
> >>>>>>>>>> Re Flavio' s comments, I agree that the Jira reporter can leave
> >>>>>> most
> >>>>>>> of
> >>>>>>>>>> the fields empty if not confirmed of them, then the respective
> >>>>>>>> component
> >>>>>>>>>> maintainer or committer can update them accordingly later.
> >>>>>>>>>> But the state of Jira should not be marked as "resolved" when
> >> the
> >>>>>> PR
> >>>>>>> is
> >>>>>>>>>> detected, that is not fitting into the resolved semantic I
> >> guess.
> >>>>>> If
> >>>>>>>>>> possible, the Jira can be updated as "in progress" automatically
> >>> if
> >>>>>>>>>> the respective PR is ready, then it will save some time to stat
> >>>>>>>> progress
> >>>>>>>>>> of related issues during release process.
> >>>>>>>>>>
> >>>>>>>>>> Best,
> >>>>>>>>>> Zhijiang
> >>>>>>>>>>
> >> ------------------------------------------------------------------
> >>>>>>>>>> From:Congxian Qiu <qc...@gmail.com>
> >>>>>>>>>> Send Time:2020年5月25日(星期一) 13:57
> >>>>>>>>>> To:dev@flink.apache.org <de...@flink.apache.org>
> >>>>>>>>>> Subject:Re: [DISCUSS] Semantics of our JIRA fields
> >>>>>>>>>>
> >>>>>>>>>> Hi
> >>>>>>>>>>
> >>>>>>>>>> Currently, when I'm going to create an issue for
> >> Project-website.
> >>>>>> I'm
> >>>>>>>> not
> >>>>>>>>>> very sure what the "Affect Version/s" should be set. The problem
> >>> is
> >>>>>>>> that
> >>>>>>>>>> the current Dockfile[1] in flink-web is broken because of the
> >> EOL
> >>>>>> of
> >>>>>>>>> Ubuntu
> >>>>>>>>>> 18.10[2], the project-web does not affect any release of Flink,
> >> it
> >>>>>>> does
> >>>>>>>>>> affect the process to build website, so what's the version
> >> should
> >>> I
> >>>>>>> set
> >>>>>>>>> to?
> >>>>>>>>>>
> >>>>>>>>>> [1]
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>
> >>>
> >>
> https://github.com/apache/flink-web/blob/bc66f0f0f463ab62a22e81df7d7efd301b76a6b4/docker/Dockerfile#L17
> >>>>>>>>>> [2] https://wiki.ubuntu.com/Releases
> >>>>>>>>>>
> >>>>>>>>>> Best,
> >>>>>>>>>> Congxian
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> Flavio Pompermaier <po...@okkam.it> 于2020年5月24日周日
> >> 下午1:23写道：
> >>>>>>>>>>
> >>>>>>>>>>> In my experience it's quite complicated for a normal reporter
> >> to
> >>>>>> be
> >>>>>>>>> able
> >>>>>>>>>> to
> >>>>>>>>>>> fill all the fields correctly (especially for new users).
> >>>>>>>>>>> Usually you just wanto to report a problem, remember to add a
> >> new
> >>>>>>>>> feature
> >>>>>>>>>>> or improve code/documentation but you can't really give a
> >>>>>> priority,
> >>>>>>>>>> assign
> >>>>>>>>>>> the correct label or decide which releases will benefit of the
> >>>>>>>> fix/new
> >>>>>>>>>>> feature. This is something that only core developers could
> >> decide
> >>>>>>>>> (IMHO).
> >>>>>>>>>>>
> >>>>>>>>>>> My experiece says that it's better if normal users could just
> >>>>>> open
> >>>>>>>>>> tickets
> >>>>>>>>>>> with some default (or mark ticket as new) and leave them in
> >> such
> >>>>>> a
> >>>>>>>>> state
> >>>>>>>>>>> until an experienced user, one that can assign tickets, have
> >> the
> >>>>>>> time
> >>>>>>>>> to
> >>>>>>>>>>> read it and immediately reject the ticket or accept it and
> >>>>>> properly
> >>>>>>>>>> assign
> >>>>>>>>>>> priorities, fix version, etc.
> >>>>>>>>>>>
> >>>>>>>>>>> With respect to resolve/close I think that a good practice
> >> could
> >>>>>> be
> >>>>>>>> to
> >>>>>>>>>> mark
> >>>>>>>>>>> automatically a ticket as resolved once that a PR is detected
> >> for
> >>>>>>>> that
> >>>>>>>>>>> ticket, while marking it as closed should be done by the
> >> commiter
> >>>>>>> who
> >>>>>>>>>> merge
> >>>>>>>>>>> the PR.
> >>>>>>>>>>>
> >>>>>>>>>>> Probably this process would slightly increase the work of a
> >>>>>>> committer
> >>>>>>>>> but
> >>>>>>>>>>> this would make things a lot cleaner and will bring the benefit
> >>>>>> of
> >>>>>>>>>> having a
> >>>>>>>>>>> reliable and semantically sound JIRA state.
> >>>>>>>>>>>
> >>>>>>>>>>> Cheers,
> >>>>>>>>>>> Flavio
> >>>>>>>>>>>
> >>>>>>>>>>> Il Dom 24 Mag 2020, 05:05 Israel Ekpo <is...@gmail.com>
> >> ha
> >>>>>>>>> scritto:
> >>>>>>>>>>>
> >>>>>>>>>>>> +1 for the proposal
> >>>>>>>>>>>>
> >>>>>>>>>>>> This will bring some consistency to the process
> >>>>>>>>>>>>
> >>>>>>>>>>>> Regarding Closed vs Resolved, should we go back and switch all
> >>>>>>> the
> >>>>>>>>>>> Resolved
> >>>>>>>>>>>> issues to Closed so that is is not confusing to new people to
> >>>>>> the
> >>>>>>>>>> project
> >>>>>>>>>>>> in the future that may not have seen this discussion?
> >>>>>>>>>>>>
> >>>>>>>>>>>> I would recommend changing it to Closed just to be consistent
> >>>>>>> since
> >>>>>>>>>> that
> >>>>>>>>>>> is
> >>>>>>>>>>>> the majority and the new process will be using Closed going
> >>>>>>> forward
> >>>>>>>>>>>>
> >>>>>>>>>>>> Those are my thoughts for now
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Sat, May 23, 2020 at 7:48 AM Congxian Qiu <
> >>>>>>>> qcx978132955@gmail.com
> >>>>>>>>>>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> +1 for the proposal. It's good to have a unified description
> >>>>>>> and
> >>>>>>>>>> write
> >>>>>>>>>>> it
> >>>>>>>>>>>>> down in the wiki, so that every contributor has the same
> >>>>>>>>>> understanding
> >>>>>>>>>>> of
> >>>>>>>>>>>>> all the fields.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Best,
> >>>>>>>>>>>>> Congxian
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Till Rohrmann <tr...@apache.org> 于2020年5月23日周六
> >>>>>> 上午12:04写道：
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> Thanks for drafting this proposal Robert. +1 for the
> >>>>>>> proposal.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Cheers,
> >>>>>>>>>>>>>> Till
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Fri, May 22, 2020 at 5:39 PM Leonard Xu <
> >>>>>>> xbjtdcq@gmail.com>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Thanks bringing up this topic @Robert, +1 to the
> >>>>>> proposal.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> It clarifies the JIRA fields well and should be a rule to
> >>>>>>>>> follow.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Best,
> >>>>>>>>>>>>>>> Leonard Xu
> >>>>>>>>>>>>>>>> 在 2020年5月22日，20:24，Aljoscha Krettek <
> >>>>>> aljoscha@apache.org
> >>>>>>>>
> >>>>>>>>> 写道：
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> +1 That's also how I think of the semantics of the
> >>>>>>> fields.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Aljoscha
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On 22.05.20 08:07, Robert Metzger wrote:
> >>>>>>>>>>>>>>>>> Hi all,
> >>>>>>>>>>>>>>>>> I have the feeling that the semantics of some of our
> >>>>>>> JIRA
> >>>>>>>>>> fields
> >>>>>>>>>>>>>> (mostly
> >>>>>>>>>>>>>>>>> "affects versions", "fix versions" and resolve /
> >>>>>> close)
> >>>>>>>> are
> >>>>>>>>>> not
> >>>>>>>>>>>>>> defined
> >>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>> the same way by all the core Flink contributors, which
> >>>>>>>> leads
> >>>>>>>>>> to
> >>>>>>>>>>>>> cases
> >>>>>>>>>>>>>>> where
> >>>>>>>>>>>>>>>>> I spend quite some time on filling the fields
> >>>>>> correctly
> >>>>>>>> (at
> >>>>>>>>>>> least
> >>>>>>>>>>>>>> what I
> >>>>>>>>>>>>>>>>> consider correctly), and then others changing them
> >>>>>> again
> >>>>>>>> to
> >>>>>>>>>>> match
> >>>>>>>>>>>>>> their
> >>>>>>>>>>>>>>>>> semantics.
> >>>>>>>>>>>>>>>>> In an effort to increase our efficiency, and since I'm
> >>>>>>>>>> creating
> >>>>>>>>>>> a
> >>>>>>>>>>>>> lot
> >>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>> (test instability-related) tickets these days, I would
> >>>>>>>> like
> >>>>>>>>> to
> >>>>>>>>>>>>> discuss
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> semantics, come to a conclusion and document this in
> >>>>>> our
> >>>>>>>>> Wiki.
> >>>>>>>>>>>>>>>>> *Proposal:*
> >>>>>>>>>>>>>>>>> *Priority:*
> >>>>>>>>>>>>>>>>> "Blocker": needs to be resolved before a release
> >>>>>>> (matched
> >>>>>>>>>> based
> >>>>>>>>>>> on
> >>>>>>>>>>>>> fix
> >>>>>>>>>>>>>>>>> versions)
> >>>>>>>>>>>>>>>>> "Critical": strongly considered before a release
> >>>>>>>>>>>>>>>>> other priorities have no practical meaning in Flink.
> >>>>>>>>>>>>>>>>> *Component/s:*
> >>>>>>>>>>>>>>>>> Primary component relevant for this feature / fix.
> >>>>>>>>>>>>>>>>> For test-related issues, add the component the test
> >>>>>>>> belongs
> >>>>>>>>> to
> >>>>>>>>>>>> (for
> >>>>>>>>>>>>>>> example
> >>>>>>>>>>>>>>>>> "Connectors / Kafka" for Kafka test failures) +
> >>>>>> "Test".
> >>>>>>>>>>>>>>>>> The same applies for documentation tickets. For
> >>>>>> example,
> >>>>>>>> if
> >>>>>>>>>>>> there's
> >>>>>>>>>>>>>>>>> something wrong with the DataStream API, add it to the
> >>>>>>>> "API
> >>>>>>>>> /
> >>>>>>>>>>>>>>> DataStream"
> >>>>>>>>>>>>>>>>> and "Documentation" components.
> >>>>>>>>>>>>>>>>> *Affects Version/s:*Only for Bug / Task-type tickets:
> >>>>>> We
> >>>>>>>>> list
> >>>>>>>>>>> all
> >>>>>>>>>>>>>>> currently
> >>>>>>>>>>>>>>>>> supported and unreleased Flink versions known to be
> >>>>>>>> affected
> >>>>>>>>>> by
> >>>>>>>>>>>>> this.
> >>>>>>>>>>>>>>>>> Example: If I see a test failure that happens on
> >>>>>>> "master"
> >>>>>>>>> and
> >>>>>>>>>>>>>>>>> "release-1.11", I set "affects version" to "1.12.0"
> >>>>>> and
> >>>>>>>>>>> "1.11.0".
> >>>>>>>>>>>>>>>>> *Fix Version/s:*
> >>>>>>>>>>>>>>>>> For closed/resolved tickets, this field lists the
> >>>>>>> released
> >>>>>>>>>> Flink
> >>>>>>>>>>>>>>> versions
> >>>>>>>>>>>>>>>>> that contain a fix or feature for the first time.
> >>>>>>>>>>>>>>>>> For open tickets, it indicates that a fix / feature
> >>>>>>> should
> >>>>>>>>> be
> >>>>>>>>>>>>>> contained
> >>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>> the listed versions. Only blocker issues can block a
> >>>>>>>>> release,
> >>>>>>>>>>> all
> >>>>>>>>>>>>>> other
> >>>>>>>>>>>>>>>>> tickets which have "fix version/s" set at the time of
> >>>>>> a
> >>>>>>>>>> release
> >>>>>>>>>>>> and
> >>>>>>>>>>>>>> are
> >>>>>>>>>>>>>>>>> unresolved will be moved to the next version.
> >>>>>>>>>>>>>>>>> *Assignee:*
> >>>>>>>>>>>>>>>>> Person currently working on the ticket. Assigned after
> >>>>>>>>>>> conclusion
> >>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>> approach by a committer.
> >>>>>>>>>>>>>>>>> Often, fixes are obvious and committers self-assign
> >>>>>> w/o
> >>>>>>>>>>>> discussion.
> >>>>>>>>>>>>>>>>> *Resolve / Close:*
> >>>>>>>>>>>>>>>>> You can either Resolve or Close a ticket once it is
> >>>>>> done
> >>>>>>>>>> (fixed,
> >>>>>>>>>>>>>>> rejected,
> >>>>>>>>>>>>>>>>> invalid, ...).
> >>>>>>>>>>>>>>>>> As a rule, we Close tickets instead of Resolving them
> >>>>>>> when
> >>>>>>>>>> they
> >>>>>>>>>>>> are
> >>>>>>>>>>>>>>> done.
> >>>>>>>>>>>>>>>>> Background: There are semantic differences for Resolve
> >>>>>>> and
> >>>>>>>>>> Close
> >>>>>>>>>>>>>>>>> (implementor vs reporter considers it done), but I
> >>>>>> don't
> >>>>>>>> see
> >>>>>>>>>> how
> >>>>>>>>>>>>> they
> >>>>>>>>>>>>>>>>> practically apply to the Flink project. Looking at the
> >>>>>>>>>> numbers,
> >>>>>>>>>>>>> Flink
> >>>>>>>>>>>>>>> has
> >>>>>>>>>>>>>>>>> 11066 closed tickets, and 3372 resolved tickets
> >>>>>> (that's
> >>>>>>>> why
> >>>>>>>>> I
> >>>>>>>>>>>>> propose
> >>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>> close instead of resolve)
> >>>>>>>>>>>>>>>>> *Labels:*
> >>>>>>>>>>>>>>>>> "test-stability" for all test instabilities
> >>>>>>>>>>>>>>>>> "starter" for tickets suitable for new contributors
> >>>>>>>>>>>>>>>>> *Release Note:*
> >>>>>>>>>>>>>>>>> Small notes that will be included into the release
> >>>>>> notes
> >>>>>>>>>>> published
> >>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> release.
> >>>>>>>>>>>>>>>>> *All other fields are not used not used on a regular
> >>>>>>>> basis.*
> >>>>>>>>>>>>>>>>> Please +1 my proposal if you want it to be published
> >>>>>> in
> >>>>>>>> our
> >>>>>>>>>> Wiki
> >>>>>>>>>>>>> like
> >>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>> or let me know if I got something wrong here.
> >>>>>>>>>>>>>>>>> Best,
> >>>>>>>>>>>>>>>>> Robert
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> --
> >>>>>>>>> Best, Jingsong Lee
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>
> >>>>
> >>>
> >>
>
>
Re: [DISCUSS] Semantics of our JIRA fields
Posted by Piotr Nowojski <pi...@ververica.com>.
> On 12 Jun 2020, at 15:44, Robert Metzger <rm...@apache.org> wrote:
>
> Piotrek, do you agree with my "affects version" explanation? I would like
> to bring this discussion to a conclusion.
>
+0 for this semantic from my side.
> On Tue, May 26, 2020 at 4:51 PM Till Rohrmann <tr...@apache.org> wrote:
>
>> If we change the meaning of the priority levels, then I would suggest to
>> have a dedicated discussion for it. This would also be more visible than
>> compared to being hidden in some lengthy discussion thread. I think the
>> proposed definitions of priority levels differ slightly from how the
>> community worked in the past.
>>
>> Cheers,
>> Till
>>
>> On Tue, May 26, 2020 at 4:30 PM Robert Metzger <rm...@apache.org>
>> wrote:
>>
>>> Hi,
>>>
>>> 1. I'm okay with updating the definition of the priorities for the reason
>>> you've mentioned.
>>>
>>> 2. "Affects version"
>>>
>>> The reason why like to mark affects version on unreleased versions is to
>>> clearly indicate which branch is affected by a bug. Given the current
>> Flink
>>> release status, if there's a bug only in "release-1.11", but not in
>>> "master", there is no way of figuring that out, if we only allow released
>>> versions for "affects version" (In my proposal, you would set "affects
>>> version" to '1.11.0', '1.12.0' to indicate that).
>>>
>>> What we could do is introduce "1.12-SNAPSHOT" as version to mark issues
>> on
>>> unreleased versions. (But then people might accidentally set the "fix
>>> version" to a "-SNAPSHOT" version.)
>>>
>>> I'm still in favor of my proposal. I have never heard a report from a
>>> confused user about our Jira fields (I guess they usually check bugs for
>>> released versions only)
>>>
>>>
>>> On Tue, May 26, 2020 at 12:39 PM Piotr Nowojski <pi...@ververica.com>
>>> wrote:
>>>
>>>> Hi,
>>>>
>>>> Sorry for a bit late response. I have two concerns:
>>>>
>>>> 1. Priority
>>>>
>>>> I would propose to stretch priorities that we are using to
>> differentiate
>>>> between things that must be fixed for given release:
>>>>
>>>> BLOCKER - drop anything you are doing, this issue must be fixed right
>> now
>>>> CRITICAL - release can not happen without fixing it, but can be fixed a
>>>> bit later (for example without context switching and dropping whatever
>>> I’m
>>>> doing right now)
>>>> MAJOR - default, nice to have
>>>> Anything below - meh
>>>>
>>>> We were already using this semantic for tracking test instabilities
>>> during
>>>> the 1.11 release cycle. Good examples:
>>>>
>>>> BLOCKER - master branch not compiling, very frequent test failures (for
>>>> example almost every build affected), …
>>>> CRITICAL - performance regression/bug that we introduced in some
>> feature,
>>>> but which is not affecting other developers as much
>>>> MAJOR - freshly discovered test instability with unknown
>> impact/frequency
>>>> (could be happening once a year),
>>>>
>>>> 2. Affects version
>>>>
>>>> If bug is only on the master branch, does it affect an unreleased
>>> version?
>>>>
>>>> So far I was assuming that it doesn’t - unreleased bugs would have
>> empty
>>>> “affects version” field. My reasoning was that this field should be
>> used
>>>> for Flink users, to check which RELEASED Flink versions are affected by
>>>> some bug, that user is searching for. Otherwise it might be a bit
>>> confusing
>>>> if there are lots of bugs with both affects version and fix version set
>>> to
>>>> the same value.
>>>>
>>>> Piotrek
>>>>
>>>>> On 25 May 2020, at 16:40, Robert Metzger <rm...@apache.org>
>> wrote:
>>>>>
>>>>> Hi all,
>>>>> thanks a lot for the feedback. The majority of responses are very
>>>> positive
>>>>> to my proposal.
>>>>>
>>>>> I have put my proposal into our wiki:
>>>>>
>>>>
>>>
>> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=154995514
>>>>>
>>>>> Regarding the comments so far:
>>>>> @Jark: I clarified this in the wiki.
>>>>>
>>>>> @Israel: I have not considered build changing all 3000 resolved
>> tickets
>>>> to
>>>>> closed yet, but after consideration I don't think it is necessary. If
>>>>> others in the community would like to change them, please speak up in
>>>> this
>>>>> thread :)
>>>>>
>>>>> @Flavio: I agree that we can not ask new or infrequent users to fully
>>>>> adhere to these definitions. I added a note in the Wiki.
>>>>> Using the resolved state for indicating "PR available" is problematic
>>>>> because there are plenty of cases where PRs are stale (and this
>> ticket
>>>>> would then appear as a "resolved"). The Apache tools are adding a
>> link
>>> to
>>>>> the PR, and some contributors are setting the ticket to "In
>> Progress".
>>> I
>>>>> don't see a problem that we need to solve here.
>>>>>
>>>>> @Yun: Thank you for your comment. I added an example clarifying how I
>>>> would
>>>>> handle such a case. It is slightly different from your proposal: You
>>>>> suggested using x.y.0 versions, I used "the next supported,
>> unreleased
>>>>> version", because that's how we've done it so far (and I don't want
>> to
>>>>> change things, I just want to document how the majority of the core
>>>>> contributors are using JIRA).
>>>>>
>>>>> Here are all the changes (in green, blue are just formatting
>> changes) I
>>>>> made compared to my initial proposal:
>>>>>
>>>>
>>>
>> https://cwiki.apache.org/confluence/pages/diffpagesbyversion.action?pageId=154995514&selectedPageVersions=4&selectedPageVersions=1
>>>>>
>>>>>
>>>>>
>>>>> On Mon, May 25, 2020 at 2:28 PM Congxian Qiu <qcx978132955@gmail.com
>>>
>>>> wrote:
>>>>>
>>>>>> @chesnay@apache.org <ch...@apache.org> Thanks for the
>> confirmation
>>>>>>
>>>>>> Best,
>>>>>> Congxian
>>>>>>
>>>>>>
>>>>>> Zhu Zhu <re...@gmail.com> 于2020年5月25日周一 下午4:13写道：
>>>>>>
>>>>>>> This is very helpful!
>>>>>>> +1
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Zhu Zhu
>>>>>>>
>>>>>>> Yang Wang <da...@gmail.com> 于2020年5月25日周一 下午4:04写道：
>>>>>>>
>>>>>>>> +1 from this useful proposal.
>>>>>>>>
>>>>>>>> This makes me clearer about "Resolve" and "Close" since I used to
>> be
>>>>>>>> confused by this two button.
>>>>>>>>
>>>>>>>> Best,
>>>>>>>> Yang
>>>>>>>>
>>>>>>>> Jingsong Li <ji...@gmail.com> 于2020年5月25日周一 下午3:10写道：
>>>>>>>>
>>>>>>>>> +1 for the proposal.
>>>>>>>>> It makes me clearer.
>>>>>>>>>
>>>>>>>>> Best,
>>>>>>>>> Jingsong Lee
>>>>>>>>>
>>>>>>>>> On Mon, May 25, 2020 at 2:51 PM Zhijiang <
>>> wangzhijiang999@aliyun.com
>>>>>>>>> .invalid>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Thanks for launching this discussion and giving so detailed
>> infos,
>>>>>>>>>> Robert! +1 on my side for the proposal.
>>>>>>>>>>
>>>>>>>>>> For "Affects Version", I previously thought it was only for the
>>>>>>> already
>>>>>>>>>> released versions, so it can give a reminder that the fix should
>>>>>> also
>>>>>>>>> pick
>>>>>>>>>> into the related released branches for future minor versions.
>>>>>>>>>> I saw that Jark had somehow similar concerns for this field in
>>>>>> below
>>>>>>>>>> replies. Either way makes sense for me as long as we give a
>>>>>>> determined
>>>>>>>>>> rule in Wiki.
>>>>>>>>>>
>>>>>>>>>> Re Flavio' s comments, I agree that the Jira reporter can leave
>>>>>> most
>>>>>>> of
>>>>>>>>>> the fields empty if not confirmed of them, then the respective
>>>>>>>> component
>>>>>>>>>> maintainer or committer can update them accordingly later.
>>>>>>>>>> But the state of Jira should not be marked as "resolved" when
>> the
>>>>>> PR
>>>>>>> is
>>>>>>>>>> detected, that is not fitting into the resolved semantic I
>> guess.
>>>>>> If
>>>>>>>>>> possible, the Jira can be updated as "in progress" automatically
>>> if
>>>>>>>>>> the respective PR is ready, then it will save some time to stat
>>>>>>>> progress
>>>>>>>>>> of related issues during release process.
>>>>>>>>>>
>>>>>>>>>> Best,
>>>>>>>>>> Zhijiang
>>>>>>>>>>
>> ------------------------------------------------------------------
>>>>>>>>>> From:Congxian Qiu <qc...@gmail.com>
>>>>>>>>>> Send Time:2020年5月25日(星期一) 13:57
>>>>>>>>>> To:dev@flink.apache.org <de...@flink.apache.org>
>>>>>>>>>> Subject:Re: [DISCUSS] Semantics of our JIRA fields
>>>>>>>>>>
>>>>>>>>>> Hi
>>>>>>>>>>
>>>>>>>>>> Currently, when I'm going to create an issue for
>> Project-website.
>>>>>> I'm
>>>>>>>> not
>>>>>>>>>> very sure what the "Affect Version/s" should be set. The problem
>>> is
>>>>>>>> that
>>>>>>>>>> the current Dockfile[1] in flink-web is broken because of the
>> EOL
>>>>>> of
>>>>>>>>> Ubuntu
>>>>>>>>>> 18.10[2], the project-web does not affect any release of Flink,
>> it
>>>>>>> does
>>>>>>>>>> affect the process to build website, so what's the version
>> should
>>> I
>>>>>>> set
>>>>>>>>> to?
>>>>>>>>>>
>>>>>>>>>> [1]
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>
>>>
>> https://github.com/apache/flink-web/blob/bc66f0f0f463ab62a22e81df7d7efd301b76a6b4/docker/Dockerfile#L17
>>>>>>>>>> [2] https://wiki.ubuntu.com/Releases
>>>>>>>>>>
>>>>>>>>>> Best,
>>>>>>>>>> Congxian
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Flavio Pompermaier <po...@okkam.it> 于2020年5月24日周日
>> 下午1:23写道：
>>>>>>>>>>
>>>>>>>>>>> In my experience it's quite complicated for a normal reporter
>> to
>>>>>> be
>>>>>>>>> able
>>>>>>>>>> to
>>>>>>>>>>> fill all the fields correctly (especially for new users).
>>>>>>>>>>> Usually you just wanto to report a problem, remember to add a
>> new
>>>>>>>>> feature
>>>>>>>>>>> or improve code/documentation but you can't really give a
>>>>>> priority,
>>>>>>>>>> assign
>>>>>>>>>>> the correct label or decide which releases will benefit of the
>>>>>>>> fix/new
>>>>>>>>>>> feature. This is something that only core developers could
>> decide
>>>>>>>>> (IMHO).
>>>>>>>>>>>
>>>>>>>>>>> My experiece says that it's better if normal users could just
>>>>>> open
>>>>>>>>>> tickets
>>>>>>>>>>> with some default (or mark ticket as new) and leave them in
>> such
>>>>>> a
>>>>>>>>> state
>>>>>>>>>>> until an experienced user, one that can assign tickets, have
>> the
>>>>>>> time
>>>>>>>>> to
>>>>>>>>>>> read it and immediately reject the ticket or accept it and
>>>>>> properly
>>>>>>>>>> assign
>>>>>>>>>>> priorities, fix version, etc.
>>>>>>>>>>>
>>>>>>>>>>> With respect to resolve/close I think that a good practice
>> could
>>>>>> be
>>>>>>>> to
>>>>>>>>>> mark
>>>>>>>>>>> automatically a ticket as resolved once that a PR is detected
>> for
>>>>>>>> that
>>>>>>>>>>> ticket, while marking it as closed should be done by the
>> commiter
>>>>>>> who
>>>>>>>>>> merge
>>>>>>>>>>> the PR.
>>>>>>>>>>>
>>>>>>>>>>> Probably this process would slightly increase the work of a
>>>>>>> committer
>>>>>>>>> but
>>>>>>>>>>> this would make things a lot cleaner and will bring the benefit
>>>>>> of
>>>>>>>>>> having a
>>>>>>>>>>> reliable and semantically sound JIRA state.
>>>>>>>>>>>
>>>>>>>>>>> Cheers,
>>>>>>>>>>> Flavio
>>>>>>>>>>>
>>>>>>>>>>> Il Dom 24 Mag 2020, 05:05 Israel Ekpo <is...@gmail.com>
>> ha
>>>>>>>>> scritto:
>>>>>>>>>>>
>>>>>>>>>>>> +1 for the proposal
>>>>>>>>>>>>
>>>>>>>>>>>> This will bring some consistency to the process
>>>>>>>>>>>>
>>>>>>>>>>>> Regarding Closed vs Resolved, should we go back and switch all
>>>>>>> the
>>>>>>>>>>> Resolved
>>>>>>>>>>>> issues to Closed so that is is not confusing to new people to
>>>>>> the
>>>>>>>>>> project
>>>>>>>>>>>> in the future that may not have seen this discussion?
>>>>>>>>>>>>
>>>>>>>>>>>> I would recommend changing it to Closed just to be consistent
>>>>>>> since
>>>>>>>>>> that
>>>>>>>>>>> is
>>>>>>>>>>>> the majority and the new process will be using Closed going
>>>>>>> forward
>>>>>>>>>>>>
>>>>>>>>>>>> Those are my thoughts for now
>>>>>>>>>>>>
>>>>>>>>>>>> On Sat, May 23, 2020 at 7:48 AM Congxian Qiu <
>>>>>>>> qcx978132955@gmail.com
>>>>>>>>>>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> +1 for the proposal. It's good to have a unified description
>>>>>>> and
>>>>>>>>>> write
>>>>>>>>>>> it
>>>>>>>>>>>>> down in the wiki, so that every contributor has the same
>>>>>>>>>> understanding
>>>>>>>>>>> of
>>>>>>>>>>>>> all the fields.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Best,
>>>>>>>>>>>>> Congxian
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Till Rohrmann <tr...@apache.org> 于2020年5月23日周六
>>>>>> 上午12:04写道：
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks for drafting this proposal Robert. +1 for the
>>>>>>> proposal.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>>> Till
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Fri, May 22, 2020 at 5:39 PM Leonard Xu <
>>>>>>> xbjtdcq@gmail.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks bringing up this topic @Robert, +1 to the
>>>>>> proposal.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> It clarifies the JIRA fields well and should be a rule to
>>>>>>>>> follow.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Best,
>>>>>>>>>>>>>>> Leonard Xu
>>>>>>>>>>>>>>>> 在 2020年5月22日，20:24，Aljoscha Krettek <
>>>>>> aljoscha@apache.org
>>>>>>>>
>>>>>>>>> 写道：
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> +1 That's also how I think of the semantics of the
>>>>>>> fields.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Aljoscha
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On 22.05.20 08:07, Robert Metzger wrote:
>>>>>>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>>>>>> I have the feeling that the semantics of some of our
>>>>>>> JIRA
>>>>>>>>>> fields
>>>>>>>>>>>>>> (mostly
>>>>>>>>>>>>>>>>> "affects versions", "fix versions" and resolve /
>>>>>> close)
>>>>>>>> are
>>>>>>>>>> not
>>>>>>>>>>>>>> defined
>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>> the same way by all the core Flink contributors, which
>>>>>>>> leads
>>>>>>>>>> to
>>>>>>>>>>>>> cases
>>>>>>>>>>>>>>> where
>>>>>>>>>>>>>>>>> I spend quite some time on filling the fields
>>>>>> correctly
>>>>>>>> (at
>>>>>>>>>>> least
>>>>>>>>>>>>>> what I
>>>>>>>>>>>>>>>>> consider correctly), and then others changing them
>>>>>> again
>>>>>>>> to
>>>>>>>>>>> match
>>>>>>>>>>>>>> their
>>>>>>>>>>>>>>>>> semantics.
>>>>>>>>>>>>>>>>> In an effort to increase our efficiency, and since I'm
>>>>>>>>>> creating
>>>>>>>>>>> a
>>>>>>>>>>>>> lot
>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>> (test instability-related) tickets these days, I would
>>>>>>>> like
>>>>>>>>> to
>>>>>>>>>>>>> discuss
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> semantics, come to a conclusion and document this in
>>>>>> our
>>>>>>>>> Wiki.
>>>>>>>>>>>>>>>>> *Proposal:*
>>>>>>>>>>>>>>>>> *Priority:*
>>>>>>>>>>>>>>>>> "Blocker": needs to be resolved before a release
>>>>>>> (matched
>>>>>>>>>> based
>>>>>>>>>>> on
>>>>>>>>>>>>> fix
>>>>>>>>>>>>>>>>> versions)
>>>>>>>>>>>>>>>>> "Critical": strongly considered before a release
>>>>>>>>>>>>>>>>> other priorities have no practical meaning in Flink.
>>>>>>>>>>>>>>>>> *Component/s:*
>>>>>>>>>>>>>>>>> Primary component relevant for this feature / fix.
>>>>>>>>>>>>>>>>> For test-related issues, add the component the test
>>>>>>>> belongs
>>>>>>>>> to
>>>>>>>>>>>> (for
>>>>>>>>>>>>>>> example
>>>>>>>>>>>>>>>>> "Connectors / Kafka" for Kafka test failures) +
>>>>>> "Test".
>>>>>>>>>>>>>>>>> The same applies for documentation tickets. For
>>>>>> example,
>>>>>>>> if
>>>>>>>>>>>> there's
>>>>>>>>>>>>>>>>> something wrong with the DataStream API, add it to the
>>>>>>>> "API
>>>>>>>>> /
>>>>>>>>>>>>>>> DataStream"
>>>>>>>>>>>>>>>>> and "Documentation" components.
>>>>>>>>>>>>>>>>> *Affects Version/s:*Only for Bug / Task-type tickets:
>>>>>> We
>>>>>>>>> list
>>>>>>>>>>> all
>>>>>>>>>>>>>>> currently
>>>>>>>>>>>>>>>>> supported and unreleased Flink versions known to be
>>>>>>>> affected
>>>>>>>>>> by
>>>>>>>>>>>>> this.
>>>>>>>>>>>>>>>>> Example: If I see a test failure that happens on
>>>>>>> "master"
>>>>>>>>> and
>>>>>>>>>>>>>>>>> "release-1.11", I set "affects version" to "1.12.0"
>>>>>> and
>>>>>>>>>>> "1.11.0".
>>>>>>>>>>>>>>>>> *Fix Version/s:*
>>>>>>>>>>>>>>>>> For closed/resolved tickets, this field lists the
>>>>>>> released
>>>>>>>>>> Flink
>>>>>>>>>>>>>>> versions
>>>>>>>>>>>>>>>>> that contain a fix or feature for the first time.
>>>>>>>>>>>>>>>>> For open tickets, it indicates that a fix / feature
>>>>>>> should
>>>>>>>>> be
>>>>>>>>>>>>>> contained
>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>> the listed versions. Only blocker issues can block a
>>>>>>>>> release,
>>>>>>>>>>> all
>>>>>>>>>>>>>> other
>>>>>>>>>>>>>>>>> tickets which have "fix version/s" set at the time of
>>>>>> a
>>>>>>>>>> release
>>>>>>>>>>>> and
>>>>>>>>>>>>>> are
>>>>>>>>>>>>>>>>> unresolved will be moved to the next version.
>>>>>>>>>>>>>>>>> *Assignee:*
>>>>>>>>>>>>>>>>> Person currently working on the ticket. Assigned after
>>>>>>>>>>> conclusion
>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>> approach by a committer.
>>>>>>>>>>>>>>>>> Often, fixes are obvious and committers self-assign
>>>>>> w/o
>>>>>>>>>>>> discussion.
>>>>>>>>>>>>>>>>> *Resolve / Close:*
>>>>>>>>>>>>>>>>> You can either Resolve or Close a ticket once it is
>>>>>> done
>>>>>>>>>> (fixed,
>>>>>>>>>>>>>>> rejected,
>>>>>>>>>>>>>>>>> invalid, ...).
>>>>>>>>>>>>>>>>> As a rule, we Close tickets instead of Resolving them
>>>>>>> when
>>>>>>>>>> they
>>>>>>>>>>>> are
>>>>>>>>>>>>>>> done.
>>>>>>>>>>>>>>>>> Background: There are semantic differences for Resolve
>>>>>>> and
>>>>>>>>>> Close
>>>>>>>>>>>>>>>>> (implementor vs reporter considers it done), but I
>>>>>> don't
>>>>>>>> see
>>>>>>>>>> how
>>>>>>>>>>>>> they
>>>>>>>>>>>>>>>>> practically apply to the Flink project. Looking at the
>>>>>>>>>> numbers,
>>>>>>>>>>>>> Flink
>>>>>>>>>>>>>>> has
>>>>>>>>>>>>>>>>> 11066 closed tickets, and 3372 resolved tickets
>>>>>> (that's
>>>>>>>> why
>>>>>>>>> I
>>>>>>>>>>>>> propose
>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>> close instead of resolve)
>>>>>>>>>>>>>>>>> *Labels:*
>>>>>>>>>>>>>>>>> "test-stability" for all test instabilities
>>>>>>>>>>>>>>>>> "starter" for tickets suitable for new contributors
>>>>>>>>>>>>>>>>> *Release Note:*
>>>>>>>>>>>>>>>>> Small notes that will be included into the release
>>>>>> notes
>>>>>>>>>>> published
>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> release.
>>>>>>>>>>>>>>>>> *All other fields are not used not used on a regular
>>>>>>>> basis.*
>>>>>>>>>>>>>>>>> Please +1 my proposal if you want it to be published
>>>>>> in
>>>>>>>> our
>>>>>>>>>> Wiki
>>>>>>>>>>>>> like
>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>> or let me know if I got something wrong here.
>>>>>>>>>>>>>>>>> Best,
>>>>>>>>>>>>>>>>> Robert
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Best, Jingsong Lee
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>
>>>>
>>>
>>