You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@yetus.apache.org by Jonathan Eagles <je...@gmail.com> on 2020/12/17 23:11:27 UTC
Help with yetus github integration for Tez PreCommit
Essentially, I want the yetus report to comment on the appropriate jira. I
have tried the effective command-line from curl with success, verifying
credentials. However, when running with cloudbees setup, debugging has
shown JIRA_ISSUE is empty. So the effective URL changes to as follows
(secrets hidden), both of which return successfully.
curl -X POST -H "Accept: application/json" -H "Content-Type:
application/json" -d '{"body" : "test jira rest api message"}' --location "
https://issues.apache.org/jira/rest/api/2/issue/TEZ-3891/comment" -u
"user:secret"
echo $?
0
curl -X POST -H "Accept: application/json" -H "Content-Type:
application/json" -d '{"body" : "test jira rest api message"}' --location "
https://issues.apache.org/jira/rest/api/2/issue/comment" -u "user:secret"
echo $?
0
Questions,
1) How does JIRA_ISSUE get extracted from PR? (branch-name, commit message,
PR title)
Issues: TEZ-3891
2) Is there any interest in a fix that would improve this silent failure
where JIRA is not determined?
Setup
Tez PR. https://github.com/apache/tez/pull/85
Yetus: rel/0.12.0
Jenkins ci build: derived from hadoop-multipbranch
debugging enabled for below build.
https://ci-hadoop.apache.org/view/Tez/job/tez-multibranch/view/change-requests/job/PR-85/17/console
Thanks,
Jon
Re: Help with yetus github integration for Tez PreCommit
Posted by Allen Wittenauer <aw...@apache.org>.
> On Dec 22, 2020, at 7:05 PM, Jonathan Eagles <je...@gmail.com> wrote:
>
> Thanks for your reply. Commenting to github and possibly mirroring to JIRA
> would be fine.
>
> In this case the yetus report does not get posted to either jira or github.
> Instead I get a PR check status update that shows the success status of the
> jenkins build and a link back.
> PR reference: https://github.com/apache/tez/pull/85
>
> It sounds like something is not working with the github comment, but the
> build check. Those look like they have separate credentials being used as
> one is part of yetus and one is part of jenkins.
>
> Upgrading to 0.13.0 would require using the github app feature. I tried
> upgrading to 0.13.0 and using the built-in credentials from apache's
> ci-hadoop, still no comments, but also build check status stopped working.
> I will try running off of a fork adding some debugging information to the
> github comment feature as the output is silenced and no failures are
> printed even with --debugging on.
Looking at what is going on, it appears that Tez is way way way off-course, even with the current stuff it is running.
Try something like this:
https://github.com/aw-was-here/tez/tree/y0130
Not really tested, but should push you closer to what you probably want/need. It was tempting to just use a FROM of apache/yetus:0.13.0 and then plop in that ancient version of protobuf to make the Docker build super fast. But giving you a raw Dockerfile should help out if someone wants to see what is happening under the hood.
Re: Help with yetus github integration for Tez PreCommit
Posted by Jonathan Eagles <je...@gmail.com>.
Thanks for your reply. Commenting to github and possibly mirroring to JIRA
would be fine.
In this case the yetus report does not get posted to either jira or github.
Instead I get a PR check status update that shows the success status of the
jenkins build and a link back.
PR reference: https://github.com/apache/tez/pull/85
It sounds like something is not working with the github comment, but the
build check. Those look like they have separate credentials being used as
one is part of yetus and one is part of jenkins.
Upgrading to 0.13.0 would require using the github app feature. I tried
upgrading to 0.13.0 and using the built-in credentials from apache's
ci-hadoop, still no comments, but also build check status stopped working.
I will try running off of a fork adding some debugging information to the
github comment feature as the output is silenced and no failures are
printed even with --debugging on.
Jon
On Fri, Dec 18, 2020 at 1:25 AM Allen Wittenauer <aw...@apache.org> wrote:
>
>
> > On Dec 17, 2020, at 3:11 PM, Jonathan Eagles <je...@gmail.com> wrote:
> >
> > Essentially, I want the yetus report to comment on the appropriate jira.
> I
> > have tried the effective command-line from curl with success, verifying
> > credentials. However, when running with cloudbees setup, debugging has
> > shown JIRA_ISSUE is empty. So the effective URL changes to as follows
> > (secrets hidden), both of which return successfully.
> >
> ...
>
> > Questions,
> > 1) How does JIRA_ISSUE get extracted from PR? (branch-name, commit
> message,
> > PR title)
> > Issues: TEZ-3891
>
> OK, with this extra bit of info, I think I understand what you're
> trying to accomplish. Please correct me if I'm wrong:
>
> a) JIRA issue gets filed
> b) Github PR referencing that JIRA issue is opened
> c) Yetus runs
> d) report shows up in Github (as a comment, since you specifically
> mentioned 0.12.0 later on... that functionality is very different and much
> more Github-native in 0.13.0)
> e) report does NOT show up in JIRA but you would like it to
>
> If that's correct, then ...
>
> At one point in time, Yetus was specifically coded to pick _one_
> bug system (the originating bug system) to file the report. For example,
> before PRs were widely accepted, it was possible to reference a GitHub PR
> URL in JIRA and Yetus would pick up the diff from GitHub but put the
> comment in JIRA since that was where Yetus first learned of the PR. I'm
> fairly certain those restrictions are still in place.
>
> The actual answer, BTW, is from the PR title. But I seem to
> recall that direction (GH -> JIRA) was buggy and/or missing some
> conventions that people use. But frankly no one ever used it because it
> seems once people start using PRs, they want the comments there too and
> quickly abandon JIRA for most of the discussion. [For $DAYJOB, I'm
> contemplating the changes needed/total rewrite to make releasedocmaker work
> natively with Github since that's all we use. I suspect once that happens,
> folks in the ASF who are using releasedocmaker will also pretty much stop
> using JIRA for changelog/release notes info too. Especially since I don't
> see JIRA being viable for the ASF for long with some of the changes
> Atlassian is making.]
>
> > 2) Is there any interest in a fix that would improve this silent failure
> > where JIRA is not determined?
>
> As stated above, I'm fairly certain it is this way by the original
> design. But times change ... With the changes in how 0.13.0 reports back
> to GitHub (no more comments but instead Status and optionally Checks
> Annotations), it probably makes sense to break that limitation, assuming it
> is still there. Most people outside the ASF don't use two systems so it'd
> really only impact ASF folks. I'll have to look a bit closer at how the
> various *_determine_issue functions work and if it is even possible to
> support two+. (My hunch is yes, but will need some work on making it
> deterministic. ie, if JIRA references a GitHub PR, but the PR title
> references a different JIRA issue, which one is used?)
>
> It should also be noted that ASF's gitbox has ways to mirror
> GitHub comments to JIRA. That's probably the fastest way to get double
> publishing.
>
>
> In the end, if you really want that functionality, please file an
> issue and I'll try to take a look at it over the holiday break assuming no
> one else gets to it. Should have some time then.
>
> Thanks.
Re: Help with yetus github integration for Tez PreCommit
Posted by Allen Wittenauer <aw...@apache.org>.
> On Dec 17, 2020, at 3:11 PM, Jonathan Eagles <je...@gmail.com> wrote:
>
> Essentially, I want the yetus report to comment on the appropriate jira. I
> have tried the effective command-line from curl with success, verifying
> credentials. However, when running with cloudbees setup, debugging has
> shown JIRA_ISSUE is empty. So the effective URL changes to as follows
> (secrets hidden), both of which return successfully.
>
...
> Questions,
> 1) How does JIRA_ISSUE get extracted from PR? (branch-name, commit message,
> PR title)
> Issues: TEZ-3891
OK, with this extra bit of info, I think I understand what you're trying to accomplish. Please correct me if I'm wrong:
a) JIRA issue gets filed
b) Github PR referencing that JIRA issue is opened
c) Yetus runs
d) report shows up in Github (as a comment, since you specifically mentioned 0.12.0 later on... that functionality is very different and much more Github-native in 0.13.0)
e) report does NOT show up in JIRA but you would like it to
If that's correct, then ...
At one point in time, Yetus was specifically coded to pick _one_ bug system (the originating bug system) to file the report. For example, before PRs were widely accepted, it was possible to reference a GitHub PR URL in JIRA and Yetus would pick up the diff from GitHub but put the comment in JIRA since that was where Yetus first learned of the PR. I'm fairly certain those restrictions are still in place.
The actual answer, BTW, is from the PR title. But I seem to recall that direction (GH -> JIRA) was buggy and/or missing some conventions that people use. But frankly no one ever used it because it seems once people start using PRs, they want the comments there too and quickly abandon JIRA for most of the discussion. [For $DAYJOB, I'm contemplating the changes needed/total rewrite to make releasedocmaker work natively with Github since that's all we use. I suspect once that happens, folks in the ASF who are using releasedocmaker will also pretty much stop using JIRA for changelog/release notes info too. Especially since I don't see JIRA being viable for the ASF for long with some of the changes Atlassian is making.]
> 2) Is there any interest in a fix that would improve this silent failure
> where JIRA is not determined?
As stated above, I'm fairly certain it is this way by the original design. But times change ... With the changes in how 0.13.0 reports back to GitHub (no more comments but instead Status and optionally Checks Annotations), it probably makes sense to break that limitation, assuming it is still there. Most people outside the ASF don't use two systems so it'd really only impact ASF folks. I'll have to look a bit closer at how the various *_determine_issue functions work and if it is even possible to support two+. (My hunch is yes, but will need some work on making it deterministic. ie, if JIRA references a GitHub PR, but the PR title references a different JIRA issue, which one is used?)
It should also be noted that ASF's gitbox has ways to mirror GitHub comments to JIRA. That's probably the fastest way to get double publishing.
In the end, if you really want that functionality, please file an issue and I'll try to take a look at it over the holiday break assuming no one else gets to it. Should have some time then.
Thanks.