Re: [TC Bot] Proposal of improvement

[Dmitriy Pavlov <dp...@gmail.com>]

Hi Nikolay, Igniters,

Please check the newest version of the Bot.

It contains huge UI refactoring and simplified navigation to open PR and
its test results. Just couple seconds is needed to find a PR now.

Thanks to Dmitrii Ryabov, who developed initial GitHub integration. It is
more or less reused, and PRs are cached in the Ignite instance.

As always, I appreciate the feedback.

Sincerely,
Dmitriy Pavlov

вт, 25 сент. 2018 г. в 13:00, Dmitriy Pavlov <dp...@gmail.com>:

> JIRA ticket is some elementary change that needs to be reviewed. We don't
> review the patch, we review ticket (with motivation, implementation
> details, history of discussion), so reviewer will look at a ticket first.
>
> PR does not have a mark, that it is ready to be merged. Some PRs are
> created just for research, but Patch Available ticket is something that is
> ready to be in the product.
>
> So if we concentrate on a ticket, from the point of view of a new
> contributor,
> - he or she creates a branch, PR and sets ticket to PA,
> - and the bot will do all necessary mechanic work.
>
> No issues with asking newcomers to run (proper) tests. A newbie needs only
> to follow the first steps in How To Contribute. A reviewer will see a
> ticket with the bot Visa after 2-3 hours after setting of PA state.
>
> But only one concern I have here, I'm not sure I have spare cycles to do
> this project. I'd rather move towards it in step-by-step mode, doing small
> changes in each version. Any assistance is appreciated here.
>
> вт, 25 сент. 2018 г. в 11:25, Nikolay Izhikov <ni...@apache.org>:
>
>> Hello, Dmitriy
>>
>> > What about the case when committer creates ignite-9679 branch and tests
>> it> without PR?
>>
>> It means, committer is experienced enough to run tests via Team City
>> interface :).
>>
>> > So scanning seems to be possible only in JIRA
>>
>> I don't understand you here.
>> You can retrieve comments filtered by *date*.
>> You don't have to scan all 1000 PR's one by one.
>> Anyway 1000 PR doesn't sound like big issue for me.
>>
>> My vote goes strong to GiHub user interface.
>> I think we should have closer integration with GitHub, not Jira.
>>
>> Jira is about tickets and project management.
>> GitHub is about code, commits and patches.
>> We test patch, not ticket.
>>
>>
>> В Вт, 25/09/2018 в 00:06 +0300, Dmitriy Pavlov пишет:
>> > Hi Nikolay,
>> >
>> > What about the case when committer creates ignite-9679 branch and tests
>> it
>> > without PR?
>> >
>> > We have 1100+ open PRs and less than 100 open tickets. So scanning
>> seems to
>> > be possible only in JIRA. Mention probably will work for GitHub, but it
>> > needs to be researched.
>> >
>> > Two open PRs is not a valid situation in the majority of cases and How
>> To
>> > Contribute asks to avoid it. The bot can ignore closed PRs and the bot
>> can
>> > expect there is only one open PR per ticket.
>> >
>> > Sincerely,
>> > Dmitriy Pavlov
>> >
>> > пн, 24 сент. 2018 г. в 23:41, Nikolay Izhikov <ni...@apache.org>:
>> >
>> > > Hello, Dmitriy.
>> > >
>> > > > But it could be a lot of work to handle mentions (probably from the
>> > >
>> > > email> account and inbox).
>> > >
>> > > Actually, it can be done via GitHub REST API [1].
>> > > It has 'since' param, so getting new GitHub comments is a very basic
>> task.
>> > >
>> > > > Patch available ticket
>> > >
>> > > I think we shouldn't take a ticket as an entity that should be tested.
>> > > For me, it's a PR.
>> > >
>> > > Moreover, it's a common case when we have several PR in a ticket.
>> > > And it's a common case when both of them has to be tested.
>> > >
>> > > My vote goes to the closer integration with GitHub.
>> > >
>> > > [1]
>> > >
>> https://developer.github.com/v3/pulls/comments/#list-comments-in-a-repository
>> > >
>> > > В Пн, 24/09/2018 в 22:36 +0300, Dmitriy Pavlov пишет:
>> > > > Hi Nikolay,
>> > > >
>> > > > The idea makes perfect sense for me, and we should definitely take
>> the
>> > >
>> > > best
>> > > > practices from other big Apache projects.
>> > > >
>> > > > But it could be a lot of work to handle mentions (probably from the
>> email
>> > > > account and inbox).
>> > > >
>> > > > I would like to suggest the following algorithm:
>> > > >
>> > > > Patch available ticket, which was never checked by the bot will be
>> > > > processed in the following steps:
>> > > > 1. check existing run all (by PR or by branch name), if found go to
>> the
>> > > > step 3
>> > > > 2. run-all to be triggered by PR
>> > > > 3. results should be analyzed for the presence of possible
>> blockers. If
>> > > > there is no blockers go to step 5.
>> > > > 4. re-run of particular suites containing possible blockers should
>> be
>> > > > applied to try to get success for very rare flaky failures (<1%).
>> Go to 3
>> > > > (this go to should be done only once).
>> > > > 5. comment should be added to JIRA ticket containing information
>> about
>> > > > results.
>> > > >
>> > > > If a ticket was processed by bot early (probably author added some
>> fixes)
>> > > > but still in PA state, the bot will check comments list and find
>> possible
>> > > > new mentions (made after the previous build complete date). If it
>> finds
>> > > > such comments it goes to step 1 (trying to find only new builds
>> > >
>> > > available).
>> > > >
>> > > > What do you think?
>> > > >
>> > > > Sincerely,
>> > > > Dmitriy Pavlov
>> > > >
>> > > > пн, 24 сент. 2018 г. в 21:43, Nikolay Izhikov <nizhikov@apache.org
>> >:
>> > > >
>> > > > > Hello, Igniters.
>> > > > >
>> > > > > I propose to implement following behaviour:
>> > > > >
>> > > > > 1. Execute "Run all" suite for specific PR when the author of PR
>> makes
>> > >
>> > > a
>> > > > > comment
>> > > > > "@mtcga.bot Run Tests!" in GitHub comments.
>> > > > >
>> > > > > 2. Send a comment with "Run All" results both: to a Jira ticket
>> and
>> > >
>> > > GitHub
>> > > > > comment.
>> > > > >
>> > > > > 3. Label PR based on "Run All" results like it done in Apache
>> Kafka [1]
>> > > > >
>> > > > > I've create ticket for this proposal [2]
>> > > > >
>> > > > > Thoughts?
>> > > > >
>> > > > > [1] https://github.com/apache/kafka/pulls
>> > > > > [2] https://issues.apache.org/jira/browse/IGNITE-9678
>
>

Re: [TC Bot] Proposal of improvement

[Dmitriy Pavlov <dp...@gmail.com>]

Hi Igniters,

I'm glad to announce that one more change is done related to PR easy
navigation. Now TC bot detects if there was a runAll and shows button
<<show report>> only if there is some completed build.

It is not a final-state of UI, but currently, I think
https://issues.apache.org/jira/browse/IGNITE-9800 is done.

In pair with the contribution done by Nikolay Kulagin (
https://issues.apache.org/jira/browse/IGNITE-9770 ), it allows navigating
from the main screen to getting bot Visa without tons of data to be entered.

I've documented flow as graph here:
https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+Teamcity+Bot#ApacheIgniteTeamcityBot-HowtocheckaPRwiththeTCBot
- this to be enhanced by automatic scenarios once it is available in simple
navigation mode.

Thanks to everyone involved. I hope I will have spare cycles to continue to
contribute.

Sincerely,
Dmitriy Pavlov


пн, 8 окт. 2018 г. в 15:39, Dmitriy Pavlov <dp...@gmail.com>:

> Hi Nikolay, Igniters,
>
> Please check the newest version of the Bot.
>
> It contains huge UI refactoring and simplified navigation to open PR and
> its test results. Just couple seconds is needed to find a PR now.
>
> Thanks to Dmitrii Ryabov, who developed initial GitHub integration. It is
> more or less reused, and PRs are cached in the Ignite instance.
>
> As always, I appreciate the feedback.
>
> Sincerely,
> Dmitriy Pavlov
>
> вт, 25 сент. 2018 г. в 13:00, Dmitriy Pavlov <dp...@gmail.com>:
>
>> JIRA ticket is some elementary change that needs to be reviewed. We don't
>> review the patch, we review ticket (with motivation, implementation
>> details, history of discussion), so reviewer will look at a ticket first.
>>
>> PR does not have a mark, that it is ready to be merged. Some PRs are
>> created just for research, but Patch Available ticket is something that is
>> ready to be in the product.
>>
>> So if we concentrate on a ticket, from the point of view of a new
>> contributor,
>> - he or she creates a branch, PR and sets ticket to PA,
>> - and the bot will do all necessary mechanic work.
>>
>> No issues with asking newcomers to run (proper) tests. A newbie needs
>> only to follow the first steps in How To Contribute. A reviewer will see a
>> ticket with the bot Visa after 2-3 hours after setting of PA state.
>>
>> But only one concern I have here, I'm not sure I have spare cycles to do
>> this project. I'd rather move towards it in step-by-step mode, doing small
>> changes in each version. Any assistance is appreciated here.
>>
>> вт, 25 сент. 2018 г. в 11:25, Nikolay Izhikov <ni...@apache.org>:
>>
>>> Hello, Dmitriy
>>>
>>> > What about the case when committer creates ignite-9679 branch and
>>> tests it> without PR?
>>>
>>> It means, committer is experienced enough to run tests via Team City
>>> interface :).
>>>
>>> > So scanning seems to be possible only in JIRA
>>>
>>> I don't understand you here.
>>> You can retrieve comments filtered by *date*.
>>> You don't have to scan all 1000 PR's one by one.
>>> Anyway 1000 PR doesn't sound like big issue for me.
>>>
>>> My vote goes strong to GiHub user interface.
>>> I think we should have closer integration with GitHub, not Jira.
>>>
>>> Jira is about tickets and project management.
>>> GitHub is about code, commits and patches.
>>> We test patch, not ticket.
>>>
>>>
>>> В Вт, 25/09/2018 в 00:06 +0300, Dmitriy Pavlov пишет:
>>> > Hi Nikolay,
>>> >
>>> > What about the case when committer creates ignite-9679 branch and
>>> tests it
>>> > without PR?
>>> >
>>> > We have 1100+ open PRs and less than 100 open tickets. So scanning
>>> seems to
>>> > be possible only in JIRA. Mention probably will work for GitHub, but it
>>> > needs to be researched.
>>> >
>>> > Two open PRs is not a valid situation in the majority of cases and How
>>> To
>>> > Contribute asks to avoid it. The bot can ignore closed PRs and the bot
>>> can
>>> > expect there is only one open PR per ticket.
>>> >
>>> > Sincerely,
>>> > Dmitriy Pavlov
>>> >
>>> > пн, 24 сент. 2018 г. в 23:41, Nikolay Izhikov <ni...@apache.org>:
>>> >
>>> > > Hello, Dmitriy.
>>> > >
>>> > > > But it could be a lot of work to handle mentions (probably from the
>>> > >
>>> > > email> account and inbox).
>>> > >
>>> > > Actually, it can be done via GitHub REST API [1].
>>> > > It has 'since' param, so getting new GitHub comments is a very basic
>>> task.
>>> > >
>>> > > > Patch available ticket
>>> > >
>>> > > I think we shouldn't take a ticket as an entity that should be
>>> tested.
>>> > > For me, it's a PR.
>>> > >
>>> > > Moreover, it's a common case when we have several PR in a ticket.
>>> > > And it's a common case when both of them has to be tested.
>>> > >
>>> > > My vote goes to the closer integration with GitHub.
>>> > >
>>> > > [1]
>>> > >
>>> https://developer.github.com/v3/pulls/comments/#list-comments-in-a-repository
>>> > >
>>> > > В Пн, 24/09/2018 в 22:36 +0300, Dmitriy Pavlov пишет:
>>> > > > Hi Nikolay,
>>> > > >
>>> > > > The idea makes perfect sense for me, and we should definitely take
>>> the
>>> > >
>>> > > best
>>> > > > practices from other big Apache projects.
>>> > > >
>>> > > > But it could be a lot of work to handle mentions (probably from
>>> the email
>>> > > > account and inbox).
>>> > > >
>>> > > > I would like to suggest the following algorithm:
>>> > > >
>>> > > > Patch available ticket, which was never checked by the bot will be
>>> > > > processed in the following steps:
>>> > > > 1. check existing run all (by PR or by branch name), if found go
>>> to the
>>> > > > step 3
>>> > > > 2. run-all to be triggered by PR
>>> > > > 3. results should be analyzed for the presence of possible
>>> blockers. If
>>> > > > there is no blockers go to step 5.
>>> > > > 4. re-run of particular suites containing possible blockers should
>>> be
>>> > > > applied to try to get success for very rare flaky failures (<1%).
>>> Go to 3
>>> > > > (this go to should be done only once).
>>> > > > 5. comment should be added to JIRA ticket containing information
>>> about
>>> > > > results.
>>> > > >
>>> > > > If a ticket was processed by bot early (probably author added some
>>> fixes)
>>> > > > but still in PA state, the bot will check comments list and find
>>> possible
>>> > > > new mentions (made after the previous build complete date). If it
>>> finds
>>> > > > such comments it goes to step 1 (trying to find only new builds
>>> > >
>>> > > available).
>>> > > >
>>> > > > What do you think?
>>> > > >
>>> > > > Sincerely,
>>> > > > Dmitriy Pavlov
>>> > > >
>>> > > > пн, 24 сент. 2018 г. в 21:43, Nikolay Izhikov <nizhikov@apache.org
>>> >:
>>> > > >
>>> > > > > Hello, Igniters.
>>> > > > >
>>> > > > > I propose to implement following behaviour:
>>> > > > >
>>> > > > > 1. Execute "Run all" suite for specific PR when the author of PR
>>> makes
>>> > >
>>> > > a
>>> > > > > comment
>>> > > > > "@mtcga.bot Run Tests!" in GitHub comments.
>>> > > > >
>>> > > > > 2. Send a comment with "Run All" results both: to a Jira ticket
>>> and
>>> > >
>>> > > GitHub
>>> > > > > comment.
>>> > > > >
>>> > > > > 3. Label PR based on "Run All" results like it done in Apache
>>> Kafka [1]
>>> > > > >
>>> > > > > I've create ticket for this proposal [2]
>>> > > > >
>>> > > > > Thoughts?
>>> > > > >
>>> > > > > [1] https://github.com/apache/kafka/pulls
>>> > > > > [2] https://issues.apache.org/jira/browse/IGNITE-9678
>>
>>