You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Adam Szita <sz...@cloudera.com> on 2017/11/06 15:43:08 UTC
Integrating Yetus with Precommit job
Hi all,
As a next step of test subsystem improvements we would like to have the
Yetus check integrated with the ptest framework. This means that whenever a
new patch is uploaded - along with the already existing Precommit test run
- Hive's Yetus patch check script would be triggered also. This script runs
checkstyle, findbugs, ASF license check, etc with and without the submitted
patch applied and reports the diffs (i.e. how many checkstyle problems does
the patch introduce).
It would be executed parallel to the ptest test execution and report back
the results as a (another) jira comment to the issue in question.
In the last days I've been working on this (HIVE-16748) and a patch is
ready to make this happen. A sample Yetus result comment is available at
https://issues.apache.org/jira/browse/HIVE-16748?focusedCommentId=16218616&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16218616
We think this would be a useful tool for us developers and would like to go
ahead with this change, but we're also curious about your input in this
matter. Please let us know what you think about this change.
Thanks,
Adam
Re: Integrating Yetus with Precommit job
Posted by Xuefu Zhang <xu...@uber.com>.
+1 on the ideas.
On Tue, Nov 7, 2017 at 6:17 AM, Adam Szita <sz...@cloudera.com> wrote:
> Thanks for all the replies.
>
> Vihang: Good idea on making everything green before turning this on. For
> this purpose I've filed a couple of jiras:
> -HIVE-17995 <https://issues.apache.org/jira/browse/HIVE-17995> Run
> checkstyle on standalone-metastore module with proper configuration
> -HIVE-17996 <https://issues.apache.org/jira/browse/HIVE-17996> Fix ASF
> headers
> -HIVE-17997 <https://issues.apache.org/jira/browse/HIVE-17997> Add rat
> plugin and configuration to standalone metastore pom
>
> Sahil: there is an umbrella jira (HIVE-13503
> <https://issues.apache.org/jira/browse/HIVE-13503>) for test improvements,
> the Yetus integration itself is also a subtask of it. I think any further
> improvements on what Yetus features we want to enable should go here too.
>
> Adam
>
>
>
> On 6 November 2017 at 22:02, Sahil Takiar <ta...@gmail.com> wrote:
>
> > +1 - think this will be a great addition to Hive. Helping us catch issues
> > earlier, keeping the Hive code cleaner, etc. Getting the basic Yetus
> checks
> > to work seems like a great start, do we have follow JIRAs to get more
> YETUS
> > tests integrated - e.g. FindBugs?
> >
> > On Mon, Nov 6, 2017 at 10:29 AM, Vihang Karajgaonkar <
> vihang@cloudera.com>
> > wrote:
> >
> > > Thanks Adam for this work. This is definitely useful and a good
> addition
> > to
> > > our test infrastructure.
> > >
> > > Can we fix the existing issues pointed by Yetus (in a separate JIRA) so
> > > that we have a +1 from yetus on the current code? Once that is done,
> > > committers can help keep it green as they review patches and merge it.
> > >
> > > Thanks,
> > > Vihang
> > >
> > > On Mon, Nov 6, 2017 at 9:04 AM, Thejas Nair <th...@gmail.com>
> > wrote:
> > >
> > > > +1
> > > > Yes, I think this can help us catch many issues early on, it will be
> > very
> > > > useful!
> > > >
> > > >
> > > > On Mon, Nov 6, 2017 at 7:43 AM, Adam Szita <sz...@cloudera.com>
> wrote:
> > > >
> > > > > Hi all,
> > > > >
> > > > > As a next step of test subsystem improvements we would like to have
> > the
> > > > > Yetus check integrated with the ptest framework. This means that
> > > > whenever a
> > > > > new patch is uploaded - along with the already existing Precommit
> > test
> > > > run
> > > > > - Hive's Yetus patch check script would be triggered also. This
> > script
> > > > runs
> > > > > checkstyle, findbugs, ASF license check, etc with and without the
> > > > submitted
> > > > > patch applied and reports the diffs (i.e. how many checkstyle
> > problems
> > > > does
> > > > > the patch introduce).
> > > > >
> > > > > It would be executed parallel to the ptest test execution and
> report
> > > back
> > > > > the results as a (another) jira comment to the issue in question.
> > > > > In the last days I've been working on this (HIVE-16748) and a patch
> > is
> > > > > ready to make this happen. A sample Yetus result comment is
> available
> > > at
> > > > > https://issues.apache.org/jira/browse/HIVE-16748?
> > > > > focusedCommentId=16218616&page=com.atlassian.jira.
> > > > > plugin.system.issuetabpanels:comment-tabpanel#comment-16218616
> > > > >
> > > > > We think this would be a useful tool for us developers and would
> like
> > > to
> > > > go
> > > > > ahead with this change, but we're also curious about your input in
> > this
> > > > > matter. Please let us know what you think about this change.
> > > > >
> > > > > Thanks,
> > > > > Adam
> > > > >
> > > >
> > >
> >
> >
> >
> > --
> > Sahil Takiar
> > Software Engineer at Cloudera
> > takiar.sahil@gmail.com | (510) 673-0309
> >
>
Re: Integrating Yetus with Precommit job
Posted by Vihang Karajgaonkar <vi...@cloudera.com>.
Thanks Adam for this work!
On Wed, Nov 22, 2017 at 5:46 AM, Adam Szita <sz...@cloudera.com> wrote:
> This is now done. Patch is committed and we deployed the new war file to
> the ptest server.
>
> Jobs that were waiting in queue at the time of ptest server restart have
> been retriggered in Jenkins.
>
> I hope this change will contribute to the overall code quality of Hive in
> our future patches to come :)
>
> On 21 November 2017 at 17:39, Adam Szita <sz...@cloudera.com> wrote:
>
> > Hi,
> >
> > In the last days all prerequisites have been resolved for this:
> > -ASF headers are fixed
> > -checkstyle is upgraded to support Java8
> > -proper checkstyle configuration has been introduced to poms that are
> > disconnected from Hive's root pom
> >
> > Thanks Alan for reviewing these.
> >
> > Therefore we plan to move ahead with this tomorrow around 10:00AM CET, do
> > the commit with Peter Vary and replace the war file among ptest servers
> > Tomcat webapps.
> >
> > Thanks,
> > Adam
> >
> > On 7 November 2017 at 18:42, Alan Gates <al...@gmail.com> wrote:
> >
> >> I’ve put some feedback in HIVE-17995. 17996 and 17997 look good. I’ll
> >> commit them once the tests run.
> >>
> >> I think you’ll need to do similar patches for storage-api, as it is also
> >> not connected to the hive pom anymore.
> >>
> >> Alan.
> >>
> >> On Tue, Nov 7, 2017 at 6:17 AM, Adam Szita <sz...@cloudera.com> wrote:
> >>
> >> > Thanks for all the replies.
> >> >
> >> > Vihang: Good idea on making everything green before turning this on.
> For
> >> > this purpose I've filed a couple of jiras:
> >> > -HIVE-17995 <https://issues.apache.org/jira/browse/HIVE-17995> Run
> >> > checkstyle on standalone-metastore module with proper configuration
> >> > -HIVE-17996 <https://issues.apache.org/jira/browse/HIVE-17996> Fix
> ASF
> >> > headers
> >> > -HIVE-17997 <https://issues.apache.org/jira/browse/HIVE-17997> Add
> rat
> >> > plugin and configuration to standalone metastore pom
> >> >
> >> > Sahil: there is an umbrella jira (HIVE-13503
> >> > <https://issues.apache.org/jira/browse/HIVE-13503>) for test
> >> improvements,
> >> > the Yetus integration itself is also a subtask of it. I think any
> >> further
> >> > improvements on what Yetus features we want to enable should go here
> >> too.
> >> >
> >> > Adam
> >> >
> >> >
> >> >
> >> >
> >> >
> >>
> >
> >
>
Re: Integrating Yetus with Precommit job
Posted by Sahil Takiar <ta...@gmail.com>.
The FindBugs plugin for Yetus is now working. Yetus will give a -1 if it
finds any FindBugs warning in your patch. It gives a 0 for any patch
applied to a module that contains existing FindBugs warnings (e.g. ql has
2318 existing FindBugs issues).
On Mon, Nov 27, 2017 at 8:57 AM, Andrew Sherman <as...@cloudera.com>
wrote:
> Thanks, this is going to be useful
>
> On Wed, Nov 22, 2017 at 11:28 AM, Vineet Garg <vg...@hortonworks.com>
> wrote:
>
> > Thanks Adam!
> >
> > > On Nov 22, 2017, at 5:46 AM, Adam Szita <sz...@cloudera.com> wrote:
> > >
> > > This is now done. Patch is committed and we deployed the new war file
> to
> > > the ptest server.
> > >
> > > Jobs that were waiting in queue at the time of ptest server restart
> have
> > > been retriggered in Jenkins.
> > >
> > > I hope this change will contribute to the overall code quality of Hive
> in
> > > our future patches to come :)
> > >
> > > On 21 November 2017 at 17:39, Adam Szita <sz...@cloudera.com> wrote:
> > >
> > >> Hi,
> > >>
> > >> In the last days all prerequisites have been resolved for this:
> > >> -ASF headers are fixed
> > >> -checkstyle is upgraded to support Java8
> > >> -proper checkstyle configuration has been introduced to poms that are
> > >> disconnected from Hive's root pom
> > >>
> > >> Thanks Alan for reviewing these.
> > >>
> > >> Therefore we plan to move ahead with this tomorrow around 10:00AM CET,
> > do
> > >> the commit with Peter Vary and replace the war file among ptest
> servers
> > >> Tomcat webapps.
> > >>
> > >> Thanks,
> > >> Adam
> > >>
> > >> On 7 November 2017 at 18:42, Alan Gates <al...@gmail.com> wrote:
> > >>
> > >>> I’ve put some feedback in HIVE-17995. 17996 and 17997 look good.
> I’ll
> > >>> commit them once the tests run.
> > >>>
> > >>> I think you’ll need to do similar patches for storage-api, as it is
> > also
> > >>> not connected to the hive pom anymore.
> > >>>
> > >>> Alan.
> > >>>
> > >>> On Tue, Nov 7, 2017 at 6:17 AM, Adam Szita <sz...@cloudera.com>
> wrote:
> > >>>
> > >>>> Thanks for all the replies.
> > >>>>
> > >>>> Vihang: Good idea on making everything green before turning this on.
> > For
> > >>>> this purpose I've filed a couple of jiras:
> > >>>> -HIVE-17995 <https://issues.apache.org/jira/browse/HIVE-17995> Run
> > >>>> checkstyle on standalone-metastore module with proper configuration
> > >>>> -HIVE-17996 <https://issues.apache.org/jira/browse/HIVE-17996> Fix
> > ASF
> > >>>> headers
> > >>>> -HIVE-17997 <https://issues.apache.org/jira/browse/HIVE-17997> Add
> > rat
> > >>>> plugin and configuration to standalone metastore pom
> > >>>>
> > >>>> Sahil: there is an umbrella jira (HIVE-13503
> > >>>> <https://issues.apache.org/jira/browse/HIVE-13503>) for test
> > >>> improvements,
> > >>>> the Yetus integration itself is also a subtask of it. I think any
> > >>> further
> > >>>> improvements on what Yetus features we want to enable should go here
> > >>> too.
> > >>>>
> > >>>> Adam
> > >>>>
> > >>>>
> > >>>>
> > >>>>
> > >>>>
> > >>>
> > >>
> > >>
> >
> >
>
--
Sahil Takiar
Software Engineer
takiar.sahil@gmail.com | (510) 673-0309
Re: Integrating Yetus with Precommit job
Posted by Andrew Sherman <as...@cloudera.com>.
Thanks, this is going to be useful
On Wed, Nov 22, 2017 at 11:28 AM, Vineet Garg <vg...@hortonworks.com> wrote:
> Thanks Adam!
>
> > On Nov 22, 2017, at 5:46 AM, Adam Szita <sz...@cloudera.com> wrote:
> >
> > This is now done. Patch is committed and we deployed the new war file to
> > the ptest server.
> >
> > Jobs that were waiting in queue at the time of ptest server restart have
> > been retriggered in Jenkins.
> >
> > I hope this change will contribute to the overall code quality of Hive in
> > our future patches to come :)
> >
> > On 21 November 2017 at 17:39, Adam Szita <sz...@cloudera.com> wrote:
> >
> >> Hi,
> >>
> >> In the last days all prerequisites have been resolved for this:
> >> -ASF headers are fixed
> >> -checkstyle is upgraded to support Java8
> >> -proper checkstyle configuration has been introduced to poms that are
> >> disconnected from Hive's root pom
> >>
> >> Thanks Alan for reviewing these.
> >>
> >> Therefore we plan to move ahead with this tomorrow around 10:00AM CET,
> do
> >> the commit with Peter Vary and replace the war file among ptest servers
> >> Tomcat webapps.
> >>
> >> Thanks,
> >> Adam
> >>
> >> On 7 November 2017 at 18:42, Alan Gates <al...@gmail.com> wrote:
> >>
> >>> I’ve put some feedback in HIVE-17995. 17996 and 17997 look good. I’ll
> >>> commit them once the tests run.
> >>>
> >>> I think you’ll need to do similar patches for storage-api, as it is
> also
> >>> not connected to the hive pom anymore.
> >>>
> >>> Alan.
> >>>
> >>> On Tue, Nov 7, 2017 at 6:17 AM, Adam Szita <sz...@cloudera.com> wrote:
> >>>
> >>>> Thanks for all the replies.
> >>>>
> >>>> Vihang: Good idea on making everything green before turning this on.
> For
> >>>> this purpose I've filed a couple of jiras:
> >>>> -HIVE-17995 <https://issues.apache.org/jira/browse/HIVE-17995> Run
> >>>> checkstyle on standalone-metastore module with proper configuration
> >>>> -HIVE-17996 <https://issues.apache.org/jira/browse/HIVE-17996> Fix
> ASF
> >>>> headers
> >>>> -HIVE-17997 <https://issues.apache.org/jira/browse/HIVE-17997> Add
> rat
> >>>> plugin and configuration to standalone metastore pom
> >>>>
> >>>> Sahil: there is an umbrella jira (HIVE-13503
> >>>> <https://issues.apache.org/jira/browse/HIVE-13503>) for test
> >>> improvements,
> >>>> the Yetus integration itself is also a subtask of it. I think any
> >>> further
> >>>> improvements on what Yetus features we want to enable should go here
> >>> too.
> >>>>
> >>>> Adam
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>
> >>
> >>
>
>
Re: Integrating Yetus with Precommit job
Posted by Vineet Garg <vg...@hortonworks.com>.
Thanks Adam!
> On Nov 22, 2017, at 5:46 AM, Adam Szita <sz...@cloudera.com> wrote:
>
> This is now done. Patch is committed and we deployed the new war file to
> the ptest server.
>
> Jobs that were waiting in queue at the time of ptest server restart have
> been retriggered in Jenkins.
>
> I hope this change will contribute to the overall code quality of Hive in
> our future patches to come :)
>
> On 21 November 2017 at 17:39, Adam Szita <sz...@cloudera.com> wrote:
>
>> Hi,
>>
>> In the last days all prerequisites have been resolved for this:
>> -ASF headers are fixed
>> -checkstyle is upgraded to support Java8
>> -proper checkstyle configuration has been introduced to poms that are
>> disconnected from Hive's root pom
>>
>> Thanks Alan for reviewing these.
>>
>> Therefore we plan to move ahead with this tomorrow around 10:00AM CET, do
>> the commit with Peter Vary and replace the war file among ptest servers
>> Tomcat webapps.
>>
>> Thanks,
>> Adam
>>
>> On 7 November 2017 at 18:42, Alan Gates <al...@gmail.com> wrote:
>>
>>> I’ve put some feedback in HIVE-17995. 17996 and 17997 look good. I’ll
>>> commit them once the tests run.
>>>
>>> I think you’ll need to do similar patches for storage-api, as it is also
>>> not connected to the hive pom anymore.
>>>
>>> Alan.
>>>
>>> On Tue, Nov 7, 2017 at 6:17 AM, Adam Szita <sz...@cloudera.com> wrote:
>>>
>>>> Thanks for all the replies.
>>>>
>>>> Vihang: Good idea on making everything green before turning this on. For
>>>> this purpose I've filed a couple of jiras:
>>>> -HIVE-17995 <https://issues.apache.org/jira/browse/HIVE-17995> Run
>>>> checkstyle on standalone-metastore module with proper configuration
>>>> -HIVE-17996 <https://issues.apache.org/jira/browse/HIVE-17996> Fix ASF
>>>> headers
>>>> -HIVE-17997 <https://issues.apache.org/jira/browse/HIVE-17997> Add rat
>>>> plugin and configuration to standalone metastore pom
>>>>
>>>> Sahil: there is an umbrella jira (HIVE-13503
>>>> <https://issues.apache.org/jira/browse/HIVE-13503>) for test
>>> improvements,
>>>> the Yetus integration itself is also a subtask of it. I think any
>>> further
>>>> improvements on what Yetus features we want to enable should go here
>>> too.
>>>>
>>>> Adam
>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>
>>
Re: Integrating Yetus with Precommit job
Posted by Adam Szita <sz...@cloudera.com>.
This is now done. Patch is committed and we deployed the new war file to
the ptest server.
Jobs that were waiting in queue at the time of ptest server restart have
been retriggered in Jenkins.
I hope this change will contribute to the overall code quality of Hive in
our future patches to come :)
On 21 November 2017 at 17:39, Adam Szita <sz...@cloudera.com> wrote:
> Hi,
>
> In the last days all prerequisites have been resolved for this:
> -ASF headers are fixed
> -checkstyle is upgraded to support Java8
> -proper checkstyle configuration has been introduced to poms that are
> disconnected from Hive's root pom
>
> Thanks Alan for reviewing these.
>
> Therefore we plan to move ahead with this tomorrow around 10:00AM CET, do
> the commit with Peter Vary and replace the war file among ptest servers
> Tomcat webapps.
>
> Thanks,
> Adam
>
> On 7 November 2017 at 18:42, Alan Gates <al...@gmail.com> wrote:
>
>> I’ve put some feedback in HIVE-17995. 17996 and 17997 look good. I’ll
>> commit them once the tests run.
>>
>> I think you’ll need to do similar patches for storage-api, as it is also
>> not connected to the hive pom anymore.
>>
>> Alan.
>>
>> On Tue, Nov 7, 2017 at 6:17 AM, Adam Szita <sz...@cloudera.com> wrote:
>>
>> > Thanks for all the replies.
>> >
>> > Vihang: Good idea on making everything green before turning this on. For
>> > this purpose I've filed a couple of jiras:
>> > -HIVE-17995 <https://issues.apache.org/jira/browse/HIVE-17995> Run
>> > checkstyle on standalone-metastore module with proper configuration
>> > -HIVE-17996 <https://issues.apache.org/jira/browse/HIVE-17996> Fix ASF
>> > headers
>> > -HIVE-17997 <https://issues.apache.org/jira/browse/HIVE-17997> Add rat
>> > plugin and configuration to standalone metastore pom
>> >
>> > Sahil: there is an umbrella jira (HIVE-13503
>> > <https://issues.apache.org/jira/browse/HIVE-13503>) for test
>> improvements,
>> > the Yetus integration itself is also a subtask of it. I think any
>> further
>> > improvements on what Yetus features we want to enable should go here
>> too.
>> >
>> > Adam
>> >
>> >
>> >
>> >
>> >
>>
>
>
Re: Integrating Yetus with Precommit job
Posted by Adam Szita <sz...@cloudera.com>.
Hi,
In the last days all prerequisites have been resolved for this:
-ASF headers are fixed
-checkstyle is upgraded to support Java8
-proper checkstyle configuration has been introduced to poms that are
disconnected from Hive's root pom
Thanks Alan for reviewing these.
Therefore we plan to move ahead with this tomorrow around 10:00AM CET, do
the commit with Peter Vary and replace the war file among ptest servers
Tomcat webapps.
Thanks,
Adam
On 7 November 2017 at 18:42, Alan Gates <al...@gmail.com> wrote:
> I’ve put some feedback in HIVE-17995. 17996 and 17997 look good. I’ll
> commit them once the tests run.
>
> I think you’ll need to do similar patches for storage-api, as it is also
> not connected to the hive pom anymore.
>
> Alan.
>
> On Tue, Nov 7, 2017 at 6:17 AM, Adam Szita <sz...@cloudera.com> wrote:
>
> > Thanks for all the replies.
> >
> > Vihang: Good idea on making everything green before turning this on. For
> > this purpose I've filed a couple of jiras:
> > -HIVE-17995 <https://issues.apache.org/jira/browse/HIVE-17995> Run
> > checkstyle on standalone-metastore module with proper configuration
> > -HIVE-17996 <https://issues.apache.org/jira/browse/HIVE-17996> Fix ASF
> > headers
> > -HIVE-17997 <https://issues.apache.org/jira/browse/HIVE-17997> Add rat
> > plugin and configuration to standalone metastore pom
> >
> > Sahil: there is an umbrella jira (HIVE-13503
> > <https://issues.apache.org/jira/browse/HIVE-13503>) for test
> improvements,
> > the Yetus integration itself is also a subtask of it. I think any further
> > improvements on what Yetus features we want to enable should go here too.
> >
> > Adam
> >
> >
> >
> >
> >
>
Re: Integrating Yetus with Precommit job
Posted by Alan Gates <al...@gmail.com>.
I’ve put some feedback in HIVE-17995. 17996 and 17997 look good. I’ll
commit them once the tests run.
I think you’ll need to do similar patches for storage-api, as it is also
not connected to the hive pom anymore.
Alan.
On Tue, Nov 7, 2017 at 6:17 AM, Adam Szita <sz...@cloudera.com> wrote:
> Thanks for all the replies.
>
> Vihang: Good idea on making everything green before turning this on. For
> this purpose I've filed a couple of jiras:
> -HIVE-17995 <https://issues.apache.org/jira/browse/HIVE-17995> Run
> checkstyle on standalone-metastore module with proper configuration
> -HIVE-17996 <https://issues.apache.org/jira/browse/HIVE-17996> Fix ASF
> headers
> -HIVE-17997 <https://issues.apache.org/jira/browse/HIVE-17997> Add rat
> plugin and configuration to standalone metastore pom
>
> Sahil: there is an umbrella jira (HIVE-13503
> <https://issues.apache.org/jira/browse/HIVE-13503>) for test improvements,
> the Yetus integration itself is also a subtask of it. I think any further
> improvements on what Yetus features we want to enable should go here too.
>
> Adam
>
>
>
>
>
Re: Integrating Yetus with Precommit job
Posted by Adam Szita <sz...@cloudera.com>.
Thanks for all the replies.
Vihang: Good idea on making everything green before turning this on. For
this purpose I've filed a couple of jiras:
-HIVE-17995 <https://issues.apache.org/jira/browse/HIVE-17995> Run
checkstyle on standalone-metastore module with proper configuration
-HIVE-17996 <https://issues.apache.org/jira/browse/HIVE-17996> Fix ASF
headers
-HIVE-17997 <https://issues.apache.org/jira/browse/HIVE-17997> Add rat
plugin and configuration to standalone metastore pom
Sahil: there is an umbrella jira (HIVE-13503
<https://issues.apache.org/jira/browse/HIVE-13503>) for test improvements,
the Yetus integration itself is also a subtask of it. I think any further
improvements on what Yetus features we want to enable should go here too.
Adam
On 6 November 2017 at 22:02, Sahil Takiar <ta...@gmail.com> wrote:
> +1 - think this will be a great addition to Hive. Helping us catch issues
> earlier, keeping the Hive code cleaner, etc. Getting the basic Yetus checks
> to work seems like a great start, do we have follow JIRAs to get more YETUS
> tests integrated - e.g. FindBugs?
>
> On Mon, Nov 6, 2017 at 10:29 AM, Vihang Karajgaonkar <vi...@cloudera.com>
> wrote:
>
> > Thanks Adam for this work. This is definitely useful and a good addition
> to
> > our test infrastructure.
> >
> > Can we fix the existing issues pointed by Yetus (in a separate JIRA) so
> > that we have a +1 from yetus on the current code? Once that is done,
> > committers can help keep it green as they review patches and merge it.
> >
> > Thanks,
> > Vihang
> >
> > On Mon, Nov 6, 2017 at 9:04 AM, Thejas Nair <th...@gmail.com>
> wrote:
> >
> > > +1
> > > Yes, I think this can help us catch many issues early on, it will be
> very
> > > useful!
> > >
> > >
> > > On Mon, Nov 6, 2017 at 7:43 AM, Adam Szita <sz...@cloudera.com> wrote:
> > >
> > > > Hi all,
> > > >
> > > > As a next step of test subsystem improvements we would like to have
> the
> > > > Yetus check integrated with the ptest framework. This means that
> > > whenever a
> > > > new patch is uploaded - along with the already existing Precommit
> test
> > > run
> > > > - Hive's Yetus patch check script would be triggered also. This
> script
> > > runs
> > > > checkstyle, findbugs, ASF license check, etc with and without the
> > > submitted
> > > > patch applied and reports the diffs (i.e. how many checkstyle
> problems
> > > does
> > > > the patch introduce).
> > > >
> > > > It would be executed parallel to the ptest test execution and report
> > back
> > > > the results as a (another) jira comment to the issue in question.
> > > > In the last days I've been working on this (HIVE-16748) and a patch
> is
> > > > ready to make this happen. A sample Yetus result comment is available
> > at
> > > > https://issues.apache.org/jira/browse/HIVE-16748?
> > > > focusedCommentId=16218616&page=com.atlassian.jira.
> > > > plugin.system.issuetabpanels:comment-tabpanel#comment-16218616
> > > >
> > > > We think this would be a useful tool for us developers and would like
> > to
> > > go
> > > > ahead with this change, but we're also curious about your input in
> this
> > > > matter. Please let us know what you think about this change.
> > > >
> > > > Thanks,
> > > > Adam
> > > >
> > >
> >
>
>
>
> --
> Sahil Takiar
> Software Engineer at Cloudera
> takiar.sahil@gmail.com | (510) 673-0309
>
Re: Integrating Yetus with Precommit job
Posted by Sahil Takiar <ta...@gmail.com>.
+1 - think this will be a great addition to Hive. Helping us catch issues
earlier, keeping the Hive code cleaner, etc. Getting the basic Yetus checks
to work seems like a great start, do we have follow JIRAs to get more YETUS
tests integrated - e.g. FindBugs?
On Mon, Nov 6, 2017 at 10:29 AM, Vihang Karajgaonkar <vi...@cloudera.com>
wrote:
> Thanks Adam for this work. This is definitely useful and a good addition to
> our test infrastructure.
>
> Can we fix the existing issues pointed by Yetus (in a separate JIRA) so
> that we have a +1 from yetus on the current code? Once that is done,
> committers can help keep it green as they review patches and merge it.
>
> Thanks,
> Vihang
>
> On Mon, Nov 6, 2017 at 9:04 AM, Thejas Nair <th...@gmail.com> wrote:
>
> > +1
> > Yes, I think this can help us catch many issues early on, it will be very
> > useful!
> >
> >
> > On Mon, Nov 6, 2017 at 7:43 AM, Adam Szita <sz...@cloudera.com> wrote:
> >
> > > Hi all,
> > >
> > > As a next step of test subsystem improvements we would like to have the
> > > Yetus check integrated with the ptest framework. This means that
> > whenever a
> > > new patch is uploaded - along with the already existing Precommit test
> > run
> > > - Hive's Yetus patch check script would be triggered also. This script
> > runs
> > > checkstyle, findbugs, ASF license check, etc with and without the
> > submitted
> > > patch applied and reports the diffs (i.e. how many checkstyle problems
> > does
> > > the patch introduce).
> > >
> > > It would be executed parallel to the ptest test execution and report
> back
> > > the results as a (another) jira comment to the issue in question.
> > > In the last days I've been working on this (HIVE-16748) and a patch is
> > > ready to make this happen. A sample Yetus result comment is available
> at
> > > https://issues.apache.org/jira/browse/HIVE-16748?
> > > focusedCommentId=16218616&page=com.atlassian.jira.
> > > plugin.system.issuetabpanels:comment-tabpanel#comment-16218616
> > >
> > > We think this would be a useful tool for us developers and would like
> to
> > go
> > > ahead with this change, but we're also curious about your input in this
> > > matter. Please let us know what you think about this change.
> > >
> > > Thanks,
> > > Adam
> > >
> >
>
--
Sahil Takiar
Software Engineer at Cloudera
takiar.sahil@gmail.com | (510) 673-0309
Re: Integrating Yetus with Precommit job
Posted by Vihang Karajgaonkar <vi...@cloudera.com>.
Thanks Adam for this work. This is definitely useful and a good addition to
our test infrastructure.
Can we fix the existing issues pointed by Yetus (in a separate JIRA) so
that we have a +1 from yetus on the current code? Once that is done,
committers can help keep it green as they review patches and merge it.
Thanks,
Vihang
On Mon, Nov 6, 2017 at 9:04 AM, Thejas Nair <th...@gmail.com> wrote:
> +1
> Yes, I think this can help us catch many issues early on, it will be very
> useful!
>
>
> On Mon, Nov 6, 2017 at 7:43 AM, Adam Szita <sz...@cloudera.com> wrote:
>
> > Hi all,
> >
> > As a next step of test subsystem improvements we would like to have the
> > Yetus check integrated with the ptest framework. This means that
> whenever a
> > new patch is uploaded - along with the already existing Precommit test
> run
> > - Hive's Yetus patch check script would be triggered also. This script
> runs
> > checkstyle, findbugs, ASF license check, etc with and without the
> submitted
> > patch applied and reports the diffs (i.e. how many checkstyle problems
> does
> > the patch introduce).
> >
> > It would be executed parallel to the ptest test execution and report back
> > the results as a (another) jira comment to the issue in question.
> > In the last days I've been working on this (HIVE-16748) and a patch is
> > ready to make this happen. A sample Yetus result comment is available at
> > https://issues.apache.org/jira/browse/HIVE-16748?
> > focusedCommentId=16218616&page=com.atlassian.jira.
> > plugin.system.issuetabpanels:comment-tabpanel#comment-16218616
> >
> > We think this would be a useful tool for us developers and would like to
> go
> > ahead with this change, but we're also curious about your input in this
> > matter. Please let us know what you think about this change.
> >
> > Thanks,
> > Adam
> >
>
Re: Integrating Yetus with Precommit job
Posted by Thejas Nair <th...@gmail.com>.
+1
Yes, I think this can help us catch many issues early on, it will be very
useful!
On Mon, Nov 6, 2017 at 7:43 AM, Adam Szita <sz...@cloudera.com> wrote:
> Hi all,
>
> As a next step of test subsystem improvements we would like to have the
> Yetus check integrated with the ptest framework. This means that whenever a
> new patch is uploaded - along with the already existing Precommit test run
> - Hive's Yetus patch check script would be triggered also. This script runs
> checkstyle, findbugs, ASF license check, etc with and without the submitted
> patch applied and reports the diffs (i.e. how many checkstyle problems does
> the patch introduce).
>
> It would be executed parallel to the ptest test execution and report back
> the results as a (another) jira comment to the issue in question.
> In the last days I've been working on this (HIVE-16748) and a patch is
> ready to make this happen. A sample Yetus result comment is available at
> https://issues.apache.org/jira/browse/HIVE-16748?
> focusedCommentId=16218616&page=com.atlassian.jira.
> plugin.system.issuetabpanels:comment-tabpanel#comment-16218616
>
> We think this would be a useful tool for us developers and would like to go
> ahead with this change, but we're also curious about your input in this
> matter. Please let us know what you think about this change.
>
> Thanks,
> Adam
>