You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@accumulo.apache.org by Josh Elser <jo...@gmail.com> on 2016/01/05 07:00:28 UTC
Re: Yetus Accumulo "Personality"
FYI https://issues.apache.org/jira/browse/YETUS-263 was merged in last week.
Eric had also sent me a reply off-list which asked if it would be
possible to do a `mvn verify -Psunny` to run the small set of ITs we
have defined in the pom (Examples, ReadWrite, and ShellServer ITs, IIRC).
Ignoring the issue of whether or not we could even run those tests on
ASF infra, I would guess that we could write a plugin that runs Maven
integration tests (like they run unit tests) and approach the
functionality that way. We could expose some control which determine
whether or not these integration tests were invoked.
I also commented with some output on Matt's patch from ACCUMULO-2493 --
I found it rather pleasant to run a single command and get a nice
summary of his changes.
Josh Elser wrote:
> For those interested in following along with the PreCommit work, see
> https://issues.apache.org/jira/browse/YETUS-263
>
> A "personality", in Yetus parlance, defines the the tests/checks that
> PreCommit will run against Accumulo. For us, it's pretty simple. The
> personality I provided on YETUS-263 will, for a patch/changeset run:
>
> * Checkstyle
> * Findbugs
> * RAT check
> * @author javadoc check
> * Some extra whitespace
> * All unit tests (not just in the module where changes were made)
> * Compiler warnings
> * Javadoc warnings
> * Presence of new unit tests
>
> One already built in feature that I didn't wire up is ShellCheck for our
> shell scripts. This will require a bit of fixing on our end first.
>
> For more general information, Chris Nauroth wrote up a good explanation
> for adopting the same approach in ZooKeeper (and did a much better job
> than me
> http://mail-archives.apache.org/mod_mbox/zookeeper-dev/201512.mbox/%3CD291EB3B.3504A%25cnauroth%40hortonworks.com%3E).
>
>
> Anywho, nothing really changing here yet (I'm hoping Sean will write up
> instructions about how to configure the Jenkins job for us
> https://issues.apache.org/jira/browse/YETUS-245). That would signify a
> step for Accumulo specifically. Until then, this is just an FYI.
>
> - Josh
Re: Yetus Accumulo "Personality"
Posted by Josh Elser <jo...@gmail.com>.
Welp:
https://issues.apache.org/jira/browse/ACCUMULO-4095?focusedCommentId=15088814&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15088814
Josh Elser wrote:
> That's a good question (about a point which I haven't outlined as a
> possibility).
>
> I had been poking around some of the PreCommit jobs on ASF Jenkins which
> were Yetus-ified and noticed that some of them already had some
> parameterization to support 1) using a custom Yetus install or 2) using
> some different personality not in Yetus.
>
> Obviously, I would like to make sure that we keep Yetus up to date with
> any changes/improvements we make for our own purposes here in Accumulo,
> but don't see it as a requirement that we have to stall ourselves waiting.
>
> Although, I have mostly been talking to myself in this thread, so
> perhaps the iteration speed is my nights+weekends availability :)
>
> Sean Busbey wrote:
>> Excellent, thanks for working on this Josh!
>>
>> Is it worth us running our personality out of our code base rather than
>> Yetus'? I'm not sure if we need the faster iteration speed or not.
>>
>> -Sean
>>
>> On Tue, Jan 5, 2016 at 12:00 AM, Josh Elser<jo...@gmail.com> wrote:
>>
>>> FYI https://issues.apache.org/jira/browse/YETUS-263 was merged in last
>>> week.
>>>
>>> Eric had also sent me a reply off-list which asked if it would be
>>> possible
>>> to do a `mvn verify -Psunny` to run the small set of ITs we have
>>> defined in
>>> the pom (Examples, ReadWrite, and ShellServer ITs, IIRC).
>>>
>>> Ignoring the issue of whether or not we could even run those tests on
>>> ASF
>>> infra, I would guess that we could write a plugin that runs Maven
>>> integration tests (like they run unit tests) and approach the
>>> functionality
>>> that way. We could expose some control which determine whether or not
>>> these
>>> integration tests were invoked.
>>>
>>> I also commented with some output on Matt's patch from ACCUMULO-2493
>>> -- I
>>> found it rather pleasant to run a single command and get a nice
>>> summary of
>>> his changes.
>>>
>>>
>>> Josh Elser wrote:
>>>
>>>> For those interested in following along with the PreCommit work, see
>>>> https://issues.apache.org/jira/browse/YETUS-263
>>>>
>>>> A "personality", in Yetus parlance, defines the the tests/checks that
>>>> PreCommit will run against Accumulo. For us, it's pretty simple. The
>>>> personality I provided on YETUS-263 will, for a patch/changeset run:
>>>>
>>>> * Checkstyle
>>>> * Findbugs
>>>> * RAT check
>>>> * @author javadoc check
>>>> * Some extra whitespace
>>>> * All unit tests (not just in the module where changes were made)
>>>> * Compiler warnings
>>>> * Javadoc warnings
>>>> * Presence of new unit tests
>>>>
>>>> One already built in feature that I didn't wire up is ShellCheck for
>>>> our
>>>> shell scripts. This will require a bit of fixing on our end first.
>>>>
>>>> For more general information, Chris Nauroth wrote up a good explanation
>>>> for adopting the same approach in ZooKeeper (and did a much better job
>>>> than me
>>>>
>>>> http://mail-archives.apache.org/mod_mbox/zookeeper-dev/201512.mbox/%3CD291EB3B.3504A%25cnauroth%40hortonworks.com%3E
>>>>
>>>> ).
>>>>
>>>>
>>>> Anywho, nothing really changing here yet (I'm hoping Sean will write up
>>>> instructions about how to configure the Jenkins job for us
>>>> https://issues.apache.org/jira/browse/YETUS-245). That would signify a
>>>> step for Accumulo specifically. Until then, this is just an FYI.
>>>>
>>>> - Josh
>>>>
>>
>>
Re: Yetus Accumulo "Personality"
Posted by Josh Elser <jo...@gmail.com>.
That's a good question (about a point which I haven't outlined as a
possibility).
I had been poking around some of the PreCommit jobs on ASF Jenkins which
were Yetus-ified and noticed that some of them already had some
parameterization to support 1) using a custom Yetus install or 2) using
some different personality not in Yetus.
Obviously, I would like to make sure that we keep Yetus up to date with
any changes/improvements we make for our own purposes here in Accumulo,
but don't see it as a requirement that we have to stall ourselves waiting.
Although, I have mostly been talking to myself in this thread, so
perhaps the iteration speed is my nights+weekends availability :)
Sean Busbey wrote:
> Excellent, thanks for working on this Josh!
>
> Is it worth us running our personality out of our code base rather than
> Yetus'? I'm not sure if we need the faster iteration speed or not.
>
> -Sean
>
> On Tue, Jan 5, 2016 at 12:00 AM, Josh Elser<jo...@gmail.com> wrote:
>
>> FYI https://issues.apache.org/jira/browse/YETUS-263 was merged in last
>> week.
>>
>> Eric had also sent me a reply off-list which asked if it would be possible
>> to do a `mvn verify -Psunny` to run the small set of ITs we have defined in
>> the pom (Examples, ReadWrite, and ShellServer ITs, IIRC).
>>
>> Ignoring the issue of whether or not we could even run those tests on ASF
>> infra, I would guess that we could write a plugin that runs Maven
>> integration tests (like they run unit tests) and approach the functionality
>> that way. We could expose some control which determine whether or not these
>> integration tests were invoked.
>>
>> I also commented with some output on Matt's patch from ACCUMULO-2493 -- I
>> found it rather pleasant to run a single command and get a nice summary of
>> his changes.
>>
>>
>> Josh Elser wrote:
>>
>>> For those interested in following along with the PreCommit work, see
>>> https://issues.apache.org/jira/browse/YETUS-263
>>>
>>> A "personality", in Yetus parlance, defines the the tests/checks that
>>> PreCommit will run against Accumulo. For us, it's pretty simple. The
>>> personality I provided on YETUS-263 will, for a patch/changeset run:
>>>
>>> * Checkstyle
>>> * Findbugs
>>> * RAT check
>>> * @author javadoc check
>>> * Some extra whitespace
>>> * All unit tests (not just in the module where changes were made)
>>> * Compiler warnings
>>> * Javadoc warnings
>>> * Presence of new unit tests
>>>
>>> One already built in feature that I didn't wire up is ShellCheck for our
>>> shell scripts. This will require a bit of fixing on our end first.
>>>
>>> For more general information, Chris Nauroth wrote up a good explanation
>>> for adopting the same approach in ZooKeeper (and did a much better job
>>> than me
>>>
>>> http://mail-archives.apache.org/mod_mbox/zookeeper-dev/201512.mbox/%3CD291EB3B.3504A%25cnauroth%40hortonworks.com%3E
>>> ).
>>>
>>>
>>> Anywho, nothing really changing here yet (I'm hoping Sean will write up
>>> instructions about how to configure the Jenkins job for us
>>> https://issues.apache.org/jira/browse/YETUS-245). That would signify a
>>> step for Accumulo specifically. Until then, this is just an FYI.
>>>
>>> - Josh
>>>
>
>
Re: Yetus Accumulo "Personality"
Posted by Sean Busbey <bu...@cloudera.com>.
Excellent, thanks for working on this Josh!
Is it worth us running our personality out of our code base rather than
Yetus'? I'm not sure if we need the faster iteration speed or not.
-Sean
On Tue, Jan 5, 2016 at 12:00 AM, Josh Elser <jo...@gmail.com> wrote:
> FYI https://issues.apache.org/jira/browse/YETUS-263 was merged in last
> week.
>
> Eric had also sent me a reply off-list which asked if it would be possible
> to do a `mvn verify -Psunny` to run the small set of ITs we have defined in
> the pom (Examples, ReadWrite, and ShellServer ITs, IIRC).
>
> Ignoring the issue of whether or not we could even run those tests on ASF
> infra, I would guess that we could write a plugin that runs Maven
> integration tests (like they run unit tests) and approach the functionality
> that way. We could expose some control which determine whether or not these
> integration tests were invoked.
>
> I also commented with some output on Matt's patch from ACCUMULO-2493 -- I
> found it rather pleasant to run a single command and get a nice summary of
> his changes.
>
>
> Josh Elser wrote:
>
>> For those interested in following along with the PreCommit work, see
>> https://issues.apache.org/jira/browse/YETUS-263
>>
>> A "personality", in Yetus parlance, defines the the tests/checks that
>> PreCommit will run against Accumulo. For us, it's pretty simple. The
>> personality I provided on YETUS-263 will, for a patch/changeset run:
>>
>> * Checkstyle
>> * Findbugs
>> * RAT check
>> * @author javadoc check
>> * Some extra whitespace
>> * All unit tests (not just in the module where changes were made)
>> * Compiler warnings
>> * Javadoc warnings
>> * Presence of new unit tests
>>
>> One already built in feature that I didn't wire up is ShellCheck for our
>> shell scripts. This will require a bit of fixing on our end first.
>>
>> For more general information, Chris Nauroth wrote up a good explanation
>> for adopting the same approach in ZooKeeper (and did a much better job
>> than me
>>
>> http://mail-archives.apache.org/mod_mbox/zookeeper-dev/201512.mbox/%3CD291EB3B.3504A%25cnauroth%40hortonworks.com%3E
>> ).
>>
>>
>> Anywho, nothing really changing here yet (I'm hoping Sean will write up
>> instructions about how to configure the Jenkins job for us
>> https://issues.apache.org/jira/browse/YETUS-245). That would signify a
>> step for Accumulo specifically. Until then, this is just an FYI.
>>
>> - Josh
>>
>
--
Sean