You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@accumulo.apache.org by Josh Elser <jo...@gmail.com> on 2014/11/26 05:35:22 UTC
Fwd: [jira] [Resolved] (ACCUMULO-3167) Decouple MiniAccumuloCluster
from integration test base class
Heads up for those that might not have been following this.
I revamped how nearly all of our ITs work. In practice, it changed
relatively little in the tests themselves. If you want to know how to
use the changes I've made, check out the top-level TESTING file I
recently added.
A brief tl;dr on how write new ITs (don't worry about getting this 100%
-- I'm happy to help port new tests you might write using the old
ConfigurableMacIT or SimpleMacIT classes), is that if you need Accumulo
available for a test, extend AccumuloClusterIT and use the
getConnector(), getCluster() and getClusterControl() methods on that
class to do what you need. Try to not rely on MiniAccumuloCluster(Impl)
specific methods.
If there is something you specifically want to do with
MiniAccumuloCluster(Impl) or your test is doing something destructive or
otherwise unrecoverable to the instance, use JUnit's Assume to only run
your test when AccumuloClusterIT#getClusterType == ClusterType.MINI or
override AccumuloClusterIT#canRunTest.
As always, ask questions.
-------- Original Message --------
Subject: [jira] [Resolved] (ACCUMULO-3167) Decouple MiniAccumuloCluster
from integration test base class
Date: Wed, 26 Nov 2014 04:25:13 +0000 (UTC)
From: Josh Elser (JIRA) <ji...@apache.org>
Reply-To: jira@apache.org
To: notifications@accumulo.apache.org
[
https://issues.apache.org/jira/browse/ACCUMULO-3167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh Elser resolved ACCUMULO-3167.
----------------------------------
Resolution: Fixed
Submitted -- a couple of follow on tasks already identified.
> Decouple MiniAccumuloCluster from integration test base class
> -------------------------------------------------------------
>
> Key: ACCUMULO-3167
> URL: https://issues.apache.org/jira/browse/ACCUMULO-3167
> Project: Accumulo
> Issue Type: Improvement
> Components: mini, test
> Reporter: Josh Elser
> Assignee: Josh Elser
> Fix For: 1.6.2, 1.7.0
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> (apologies if I already had a ticket for this somewhere, I couldn't find it)
> Our integration tests are very nice and automated at the moment because we can use MiniAccumuloCluster to "provision" an Accumulo instance (or used a shared instance), and run a test against it. For the most part, this works well and provides an accurate test harness.
> Thus, to run integration tests, you need a sufficiently beefy machine since the same host will be running all of Accumulo as well as performing any client work. When resources are available to use, it would be nice to leverage them -- whether these are yarn, mesos, a vanila installation, etc.
> In addition to the additional computational power from using extra hardware, it also encourages us to use the public API as much as possible instead of relying on "hidden" impl methods in MiniAccumuloCluster.
> I propose making changes to the IT test base (AbstractMacIT, SimpleMacIT, ConfigurableMacIT) to add an extra step between them an test classes to allow "injection" of a more generic Accumulo "cluster" that is not associated with MAC.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Re: Fwd: [jira] [Resolved] (ACCUMULO-3167) Decouple MiniAccumuloCluster
from integration test base class
Posted by Josh Elser <jo...@gmail.com>.
Also, I did attempt to verify that running the ITs as before (mvn
verify). File an issue if find any newly broken test.
Josh Elser wrote:
> Heads up for those that might not have been following this.
>
> I revamped how nearly all of our ITs work. In practice, it changed
> relatively little in the tests themselves. If you want to know how to
> use the changes I've made, check out the top-level TESTING file I
> recently added.
>
> A brief tl;dr on how write new ITs (don't worry about getting this 100%
> -- I'm happy to help port new tests you might write using the old
> ConfigurableMacIT or SimpleMacIT classes), is that if you need Accumulo
> available for a test, extend AccumuloClusterIT and use the
> getConnector(), getCluster() and getClusterControl() methods on that
> class to do what you need. Try to not rely on MiniAccumuloCluster(Impl)
> specific methods.
>
> If there is something you specifically want to do with
> MiniAccumuloCluster(Impl) or your test is doing something destructive or
> otherwise unrecoverable to the instance, use JUnit's Assume to only run
> your test when AccumuloClusterIT#getClusterType == ClusterType.MINI or
> override AccumuloClusterIT#canRunTest.
>
> As always, ask questions.
>
> -------- Original Message --------
> Subject: [jira] [Resolved] (ACCUMULO-3167) Decouple MiniAccumuloCluster
> from integration test base class
> Date: Wed, 26 Nov 2014 04:25:13 +0000 (UTC)
> From: Josh Elser (JIRA) <ji...@apache.org>
> Reply-To: jira@apache.org
> To: notifications@accumulo.apache.org
>
>
> [
> https://issues.apache.org/jira/browse/ACCUMULO-3167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
> ]
>
> Josh Elser resolved ACCUMULO-3167.
> ----------------------------------
> Resolution: Fixed
>
> Submitted -- a couple of follow on tasks already identified.
>
>> Decouple MiniAccumuloCluster from integration test base class
>> -------------------------------------------------------------
>>
>> Key: ACCUMULO-3167
>> URL: https://issues.apache.org/jira/browse/ACCUMULO-3167
>> Project: Accumulo
>> Issue Type: Improvement
>> Components: mini, test
>> Reporter: Josh Elser
>> Assignee: Josh Elser
>> Fix For: 1.6.2, 1.7.0
>>
>> Time Spent: 1h 20m
>> Remaining Estimate: 0h
>>
>> (apologies if I already had a ticket for this somewhere, I couldn't
>> find it)
>> Our integration tests are very nice and automated at the moment
>> because we can use MiniAccumuloCluster to "provision" an Accumulo
>> instance (or used a shared instance), and run a test against it. For
>> the most part, this works well and provides an accurate test harness.
>> Thus, to run integration tests, you need a sufficiently beefy machine
>> since the same host will be running all of Accumulo as well as
>> performing any client work. When resources are available to use, it
>> would be nice to leverage them -- whether these are yarn, mesos, a
>> vanila installation, etc.
>> In addition to the additional computational power from using extra
>> hardware, it also encourages us to use the public API as much as
>> possible instead of relying on "hidden" impl methods in
>> MiniAccumuloCluster.
>> I propose making changes to the IT test base (AbstractMacIT,
>> SimpleMacIT, ConfigurableMacIT) to add an extra step between them an
>> test classes to allow "injection" of a more generic Accumulo "cluster"
>> that is not associated with MAC.
>
>
>
> --
> This message was sent by Atlassian JIRA
> (v6.3.4#6332)
Re: [jira] [Resolved] (ACCUMULO-3167) Decouple MiniAccumuloCluster
from integration test base class
Posted by Christopher <ct...@apache.org>.
I've thought a *tiny* bit about this, and have thought that it might be
useful to switch over the "-P sunny" profile in maven to use categories[1]
(groups in surefire[2]). I haven't put a lot of thought into it, but it
might be one way to utilize categories, if we decide to use these
annotations as part of a more comprehensive organization of our tests.
[1]
https://weblogs.java.net/blog/johnsmart/archive/2010/04/25/grouping-tests-using-junit-categories-0
[2]
http://maven.apache.org/surefire/maven-surefire-plugin/test-mojo.html#groups
--
Christopher L Tubbs II
http://gravatar.com/ctubbsii
On Wed, Nov 26, 2014 at 12:17 AM, Josh Elser <jo...@gmail.com> wrote:
> Please file a JIRA and make the proposal on how it would fit into a test.
> Preferably, change all of the tests too :)
>
>
> Mike Drob wrote:
>
>> Instead of relying on Assumes and/or an overriden method, we should have
>> category annotations.
>>
>> https://github.com/junit-team/junit/wiki/Categories
>>
>> On Tue, Nov 25, 2014 at 10:35 PM, Josh Elser<jo...@gmail.com>
>> wrote:
>>
>> Heads up for those that might not have been following this.
>>>
>>> I revamped how nearly all of our ITs work. In practice, it changed
>>> relatively little in the tests themselves. If you want to know how to use
>>> the changes I've made, check out the top-level TESTING file I recently
>>> added.
>>>
>>> A brief tl;dr on how write new ITs (don't worry about getting this 100%
>>> --
>>> I'm happy to help port new tests you might write using the old
>>> ConfigurableMacIT or SimpleMacIT classes), is that if you need Accumulo
>>> available for a test, extend AccumuloClusterIT and use the
>>> getConnector(),
>>> getCluster() and getClusterControl() methods on that class to do what you
>>> need. Try to not rely on MiniAccumuloCluster(Impl) specific methods.
>>>
>>> If there is something you specifically want to do with
>>> MiniAccumuloCluster(Impl) or your test is doing something destructive or
>>> otherwise unrecoverable to the instance, use JUnit's Assume to only run
>>> your test when AccumuloClusterIT#getClusterType == ClusterType.MINI or
>>> override AccumuloClusterIT#canRunTest.
>>>
>>> As always, ask questions.
>>>
>>> -------- Original Message --------
>>> Subject: [jira] [Resolved] (ACCUMULO-3167) Decouple MiniAccumuloCluster
>>> from integration test base class
>>> Date: Wed, 26 Nov 2014 04:25:13 +0000 (UTC)
>>> From: Josh Elser (JIRA)<ji...@apache.org>
>>> Reply-To: jira@apache.org
>>> To: notifications@accumulo.apache.org
>>>
>>>
>>> [ https://issues.apache.org/jira/browse/ACCUMULO-3167?
>>> page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
>>>
>>> Josh Elser resolved ACCUMULO-3167.
>>> ----------------------------------
>>> Resolution: Fixed
>>>
>>> Submitted -- a couple of follow on tasks already identified.
>>>
>>> Decouple MiniAccumuloCluster from integration test base class
>>>
>>>> -------------------------------------------------------------
>>>>
>>>> Key: ACCUMULO-3167
>>>> URL: https://issues.apache.org/
>>>> jira/browse/ACCUMULO-3167
>>>> Project: Accumulo
>>>> Issue Type: Improvement
>>>> Components: mini, test
>>>> Reporter: Josh Elser
>>>> Assignee: Josh Elser
>>>> Fix For: 1.6.2, 1.7.0
>>>>
>>>> Time Spent: 1h 20m
>>>> Remaining Estimate: 0h
>>>>
>>>> (apologies if I already had a ticket for this somewhere, I couldn't find
>>>> it)
>>>> Our integration tests are very nice and automated at the moment because
>>>> we can use MiniAccumuloCluster to "provision" an Accumulo instance (or
>>>> used
>>>> a shared instance), and run a test against it. For the most part, this
>>>> works well and provides an accurate test harness.
>>>> Thus, to run integration tests, you need a sufficiently beefy machine
>>>> since the same host will be running all of Accumulo as well as
>>>> performing
>>>> any client work. When resources are available to use, it would be nice
>>>> to
>>>> leverage them -- whether these are yarn, mesos, a vanila installation,
>>>> etc.
>>>> In addition to the additional computational power from using extra
>>>> hardware, it also encourages us to use the public API as much as
>>>> possible
>>>> instead of relying on "hidden" impl methods in MiniAccumuloCluster.
>>>> I propose making changes to the IT test base (AbstractMacIT,
>>>> SimpleMacIT,
>>>> ConfigurableMacIT) to add an extra step between them an test classes to
>>>> allow "injection" of a more generic Accumulo "cluster" that is not
>>>> associated with MAC.
>>>>
>>>>
>>>
>>> --
>>> This message was sent by Atlassian JIRA
>>> (v6.3.4#6332)
>>>
>>>
>>
Re: [jira] [Resolved] (ACCUMULO-3167) Decouple MiniAccumuloCluster
from integration test base class
Posted by Josh Elser <jo...@gmail.com>.
Please file a JIRA and make the proposal on how it would fit into a
test. Preferably, change all of the tests too :)
Mike Drob wrote:
> Instead of relying on Assumes and/or an overriden method, we should have
> category annotations.
>
> https://github.com/junit-team/junit/wiki/Categories
>
> On Tue, Nov 25, 2014 at 10:35 PM, Josh Elser<jo...@gmail.com> wrote:
>
>> Heads up for those that might not have been following this.
>>
>> I revamped how nearly all of our ITs work. In practice, it changed
>> relatively little in the tests themselves. If you want to know how to use
>> the changes I've made, check out the top-level TESTING file I recently
>> added.
>>
>> A brief tl;dr on how write new ITs (don't worry about getting this 100% --
>> I'm happy to help port new tests you might write using the old
>> ConfigurableMacIT or SimpleMacIT classes), is that if you need Accumulo
>> available for a test, extend AccumuloClusterIT and use the getConnector(),
>> getCluster() and getClusterControl() methods on that class to do what you
>> need. Try to not rely on MiniAccumuloCluster(Impl) specific methods.
>>
>> If there is something you specifically want to do with
>> MiniAccumuloCluster(Impl) or your test is doing something destructive or
>> otherwise unrecoverable to the instance, use JUnit's Assume to only run
>> your test when AccumuloClusterIT#getClusterType == ClusterType.MINI or
>> override AccumuloClusterIT#canRunTest.
>>
>> As always, ask questions.
>>
>> -------- Original Message --------
>> Subject: [jira] [Resolved] (ACCUMULO-3167) Decouple MiniAccumuloCluster
>> from integration test base class
>> Date: Wed, 26 Nov 2014 04:25:13 +0000 (UTC)
>> From: Josh Elser (JIRA)<ji...@apache.org>
>> Reply-To: jira@apache.org
>> To: notifications@accumulo.apache.org
>>
>>
>> [ https://issues.apache.org/jira/browse/ACCUMULO-3167?
>> page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
>>
>> Josh Elser resolved ACCUMULO-3167.
>> ----------------------------------
>> Resolution: Fixed
>>
>> Submitted -- a couple of follow on tasks already identified.
>>
>> Decouple MiniAccumuloCluster from integration test base class
>>> -------------------------------------------------------------
>>>
>>> Key: ACCUMULO-3167
>>> URL: https://issues.apache.org/jira/browse/ACCUMULO-3167
>>> Project: Accumulo
>>> Issue Type: Improvement
>>> Components: mini, test
>>> Reporter: Josh Elser
>>> Assignee: Josh Elser
>>> Fix For: 1.6.2, 1.7.0
>>>
>>> Time Spent: 1h 20m
>>> Remaining Estimate: 0h
>>>
>>> (apologies if I already had a ticket for this somewhere, I couldn't find
>>> it)
>>> Our integration tests are very nice and automated at the moment because
>>> we can use MiniAccumuloCluster to "provision" an Accumulo instance (or used
>>> a shared instance), and run a test against it. For the most part, this
>>> works well and provides an accurate test harness.
>>> Thus, to run integration tests, you need a sufficiently beefy machine
>>> since the same host will be running all of Accumulo as well as performing
>>> any client work. When resources are available to use, it would be nice to
>>> leverage them -- whether these are yarn, mesos, a vanila installation, etc.
>>> In addition to the additional computational power from using extra
>>> hardware, it also encourages us to use the public API as much as possible
>>> instead of relying on "hidden" impl methods in MiniAccumuloCluster.
>>> I propose making changes to the IT test base (AbstractMacIT, SimpleMacIT,
>>> ConfigurableMacIT) to add an extra step between them an test classes to
>>> allow "injection" of a more generic Accumulo "cluster" that is not
>>> associated with MAC.
>>>
>>
>>
>> --
>> This message was sent by Atlassian JIRA
>> (v6.3.4#6332)
>>
>
Re: [jira] [Resolved] (ACCUMULO-3167) Decouple MiniAccumuloCluster
from integration test base class
Posted by Mike Drob <md...@mdrob.com>.
Instead of relying on Assumes and/or an overriden method, we should have
category annotations.
https://github.com/junit-team/junit/wiki/Categories
On Tue, Nov 25, 2014 at 10:35 PM, Josh Elser <jo...@gmail.com> wrote:
> Heads up for those that might not have been following this.
>
> I revamped how nearly all of our ITs work. In practice, it changed
> relatively little in the tests themselves. If you want to know how to use
> the changes I've made, check out the top-level TESTING file I recently
> added.
>
> A brief tl;dr on how write new ITs (don't worry about getting this 100% --
> I'm happy to help port new tests you might write using the old
> ConfigurableMacIT or SimpleMacIT classes), is that if you need Accumulo
> available for a test, extend AccumuloClusterIT and use the getConnector(),
> getCluster() and getClusterControl() methods on that class to do what you
> need. Try to not rely on MiniAccumuloCluster(Impl) specific methods.
>
> If there is something you specifically want to do with
> MiniAccumuloCluster(Impl) or your test is doing something destructive or
> otherwise unrecoverable to the instance, use JUnit's Assume to only run
> your test when AccumuloClusterIT#getClusterType == ClusterType.MINI or
> override AccumuloClusterIT#canRunTest.
>
> As always, ask questions.
>
> -------- Original Message --------
> Subject: [jira] [Resolved] (ACCUMULO-3167) Decouple MiniAccumuloCluster
> from integration test base class
> Date: Wed, 26 Nov 2014 04:25:13 +0000 (UTC)
> From: Josh Elser (JIRA) <ji...@apache.org>
> Reply-To: jira@apache.org
> To: notifications@accumulo.apache.org
>
>
> [ https://issues.apache.org/jira/browse/ACCUMULO-3167?
> page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
>
> Josh Elser resolved ACCUMULO-3167.
> ----------------------------------
> Resolution: Fixed
>
> Submitted -- a couple of follow on tasks already identified.
>
> Decouple MiniAccumuloCluster from integration test base class
>> -------------------------------------------------------------
>>
>> Key: ACCUMULO-3167
>> URL: https://issues.apache.org/jira/browse/ACCUMULO-3167
>> Project: Accumulo
>> Issue Type: Improvement
>> Components: mini, test
>> Reporter: Josh Elser
>> Assignee: Josh Elser
>> Fix For: 1.6.2, 1.7.0
>>
>> Time Spent: 1h 20m
>> Remaining Estimate: 0h
>>
>> (apologies if I already had a ticket for this somewhere, I couldn't find
>> it)
>> Our integration tests are very nice and automated at the moment because
>> we can use MiniAccumuloCluster to "provision" an Accumulo instance (or used
>> a shared instance), and run a test against it. For the most part, this
>> works well and provides an accurate test harness.
>> Thus, to run integration tests, you need a sufficiently beefy machine
>> since the same host will be running all of Accumulo as well as performing
>> any client work. When resources are available to use, it would be nice to
>> leverage them -- whether these are yarn, mesos, a vanila installation, etc.
>> In addition to the additional computational power from using extra
>> hardware, it also encourages us to use the public API as much as possible
>> instead of relying on "hidden" impl methods in MiniAccumuloCluster.
>> I propose making changes to the IT test base (AbstractMacIT, SimpleMacIT,
>> ConfigurableMacIT) to add an extra step between them an test classes to
>> allow "injection" of a more generic Accumulo "cluster" that is not
>> associated with MAC.
>>
>
>
>
> --
> This message was sent by Atlassian JIRA
> (v6.3.4#6332)
>