You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@stratos.apache.org by Isuru Haththotuwa <is...@apache.org> on 2015/05/10 20:42:46 UTC
[Discuss] Extending Integration Tests to Support Complex Unit Tests
Hi Devs,
I had a look to see how we can improve the code coverage with unit testing.
There are a few challenges for this in Stratos:
1. Operations are not local to a single component - One component calls
other to do verifications, etc. Example: deploying a service group;
Autoscaler will call CC to verify the cartridge details
2. Services are not simple, hard to mock - Since the exposed axis2
services are complex, its hard to mock them using available tools. We can
still do this bit will involve some re-factoring to the code.
As a potential solution to improve the tests, I tried to use the
integration test mechanism and adapt it to test simple flows as well;
deploying and validating various aspects of Service Groups (dependencies,
startup orders, etc.), different kinds of policies used in Stratos, etc.
Using the integration test mechanism solves the problem of operations not
being local and difficulty to mock them; the Stratos server and MB can be
used to execute any flow and test it.
What I propose is to add a layer on top of the currently integration tests
framework to expose a set of utility methods to deploy cartridges, groups,
policies, etc. to help developers to write tests using those easily. All
the tests will be executed against a single Stratos Server without starting
a separate server each time. Did some refactoring locally to suit this
purpose and wrote several simple tests as well, which seem to work well.
Please share your thoughts.
--
Thanks and Regards,
Isuru H.
+94 716 358 048* <http://wso2.com/>*
Re: [Discuss] Extending Integration Tests to Support Complex Unit Tests
Posted by Imesh Gunaratne <im...@apache.org>.
+1 Yes we could expose more API methods in Mock IaaS to make those
configurable dynamically.
On Fri, May 22, 2015 at 3:22 PM, Reka Thirunavukkarasu <re...@wso2.com>
wrote:
> Hi Isuru,
>
> It will be good initiative for adding more samples. Can we keep the
> mock-iaas.xml and deploy or undeploy or scaleup or scaledown timeout
> configurable per sample wise? So that we can assert based on the timeout
> for deploy or undeploy or scaleup or scaledown scenarios where we can
> calculate the timeout based on the parameters given in the mock-iaas.xml.
> WDYT?
>
> However I'm not sure how feasible is to test the scaleup and scaledown.
> This is just my thought.
>
> Thanks,
> Reka
>
> On Fri, May 22, 2015 at 12:13 PM, Isuru Haththotuwa <is...@apache.org>
> wrote:
>
>> Hi Devs,
>>
>> Did the initial changes and tested locally. This provides the ability to
>> easily write tests for scenarios which are hard to cover by unit tests, due
>> to distributed nature, etc. The initial abstracted API is shown below,
>> which is overriden for artifact specific implementation.
>>
>> However, I would not commit this now to master since the community is
>> preparing for 4.1.0 release. Will try to merge and push after 4.1.0 is done.
>>
>> /**
>> * Deploy the artifact in the file ${definitionFileName}.json by
>> calling the deploy.sh script
>> * under samples/xxx/scripts/.
>> * The file name excluding .json part is passed to the script to
>> locate the relevant artifact definition.
>> *
>> * @param iaas IaaS type
>> * @param definitionFileName definition file name, excluding .json
>> suffix
>> * @return output from the bash command execution
>> */
>> * public String deploy (String iaas, String definitionFileName)*
>>
>> /**
>> * Assert if an artifact is deployed with the given id
>> *
>> * @param artifactId artifact id to check
>> */
>> *public void assertDeployed (String artifactId)*
>>
>> /**
>> * Undeploys the artifact with id 'artifactId', by calling the
>> undeploy.sh script under samples/xxx/scripts.
>> * The iaas and artifact id is passed as a parameter to the script.
>> *
>> * @param artifactId id of the artifact to undeploy
>> * @return output from the bash command execution
>> */
>> *public String undeploy (String artifactId)*
>>
>> /**
>> * Executes the script get.sh under samples/xxxx/scripts and returns
>> the command output
>> *
>> * @param artifactId id of artifact to GET
>> * @return output of the command execution
>> */
>> * public String get (String artifactId)*
>>
>> /**
>> * Assert if no artifact is deployed with the id 'artifactId'
>> *
>> * @param artifactId id of the artifact to check
>> */
>> * public void assertNotDeployed (String artifactId)*
>>
>> /**
>> * Get the relevant artifact directory
>> *
>> * @return path to the artifact directory
>> */
>> * protected String getArtifactLocation ()*
>>
>>
>> On Mon, May 11, 2015 at 12:00 PM, Udara Liyanage <ud...@wso2.com> wrote:
>>
>>> Hi,
>>>
>>> +1 for generic approach so we can automate any sample without much
>>> change.
>>>
>>>
>>> On Mon, May 11, 2015 at 11:55 AM, Isuru Haththotuwa <is...@apache.org>
>>> wrote:
>>>
>>>> Hi Imesh,
>>>>
>>>> Thanks for the feedback.
>>>>
>>>> What I tried out is similar to what is there currently; I added a
>>>> script for each operation and have it parameterized to call the relevant
>>>> file.
>>>>
>>>> For an example, in the samples/cartridges-groups directory, there will
>>>> be scripts for common operations; deploy, undeploy, etc. The scripts are
>>>> parameterized to take an input specifying which group to deploy/undeploy,
>>>> etc. These scripts will be called from the group related test utility
>>>> methods. Following this approach, we can call any operation on any of the
>>>> cartridges, groups and policies.
>>>>
>>>> On Mon, May 11, 2015 at 12:39 AM, Imesh Gunaratne <im...@apache.org>
>>>> wrote:
>>>>
>>>>> Hi Isuru,
>>>>>
>>>>> +1 for improving the Integration Test Framework with utility methods.
>>>>>
>>>>> Currently we directly invoke the deploy.sh script and it internally
>>>>> deploy cartridges, cartridge groups, policies, etc. How are we planning to
>>>>> do this with the new approach? It would be better if you could send the
>>>>> refactored code.
>>>>>
>>>>> Thanks
>>>>>
>>>>> On Mon, May 11, 2015 at 12:12 AM, Isuru Haththotuwa <isuruh@apache.org
>>>>> > wrote:
>>>>>
>>>>>> Hi Devs,
>>>>>>
>>>>>> I had a look to see how we can improve the code coverage with unit
>>>>>> testing. There are a few challenges for this in Stratos:
>>>>>>
>>>>>> 1. Operations are not local to a single component - One component
>>>>>> calls other to do verifications, etc. Example: deploying a service group;
>>>>>> Autoscaler will call CC to verify the cartridge details
>>>>>> 2. Services are not simple, hard to mock - Since the exposed
>>>>>> axis2 services are complex, its hard to mock them using available tools. We
>>>>>> can still do this bit will involve some re-factoring to the code.
>>>>>>
>>>>>> As a potential solution to improve the tests, I tried to use the
>>>>>> integration test mechanism and adapt it to test simple flows as well;
>>>>>> deploying and validating various aspects of Service Groups (dependencies,
>>>>>> startup orders, etc.), different kinds of policies used in Stratos, etc.
>>>>>> Using the integration test mechanism solves the problem of operations not
>>>>>> being local and difficulty to mock them; the Stratos server and MB can be
>>>>>> used to execute any flow and test it.
>>>>>>
>>>>>> What I propose is to add a layer on top of the currently integration
>>>>>> tests framework to expose a set of utility methods to deploy cartridges,
>>>>>> groups, policies, etc. to help developers to write tests using those
>>>>>> easily. All the tests will be executed against a single Stratos Server
>>>>>> without starting a separate server each time. Did some refactoring locally
>>>>>> to suit this purpose and wrote several simple tests as well, which seem to
>>>>>> work well.
>>>>>>
>>>>>> Please share your thoughts.
>>>>>>
>>>>>> --
>>>>>> Thanks and Regards,
>>>>>>
>>>>>> Isuru H.
>>>>>> +94 716 358 048* <http://wso2.com/>*
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Imesh Gunaratne
>>>>>
>>>>> Senior Technical Lead, WSO2
>>>>> Committer & PMC Member, Apache Stratos
>>>>>
>>>>> --
>>>>> Thanks and Regards,
>>>>>
>>>>> Isuru H.
>>>>> +94 716 358 048* <http://wso2.com/>*
>>>>>
>>>>>
>>>>> * <http://wso2.com/>*
>>>>>
>>>>>
>>>>>
>>>
>>>
>>> --
>>>
>>> Udara Liyanage
>>> Software Engineer
>>> WSO2, Inc.: http://wso2.com
>>> lean. enterprise. middleware
>>>
>>> web: http://udaraliyanage.wordpress.com
>>> phone: +94 71 443 6897
>>>
>>> --
>>> Thanks and Regards,
>>>
>>> Isuru H.
>>> +94 716 358 048* <http://wso2.com/>*
>>>
>>>
>>> * <http://wso2.com/>*
>>>
>>>
>>>
>
>
> --
> Reka Thirunavukkarasu
> Senior Software Engineer,
> WSO2, Inc.:http://wso2.com,
> Mobile: +94776442007
>
>
>
--
Imesh Gunaratne
Senior Technical Lead, WSO2
Committer & PMC Member, Apache Stratos
Re: [Discuss] Extending Integration Tests to Support Complex Unit Tests
Posted by Reka Thirunavukkarasu <re...@wso2.com>.
Hi Isuru,
It will be good initiative for adding more samples. Can we keep the
mock-iaas.xml and deploy or undeploy or scaleup or scaledown timeout
configurable per sample wise? So that we can assert based on the timeout
for deploy or undeploy or scaleup or scaledown scenarios where we can
calculate the timeout based on the parameters given in the mock-iaas.xml.
WDYT?
However I'm not sure how feasible is to test the scaleup and scaledown.
This is just my thought.
Thanks,
Reka
On Fri, May 22, 2015 at 12:13 PM, Isuru Haththotuwa <is...@apache.org>
wrote:
> Hi Devs,
>
> Did the initial changes and tested locally. This provides the ability to
> easily write tests for scenarios which are hard to cover by unit tests, due
> to distributed nature, etc. The initial abstracted API is shown below,
> which is overriden for artifact specific implementation.
>
> However, I would not commit this now to master since the community is
> preparing for 4.1.0 release. Will try to merge and push after 4.1.0 is done.
>
> /**
> * Deploy the artifact in the file ${definitionFileName}.json by
> calling the deploy.sh script
> * under samples/xxx/scripts/.
> * The file name excluding .json part is passed to the script to
> locate the relevant artifact definition.
> *
> * @param iaas IaaS type
> * @param definitionFileName definition file name, excluding .json
> suffix
> * @return output from the bash command execution
> */
> * public String deploy (String iaas, String definitionFileName)*
>
> /**
> * Assert if an artifact is deployed with the given id
> *
> * @param artifactId artifact id to check
> */
> *public void assertDeployed (String artifactId)*
>
> /**
> * Undeploys the artifact with id 'artifactId', by calling the
> undeploy.sh script under samples/xxx/scripts.
> * The iaas and artifact id is passed as a parameter to the script.
> *
> * @param artifactId id of the artifact to undeploy
> * @return output from the bash command execution
> */
> *public String undeploy (String artifactId)*
>
> /**
> * Executes the script get.sh under samples/xxxx/scripts and returns
> the command output
> *
> * @param artifactId id of artifact to GET
> * @return output of the command execution
> */
> * public String get (String artifactId)*
>
> /**
> * Assert if no artifact is deployed with the id 'artifactId'
> *
> * @param artifactId id of the artifact to check
> */
> * public void assertNotDeployed (String artifactId)*
>
> /**
> * Get the relevant artifact directory
> *
> * @return path to the artifact directory
> */
> * protected String getArtifactLocation ()*
>
>
> On Mon, May 11, 2015 at 12:00 PM, Udara Liyanage <ud...@wso2.com> wrote:
>
>> Hi,
>>
>> +1 for generic approach so we can automate any sample without much change.
>>
>>
>> On Mon, May 11, 2015 at 11:55 AM, Isuru Haththotuwa <is...@apache.org>
>> wrote:
>>
>>> Hi Imesh,
>>>
>>> Thanks for the feedback.
>>>
>>> What I tried out is similar to what is there currently; I added a script
>>> for each operation and have it parameterized to call the relevant file.
>>>
>>> For an example, in the samples/cartridges-groups directory, there will
>>> be scripts for common operations; deploy, undeploy, etc. The scripts are
>>> parameterized to take an input specifying which group to deploy/undeploy,
>>> etc. These scripts will be called from the group related test utility
>>> methods. Following this approach, we can call any operation on any of the
>>> cartridges, groups and policies.
>>>
>>> On Mon, May 11, 2015 at 12:39 AM, Imesh Gunaratne <im...@apache.org>
>>> wrote:
>>>
>>>> Hi Isuru,
>>>>
>>>> +1 for improving the Integration Test Framework with utility methods.
>>>>
>>>> Currently we directly invoke the deploy.sh script and it internally
>>>> deploy cartridges, cartridge groups, policies, etc. How are we planning to
>>>> do this with the new approach? It would be better if you could send the
>>>> refactored code.
>>>>
>>>> Thanks
>>>>
>>>> On Mon, May 11, 2015 at 12:12 AM, Isuru Haththotuwa <is...@apache.org>
>>>> wrote:
>>>>
>>>>> Hi Devs,
>>>>>
>>>>> I had a look to see how we can improve the code coverage with unit
>>>>> testing. There are a few challenges for this in Stratos:
>>>>>
>>>>> 1. Operations are not local to a single component - One component
>>>>> calls other to do verifications, etc. Example: deploying a service group;
>>>>> Autoscaler will call CC to verify the cartridge details
>>>>> 2. Services are not simple, hard to mock - Since the exposed axis2
>>>>> services are complex, its hard to mock them using available tools. We can
>>>>> still do this bit will involve some re-factoring to the code.
>>>>>
>>>>> As a potential solution to improve the tests, I tried to use the
>>>>> integration test mechanism and adapt it to test simple flows as well;
>>>>> deploying and validating various aspects of Service Groups (dependencies,
>>>>> startup orders, etc.), different kinds of policies used in Stratos, etc.
>>>>> Using the integration test mechanism solves the problem of operations not
>>>>> being local and difficulty to mock them; the Stratos server and MB can be
>>>>> used to execute any flow and test it.
>>>>>
>>>>> What I propose is to add a layer on top of the currently integration
>>>>> tests framework to expose a set of utility methods to deploy cartridges,
>>>>> groups, policies, etc. to help developers to write tests using those
>>>>> easily. All the tests will be executed against a single Stratos Server
>>>>> without starting a separate server each time. Did some refactoring locally
>>>>> to suit this purpose and wrote several simple tests as well, which seem to
>>>>> work well.
>>>>>
>>>>> Please share your thoughts.
>>>>>
>>>>> --
>>>>> Thanks and Regards,
>>>>>
>>>>> Isuru H.
>>>>> +94 716 358 048* <http://wso2.com/>*
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Imesh Gunaratne
>>>>
>>>> Senior Technical Lead, WSO2
>>>> Committer & PMC Member, Apache Stratos
>>>>
>>>> --
>>>> Thanks and Regards,
>>>>
>>>> Isuru H.
>>>> +94 716 358 048* <http://wso2.com/>*
>>>>
>>>>
>>>> * <http://wso2.com/>*
>>>>
>>>>
>>>>
>>
>>
>> --
>>
>> Udara Liyanage
>> Software Engineer
>> WSO2, Inc.: http://wso2.com
>> lean. enterprise. middleware
>>
>> web: http://udaraliyanage.wordpress.com
>> phone: +94 71 443 6897
>>
>> --
>> Thanks and Regards,
>>
>> Isuru H.
>> +94 716 358 048* <http://wso2.com/>*
>>
>>
>> * <http://wso2.com/>*
>>
>>
>>
--
Reka Thirunavukkarasu
Senior Software Engineer,
WSO2, Inc.:http://wso2.com,
Mobile: +94776442007
Re: [Discuss] Extending Integration Tests to Support Complex Unit Tests
Posted by Isuru Haththotuwa <is...@apache.org>.
Hi Devs,
Did the initial changes and tested locally. This provides the ability to
easily write tests for scenarios which are hard to cover by unit tests, due
to distributed nature, etc. The initial abstracted API is shown below,
which is overriden for artifact specific implementation.
However, I would not commit this now to master since the community is
preparing for 4.1.0 release. Will try to merge and push after 4.1.0 is done.
/**
* Deploy the artifact in the file ${definitionFileName}.json by
calling the deploy.sh script
* under samples/xxx/scripts/.
* The file name excluding .json part is passed to the script to locate
the relevant artifact definition.
*
* @param iaas IaaS type
* @param definitionFileName definition file name, excluding .json
suffix
* @return output from the bash command execution
*/
* public String deploy (String iaas, String definitionFileName)*
/**
* Assert if an artifact is deployed with the given id
*
* @param artifactId artifact id to check
*/
*public void assertDeployed (String artifactId)*
/**
* Undeploys the artifact with id 'artifactId', by calling the
undeploy.sh script under samples/xxx/scripts.
* The iaas and artifact id is passed as a parameter to the script.
*
* @param artifactId id of the artifact to undeploy
* @return output from the bash command execution
*/
*public String undeploy (String artifactId)*
/**
* Executes the script get.sh under samples/xxxx/scripts and returns
the command output
*
* @param artifactId id of artifact to GET
* @return output of the command execution
*/
* public String get (String artifactId)*
/**
* Assert if no artifact is deployed with the id 'artifactId'
*
* @param artifactId id of the artifact to check
*/
* public void assertNotDeployed (String artifactId)*
/**
* Get the relevant artifact directory
*
* @return path to the artifact directory
*/
* protected String getArtifactLocation ()*
On Mon, May 11, 2015 at 12:00 PM, Udara Liyanage <ud...@wso2.com> wrote:
> Hi,
>
> +1 for generic approach so we can automate any sample without much change.
>
>
> On Mon, May 11, 2015 at 11:55 AM, Isuru Haththotuwa <is...@apache.org>
> wrote:
>
>> Hi Imesh,
>>
>> Thanks for the feedback.
>>
>> What I tried out is similar to what is there currently; I added a script
>> for each operation and have it parameterized to call the relevant file.
>>
>> For an example, in the samples/cartridges-groups directory, there will be
>> scripts for common operations; deploy, undeploy, etc. The scripts are
>> parameterized to take an input specifying which group to deploy/undeploy,
>> etc. These scripts will be called from the group related test utility
>> methods. Following this approach, we can call any operation on any of the
>> cartridges, groups and policies.
>>
>> On Mon, May 11, 2015 at 12:39 AM, Imesh Gunaratne <im...@apache.org>
>> wrote:
>>
>>> Hi Isuru,
>>>
>>> +1 for improving the Integration Test Framework with utility methods.
>>>
>>> Currently we directly invoke the deploy.sh script and it internally
>>> deploy cartridges, cartridge groups, policies, etc. How are we planning to
>>> do this with the new approach? It would be better if you could send the
>>> refactored code.
>>>
>>> Thanks
>>>
>>> On Mon, May 11, 2015 at 12:12 AM, Isuru Haththotuwa <is...@apache.org>
>>> wrote:
>>>
>>>> Hi Devs,
>>>>
>>>> I had a look to see how we can improve the code coverage with unit
>>>> testing. There are a few challenges for this in Stratos:
>>>>
>>>> 1. Operations are not local to a single component - One component
>>>> calls other to do verifications, etc. Example: deploying a service group;
>>>> Autoscaler will call CC to verify the cartridge details
>>>> 2. Services are not simple, hard to mock - Since the exposed axis2
>>>> services are complex, its hard to mock them using available tools. We can
>>>> still do this bit will involve some re-factoring to the code.
>>>>
>>>> As a potential solution to improve the tests, I tried to use the
>>>> integration test mechanism and adapt it to test simple flows as well;
>>>> deploying and validating various aspects of Service Groups (dependencies,
>>>> startup orders, etc.), different kinds of policies used in Stratos, etc.
>>>> Using the integration test mechanism solves the problem of operations not
>>>> being local and difficulty to mock them; the Stratos server and MB can be
>>>> used to execute any flow and test it.
>>>>
>>>> What I propose is to add a layer on top of the currently integration
>>>> tests framework to expose a set of utility methods to deploy cartridges,
>>>> groups, policies, etc. to help developers to write tests using those
>>>> easily. All the tests will be executed against a single Stratos Server
>>>> without starting a separate server each time. Did some refactoring locally
>>>> to suit this purpose and wrote several simple tests as well, which seem to
>>>> work well.
>>>>
>>>> Please share your thoughts.
>>>>
>>>> --
>>>> Thanks and Regards,
>>>>
>>>> Isuru H.
>>>> +94 716 358 048* <http://wso2.com/>*
>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> Imesh Gunaratne
>>>
>>> Senior Technical Lead, WSO2
>>> Committer & PMC Member, Apache Stratos
>>>
>>> --
>>> Thanks and Regards,
>>>
>>> Isuru H.
>>> +94 716 358 048* <http://wso2.com/>*
>>>
>>>
>>> * <http://wso2.com/>*
>>>
>>>
>>>
>
>
> --
>
> Udara Liyanage
> Software Engineer
> WSO2, Inc.: http://wso2.com
> lean. enterprise. middleware
>
> web: http://udaraliyanage.wordpress.com
> phone: +94 71 443 6897
>
> --
> Thanks and Regards,
>
> Isuru H.
> +94 716 358 048* <http://wso2.com/>*
>
>
> * <http://wso2.com/>*
>
>
>
Re: [Discuss] Extending Integration Tests to Support Complex Unit Tests
Posted by Udara Liyanage <ud...@wso2.com>.
Hi,
+1 for generic approach so we can automate any sample without much change.
On Mon, May 11, 2015 at 11:55 AM, Isuru Haththotuwa <is...@apache.org>
wrote:
> Hi Imesh,
>
> Thanks for the feedback.
>
> What I tried out is similar to what is there currently; I added a script
> for each operation and have it parameterized to call the relevant file.
>
> For an example, in the samples/cartridges-groups directory, there will be
> scripts for common operations; deploy, undeploy, etc. The scripts are
> parameterized to take an input specifying which group to deploy/undeploy,
> etc. These scripts will be called from the group related test utility
> methods. Following this approach, we can call any operation on any of the
> cartridges, groups and policies.
>
> On Mon, May 11, 2015 at 12:39 AM, Imesh Gunaratne <im...@apache.org>
> wrote:
>
>> Hi Isuru,
>>
>> +1 for improving the Integration Test Framework with utility methods.
>>
>> Currently we directly invoke the deploy.sh script and it internally
>> deploy cartridges, cartridge groups, policies, etc. How are we planning to
>> do this with the new approach? It would be better if you could send the
>> refactored code.
>>
>> Thanks
>>
>> On Mon, May 11, 2015 at 12:12 AM, Isuru Haththotuwa <is...@apache.org>
>> wrote:
>>
>>> Hi Devs,
>>>
>>> I had a look to see how we can improve the code coverage with unit
>>> testing. There are a few challenges for this in Stratos:
>>>
>>> 1. Operations are not local to a single component - One component
>>> calls other to do verifications, etc. Example: deploying a service group;
>>> Autoscaler will call CC to verify the cartridge details
>>> 2. Services are not simple, hard to mock - Since the exposed axis2
>>> services are complex, its hard to mock them using available tools. We can
>>> still do this bit will involve some re-factoring to the code.
>>>
>>> As a potential solution to improve the tests, I tried to use the
>>> integration test mechanism and adapt it to test simple flows as well;
>>> deploying and validating various aspects of Service Groups (dependencies,
>>> startup orders, etc.), different kinds of policies used in Stratos, etc.
>>> Using the integration test mechanism solves the problem of operations not
>>> being local and difficulty to mock them; the Stratos server and MB can be
>>> used to execute any flow and test it.
>>>
>>> What I propose is to add a layer on top of the currently integration
>>> tests framework to expose a set of utility methods to deploy cartridges,
>>> groups, policies, etc. to help developers to write tests using those
>>> easily. All the tests will be executed against a single Stratos Server
>>> without starting a separate server each time. Did some refactoring locally
>>> to suit this purpose and wrote several simple tests as well, which seem to
>>> work well.
>>>
>>> Please share your thoughts.
>>>
>>> --
>>> Thanks and Regards,
>>>
>>> Isuru H.
>>> +94 716 358 048* <http://wso2.com/>*
>>>
>>>
>>>
>>
>>
>> --
>> Imesh Gunaratne
>>
>> Senior Technical Lead, WSO2
>> Committer & PMC Member, Apache Stratos
>>
>> --
>> Thanks and Regards,
>>
>> Isuru H.
>> +94 716 358 048* <http://wso2.com/>*
>>
>>
>> * <http://wso2.com/>*
>>
>>
>>
--
Udara Liyanage
Software Engineer
WSO2, Inc.: http://wso2.com
lean. enterprise. middleware
web: http://udaraliyanage.wordpress.com
phone: +94 71 443 6897
Re: [Discuss] Extending Integration Tests to Support Complex Unit Tests
Posted by Isuru Haththotuwa <is...@apache.org>.
Hi Imesh,
Thanks for the feedback.
What I tried out is similar to what is there currently; I added a script
for each operation and have it parameterized to call the relevant file.
For an example, in the samples/cartridges-groups directory, there will be
scripts for common operations; deploy, undeploy, etc. The scripts are
parameterized to take an input specifying which group to deploy/undeploy,
etc. These scripts will be called from the group related test utility
methods. Following this approach, we can call any operation on any of the
cartridges, groups and policies.
On Mon, May 11, 2015 at 12:39 AM, Imesh Gunaratne <im...@apache.org> wrote:
> Hi Isuru,
>
> +1 for improving the Integration Test Framework with utility methods.
>
> Currently we directly invoke the deploy.sh script and it internally deploy
> cartridges, cartridge groups, policies, etc. How are we planning to do this
> with the new approach? It would be better if you could send the refactored
> code.
>
> Thanks
>
> On Mon, May 11, 2015 at 12:12 AM, Isuru Haththotuwa <is...@apache.org>
> wrote:
>
>> Hi Devs,
>>
>> I had a look to see how we can improve the code coverage with unit
>> testing. There are a few challenges for this in Stratos:
>>
>> 1. Operations are not local to a single component - One component
>> calls other to do verifications, etc. Example: deploying a service group;
>> Autoscaler will call CC to verify the cartridge details
>> 2. Services are not simple, hard to mock - Since the exposed axis2
>> services are complex, its hard to mock them using available tools. We can
>> still do this bit will involve some re-factoring to the code.
>>
>> As a potential solution to improve the tests, I tried to use the
>> integration test mechanism and adapt it to test simple flows as well;
>> deploying and validating various aspects of Service Groups (dependencies,
>> startup orders, etc.), different kinds of policies used in Stratos, etc.
>> Using the integration test mechanism solves the problem of operations not
>> being local and difficulty to mock them; the Stratos server and MB can be
>> used to execute any flow and test it.
>>
>> What I propose is to add a layer on top of the currently integration
>> tests framework to expose a set of utility methods to deploy cartridges,
>> groups, policies, etc. to help developers to write tests using those
>> easily. All the tests will be executed against a single Stratos Server
>> without starting a separate server each time. Did some refactoring locally
>> to suit this purpose and wrote several simple tests as well, which seem to
>> work well.
>>
>> Please share your thoughts.
>>
>> --
>> Thanks and Regards,
>>
>> Isuru H.
>> +94 716 358 048* <http://wso2.com/>*
>>
>>
>>
>
>
> --
> Imesh Gunaratne
>
> Senior Technical Lead, WSO2
> Committer & PMC Member, Apache Stratos
>
> --
> Thanks and Regards,
>
> Isuru H.
> +94 716 358 048* <http://wso2.com/>*
>
>
> * <http://wso2.com/>*
>
>
>
Re: [Discuss] Extending Integration Tests to Support Complex Unit Tests
Posted by Imesh Gunaratne <im...@apache.org>.
Hi Isuru,
+1 for improving the Integration Test Framework with utility methods.
Currently we directly invoke the deploy.sh script and it internally deploy
cartridges, cartridge groups, policies, etc. How are we planning to do this
with the new approach? It would be better if you could send the refactored
code.
Thanks
On Mon, May 11, 2015 at 12:12 AM, Isuru Haththotuwa <is...@apache.org>
wrote:
> Hi Devs,
>
> I had a look to see how we can improve the code coverage with unit
> testing. There are a few challenges for this in Stratos:
>
> 1. Operations are not local to a single component - One component
> calls other to do verifications, etc. Example: deploying a service group;
> Autoscaler will call CC to verify the cartridge details
> 2. Services are not simple, hard to mock - Since the exposed axis2
> services are complex, its hard to mock them using available tools. We can
> still do this bit will involve some re-factoring to the code.
>
> As a potential solution to improve the tests, I tried to use the
> integration test mechanism and adapt it to test simple flows as well;
> deploying and validating various aspects of Service Groups (dependencies,
> startup orders, etc.), different kinds of policies used in Stratos, etc.
> Using the integration test mechanism solves the problem of operations not
> being local and difficulty to mock them; the Stratos server and MB can be
> used to execute any flow and test it.
>
> What I propose is to add a layer on top of the currently integration tests
> framework to expose a set of utility methods to deploy cartridges, groups,
> policies, etc. to help developers to write tests using those easily. All
> the tests will be executed against a single Stratos Server without starting
> a separate server each time. Did some refactoring locally to suit this
> purpose and wrote several simple tests as well, which seem to work well.
>
> Please share your thoughts.
>
> --
> Thanks and Regards,
>
> Isuru H.
> +94 716 358 048* <http://wso2.com/>*
>
>
>
--
Imesh Gunaratne
Senior Technical Lead, WSO2
Committer & PMC Member, Apache Stratos