You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@airavata.apache.org by Amila Jayasekara <th...@gmail.com> on 2013/11/21 22:12:43 UTC

Airavata Release Testing

Hi Raman,

Thank you for initiating this [1]. This list [1] is great.

I have few comments related to verifying samples.

It will be really great if we can automate most of these testing. Ideally
we should only do UI testing manually (Maybe we can automate that too).
Further we need to automate real scenarios (like running job against
trestles/stampede)

I see automation is necessary due to following reasons;

1. Humans are not perfect
2. We have lot of things in our minds so we are always eager to finish one
thing and go to next. Therefore high chance we miss something. Further we
dont have a way to assure that release manager did all the verifications.
3. We dont need to wait till release time to figure out something is broken.
4. We are a small team is small. Therefore we need to automate as much as
possible. Infact that is the way most of the apache projects do.

[1]
https://cwiki.apache.org/confluence/display/AIRAVATA/Airavata+Release+Testing

Thanks
Amila

Re: Airavata Release Testing

Posted by Raminder Singh <ra...@gmail.com>.
I agree, we need to work on automating testing for most of scenarios. My idea was to create a checklist and then think about automation.
 
My few thoughts while compiling the list.

1. We need to create more API samples to demonstrate API capabilities and those samples can help us automate testing. (current samples need updates)
2. Creating API documentation for every capability can help us to improve API and to add required integration tests. I found swagger [1] to be very use for REST API documentation. We can find similar options for Thrift and Java API. 

Thanks
Raminder

1. http://swagger.wordnik.com/

On Nov 22, 2013, at 9:18 AM, Marlon Pierce <ma...@iu.edu> wrote:

> This is why I set a bounty on integration tests. 
> 
> 
> Marlon
> 
> On 11/21/13 4:20 PM, Suresh Marru wrote:
>> One more reason for automated testing:
>> * Humans are lazy by nature :)
>> 
>> + 1 for everything. 
>> 
>> Suresh
>> 
>> On Nov 21, 2013, at 2:12 PM, Amila Jayasekara <th...@gmail.com> wrote:
>> 
>>> Hi Raman,
>>> 
>>> Thank you for initiating this [1]. This list [1] is great.
>>> 
>>> I have few comments related to verifying samples.
>>> 
>>> It will be really great if we can automate most of these testing. Ideally we should only do UI testing manually (Maybe we can automate that too). Further we need to automate real scenarios (like running job against trestles/stampede)
>>> 
>>> I see automation is necessary due to following reasons;
>>> 
>>> 1. Humans are not perfect
>>> 2. We have lot of things in our minds so we are always eager to finish one thing and go to next. Therefore high chance we miss something. Further we dont have a way to assure that release manager did all the verifications.
>>> 3. We dont need to wait till release time to figure out something is broken.
>>> 4. We are a small team is small. Therefore we need to automate as much as possible. Infact that is the way most of the apache projects do.
>>> 
>>> [1] https://cwiki.apache.org/confluence/display/AIRAVATA/Airavata+Release+Testing
>>> 
>>> Thanks
>>> Amila
>

Re: Airavata Release Testing

Posted by Marlon Pierce <ma...@iu.edu>.
This is why I set a bounty on integration tests. 


Marlon

On 11/21/13 4:20 PM, Suresh Marru wrote:
> One more reason for automated testing:
> * Humans are lazy by nature :)
>
> + 1 for everything. 
>
> Suresh
>
> On Nov 21, 2013, at 2:12 PM, Amila Jayasekara <th...@gmail.com> wrote:
>
>> Hi Raman,
>>
>> Thank you for initiating this [1]. This list [1] is great.
>>  
>> I have few comments related to verifying samples.
>>
>> It will be really great if we can automate most of these testing. Ideally we should only do UI testing manually (Maybe we can automate that too). Further we need to automate real scenarios (like running job against trestles/stampede)
>>
>> I see automation is necessary due to following reasons;
>>
>> 1. Humans are not perfect
>> 2. We have lot of things in our minds so we are always eager to finish one thing and go to next. Therefore high chance we miss something. Further we dont have a way to assure that release manager did all the verifications.
>> 3. We dont need to wait till release time to figure out something is broken.
>> 4. We are a small team is small. Therefore we need to automate as much as possible. Infact that is the way most of the apache projects do.
>>
>> [1] https://cwiki.apache.org/confluence/display/AIRAVATA/Airavata+Release+Testing
>>
>> Thanks
>> Amila

Re: Airavata Release Testing

Posted by Suresh Marru <sm...@apache.org>.
One more reason for automated testing:
* Humans are lazy by nature :)

+ 1 for everything. 

Suresh

On Nov 21, 2013, at 2:12 PM, Amila Jayasekara <th...@gmail.com> wrote:

> Hi Raman,
> 
> Thank you for initiating this [1]. This list [1] is great.
>  
> I have few comments related to verifying samples.
> 
> It will be really great if we can automate most of these testing. Ideally we should only do UI testing manually (Maybe we can automate that too). Further we need to automate real scenarios (like running job against trestles/stampede)
> 
> I see automation is necessary due to following reasons;
> 
> 1. Humans are not perfect
> 2. We have lot of things in our minds so we are always eager to finish one thing and go to next. Therefore high chance we miss something. Further we dont have a way to assure that release manager did all the verifications.
> 3. We dont need to wait till release time to figure out something is broken.
> 4. We are a small team is small. Therefore we need to automate as much as possible. Infact that is the way most of the apache projects do.
> 
> [1] https://cwiki.apache.org/confluence/display/AIRAVATA/Airavata+Release+Testing
> 
> Thanks
> Amila