You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@stratos.apache.org by Gayan Gunarathne <ga...@wso2.com> on 2015/11/25 10:50:30 UTC
Integration test timeout issue
With the release build I found that IaasProviderAttributeTestCase failed
due to the time-out[1]. Seems this is intermittent. I have increased the
timeout of that test case by introducing separate timeout value for that
test case.I will trigger another build with that.
[1]
[INFO]
testIaasProviderAttributes(org.apache.stratos.integration.tests.iaas.IaasProviderAttributeTestCase)
Time elapsed: 85.196 sec <<< FAILURE!
[INFO] org.testng.internal.thread.ThreadTimeoutException: Method
org.testng.internal.TestNGMethod.testIaasProviderAttributes() didn't finish
within the time-out 300000
[INFO] at java.lang.Thread.sleep(Native Method)
[INFO] at
org.apache.stratos.integration.common.TopologyHandler.assertApplicationStatus(TopologyHandler.java:240)
[INFO] at
org.apache.stratos.integration.tests.iaas.IaasProviderAttributeTestCase.testIaasProviderAttributes(IaasProviderAttributeTestCase.java:133)
[INFO]
[INFO]
[INFO] Results :
[INFO]
[INFO] Failed tests:
[INFO] IaasProviderAttributeTestCase.testIaasProviderAttributes:133 »
ThreadTimeout M...
Thanks,
Gayan
--
Gayan Gunarathne
Technical Lead, WSO2 Inc. (http://wso2.com)
Committer & PMC Member, Apache Stratos
email : gayang@wso2.com | mobile : +94 775030545 <%2B94%20766819985>
Re: Integration test timeout issue
Posted by Gayan Gunarathne <ga...@wso2.com>.
Hi Raj,
Are you going to remove the groups?That what we had earlier before we
introduce the test groups.But we should be able to run other test cases
except server restart test in parallel as we have unique artifacts ids.I
think if we had the correct dependency grouping it should work.
Anyway I didn't get that test failure once I increase the timeout. But
seems test failures can be occurred if server restart test occur with other
test cases.
Thanks,
Gayan
On Wed, Nov 25, 2015 at 7:40 PM, Rajkumar Rajaratnam <ra...@wso2.com>
wrote:
> Hi Isuru,
>
> I have tried listing all the classes in testng.xml, but the listed order
> is not preserved in execution. I am further investigating it. If we get it
> to work, we can follow this approach, IMO. This is a one time task, there
> is no overhead. We just need to list our new test classes in testng.xml.
>
> Thanks,
> Raj.
>
> On Wed, Nov 25, 2015 at 6:18 PM, Rajkumar Rajaratnam <ra...@wso2.com>
> wrote:
>
>> Let me have a look at it. We didn't get the timeout issue locally, it
>> only happens in jenkins builder.
>>
>> @Akila, +1 to add other groups as dependencies in server restart
>> testcase.
>>
>> Thanks,
>> Raj.
>>
>> On Wed, Nov 25, 2015 at 4:52 PM, Isuru Haththotuwa <is...@apache.org>
>> wrote:
>>
>>> I see that we have upgraded TestNG version [1]. Do we still have the
>>> issue where the tests are run in parallel, even if we specify not to?
>>>
>>> [1]. https://issues.apache.org/jira/browse/STRATOS-1627
>>>
>>> On Wed, Nov 25, 2015 at 4:23 PM, Akila Ravihansa Perera <
>>> ravihansa@wso2.com> wrote:
>>>
>>>> Hi Gayan,
>>>>
>>>> I don't think timeout value is the root cause here. It already has a
>>>> timeout value of 300000 (5mins). Can you check the
>>>> stratos-integration.log and see whether Stratos restart test case was
>>>> executed in middle of this test case?
>>>>
>>>> I noticed that Stratos restart test case only depends on
>>>> "stratos.application.deployment" test group. But there are bunch of other
>>>> test cases in the suite (including IaasProviderAttributeTestCase) that
>>>> belongs to different test groups. We might need to add dependencies to
>>>> those as well. I'm working on this now.
>>>>
>>> Without defining dependencies between test cases, if we specifically
>>> list the classes in the test-suite-all.xml, it should run the test
>>> sequentially AFAIU. Currently we have only specified the package.
>>>
>>>>
>>>> Thanks.
>>>>
>>>> On Wed, Nov 25, 2015 at 3:20 PM, Gayan Gunarathne <ga...@wso2.com>
>>>> wrote:
>>>>
>>>>>
>>>>> With the release build I found that IaasProviderAttributeTestCase
>>>>> failed due to the time-out[1]. Seems this is intermittent. I have increased
>>>>> the timeout of that test case by introducing separate timeout value for
>>>>> that test case.I will trigger another build with that.
>>>>>
>>>>>
>>>>> [1]
>>>>> [INFO]
>>>>> testIaasProviderAttributes(org.apache.stratos.integration.tests.iaas.IaasProviderAttributeTestCase)
>>>>> Time elapsed: 85.196 sec <<< FAILURE!
>>>>> [INFO] org.testng.internal.thread.ThreadTimeoutException: Method
>>>>> org.testng.internal.TestNGMethod.testIaasProviderAttributes() didn't finish
>>>>> within the time-out 300000
>>>>> [INFO] at java.lang.Thread.sleep(Native Method)
>>>>> [INFO] at
>>>>> org.apache.stratos.integration.common.TopologyHandler.assertApplicationStatus(TopologyHandler.java:240)
>>>>> [INFO] at
>>>>> org.apache.stratos.integration.tests.iaas.IaasProviderAttributeTestCase.testIaasProviderAttributes(IaasProviderAttributeTestCase.java:133)
>>>>> [INFO]
>>>>> [INFO]
>>>>> [INFO] Results :
>>>>> [INFO]
>>>>> [INFO] Failed tests:
>>>>> [INFO] IaasProviderAttributeTestCase.testIaasProviderAttributes:133
>>>>> » ThreadTimeout M...
>>>>>
>>>>> Thanks,
>>>>> Gayan
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>>
>>>>> Gayan Gunarathne
>>>>> Technical Lead, WSO2 Inc. (http://wso2.com)
>>>>> Committer & PMC Member, Apache Stratos
>>>>> email : gayang@wso2.com | mobile : +94 775030545 <%2B94%20766819985>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Akila Ravihansa Perera
>>>> WSO2 Inc.; http://wso2.com/
>>>>
>>>> Blog:
>>>> http://ravihansa3000.blogspot.com
>>>>
>>>> --
>>>> <http://ravihansa3000.blogspot.com>
>>>> <http://ravihansa3000.blogspot.com>
>>>> Thanks and Regards,
>>>>
>>>> Isuru H.
>>>> <http://ravihansa3000.blogspot.com>
>>>> +94 716 358 048 <http://ravihansa3000.blogspot.com>*
>>>> <http://wso2.com/>*
>>>>
>>>>
>>>> * <http://wso2.com/>*
>>>>
>>>>
>>>>
>>
>>
>> --
>> Rajkumar Rajaratnam
>> Committer & PMC Member, Apache Stratos
>> Software Engineer, WSO2
>>
>> Mobile : +94777568639
>> Blog : rajkumarr.com
>>
>
>
>
> --
> Rajkumar Rajaratnam
> Committer & PMC Member, Apache Stratos
> Software Engineer, WSO2
>
> Mobile : +94777568639
> Blog : rajkumarr.com
>
--
Gayan Gunarathne
Technical Lead, WSO2 Inc. (http://wso2.com)
Committer & PMC Member, Apache Stratos
email : gayang@wso2.com | mobile : +94 775030545 <%2B94%20766819985>
Re: Integration test timeout issue
Posted by Rajkumar Rajaratnam <ra...@wso2.com>.
Hi Isuru,
I have tried listing all the classes in testng.xml, but the listed order is
not preserved in execution. I am further investigating it. If we get it to
work, we can follow this approach, IMO. This is a one time task, there is
no overhead. We just need to list our new test classes in testng.xml.
Thanks,
Raj.
On Wed, Nov 25, 2015 at 6:18 PM, Rajkumar Rajaratnam <ra...@wso2.com>
wrote:
> Let me have a look at it. We didn't get the timeout issue locally, it only
> happens in jenkins builder.
>
> @Akila, +1 to add other groups as dependencies in server restart testcase.
>
> Thanks,
> Raj.
>
> On Wed, Nov 25, 2015 at 4:52 PM, Isuru Haththotuwa <is...@apache.org>
> wrote:
>
>> I see that we have upgraded TestNG version [1]. Do we still have the
>> issue where the tests are run in parallel, even if we specify not to?
>>
>> [1]. https://issues.apache.org/jira/browse/STRATOS-1627
>>
>> On Wed, Nov 25, 2015 at 4:23 PM, Akila Ravihansa Perera <
>> ravihansa@wso2.com> wrote:
>>
>>> Hi Gayan,
>>>
>>> I don't think timeout value is the root cause here. It already has a
>>> timeout value of 300000 (5mins). Can you check the
>>> stratos-integration.log and see whether Stratos restart test case was
>>> executed in middle of this test case?
>>>
>>> I noticed that Stratos restart test case only depends on
>>> "stratos.application.deployment" test group. But there are bunch of other
>>> test cases in the suite (including IaasProviderAttributeTestCase) that
>>> belongs to different test groups. We might need to add dependencies to
>>> those as well. I'm working on this now.
>>>
>> Without defining dependencies between test cases, if we specifically list
>> the classes in the test-suite-all.xml, it should run the test sequentially
>> AFAIU. Currently we have only specified the package.
>>
>>>
>>> Thanks.
>>>
>>> On Wed, Nov 25, 2015 at 3:20 PM, Gayan Gunarathne <ga...@wso2.com>
>>> wrote:
>>>
>>>>
>>>> With the release build I found that IaasProviderAttributeTestCase
>>>> failed due to the time-out[1]. Seems this is intermittent. I have increased
>>>> the timeout of that test case by introducing separate timeout value for
>>>> that test case.I will trigger another build with that.
>>>>
>>>>
>>>> [1]
>>>> [INFO]
>>>> testIaasProviderAttributes(org.apache.stratos.integration.tests.iaas.IaasProviderAttributeTestCase)
>>>> Time elapsed: 85.196 sec <<< FAILURE!
>>>> [INFO] org.testng.internal.thread.ThreadTimeoutException: Method
>>>> org.testng.internal.TestNGMethod.testIaasProviderAttributes() didn't finish
>>>> within the time-out 300000
>>>> [INFO] at java.lang.Thread.sleep(Native Method)
>>>> [INFO] at
>>>> org.apache.stratos.integration.common.TopologyHandler.assertApplicationStatus(TopologyHandler.java:240)
>>>> [INFO] at
>>>> org.apache.stratos.integration.tests.iaas.IaasProviderAttributeTestCase.testIaasProviderAttributes(IaasProviderAttributeTestCase.java:133)
>>>> [INFO]
>>>> [INFO]
>>>> [INFO] Results :
>>>> [INFO]
>>>> [INFO] Failed tests:
>>>> [INFO] IaasProviderAttributeTestCase.testIaasProviderAttributes:133 »
>>>> ThreadTimeout M...
>>>>
>>>> Thanks,
>>>> Gayan
>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> Gayan Gunarathne
>>>> Technical Lead, WSO2 Inc. (http://wso2.com)
>>>> Committer & PMC Member, Apache Stratos
>>>> email : gayang@wso2.com | mobile : +94 775030545 <%2B94%20766819985>
>>>>
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Akila Ravihansa Perera
>>> WSO2 Inc.; http://wso2.com/
>>>
>>> Blog:
>>> http://ravihansa3000.blogspot.com
>>>
>>> --
>>> <http://ravihansa3000.blogspot.com>
>>> <http://ravihansa3000.blogspot.com>
>>> Thanks and Regards,
>>>
>>> Isuru H.
>>> <http://ravihansa3000.blogspot.com>
>>> +94 716 358 048 <http://ravihansa3000.blogspot.com>* <http://wso2.com/>*
>>>
>>>
>>> * <http://wso2.com/>*
>>>
>>>
>>>
>
>
> --
> Rajkumar Rajaratnam
> Committer & PMC Member, Apache Stratos
> Software Engineer, WSO2
>
> Mobile : +94777568639
> Blog : rajkumarr.com
>
--
Rajkumar Rajaratnam
Committer & PMC Member, Apache Stratos
Software Engineer, WSO2
Mobile : +94777568639
Blog : rajkumarr.com
Re: Integration test timeout issue
Posted by Rajkumar Rajaratnam <ra...@wso2.com>.
Let me have a look at it. We didn't get the timeout issue locally, it only
happens in jenkins builder.
@Akila, +1 to add other groups as dependencies in server restart testcase.
Thanks,
Raj.
On Wed, Nov 25, 2015 at 4:52 PM, Isuru Haththotuwa <is...@apache.org>
wrote:
> I see that we have upgraded TestNG version [1]. Do we still have the issue
> where the tests are run in parallel, even if we specify not to?
>
> [1]. https://issues.apache.org/jira/browse/STRATOS-1627
>
> On Wed, Nov 25, 2015 at 4:23 PM, Akila Ravihansa Perera <
> ravihansa@wso2.com> wrote:
>
>> Hi Gayan,
>>
>> I don't think timeout value is the root cause here. It already has a
>> timeout value of 300000 (5mins). Can you check the
>> stratos-integration.log and see whether Stratos restart test case was
>> executed in middle of this test case?
>>
>> I noticed that Stratos restart test case only depends on
>> "stratos.application.deployment" test group. But there are bunch of other
>> test cases in the suite (including IaasProviderAttributeTestCase) that
>> belongs to different test groups. We might need to add dependencies to
>> those as well. I'm working on this now.
>>
> Without defining dependencies between test cases, if we specifically list
> the classes in the test-suite-all.xml, it should run the test sequentially
> AFAIU. Currently we have only specified the package.
>
>>
>> Thanks.
>>
>> On Wed, Nov 25, 2015 at 3:20 PM, Gayan Gunarathne <ga...@wso2.com>
>> wrote:
>>
>>>
>>> With the release build I found that IaasProviderAttributeTestCase failed
>>> due to the time-out[1]. Seems this is intermittent. I have increased the
>>> timeout of that test case by introducing separate timeout value for that
>>> test case.I will trigger another build with that.
>>>
>>>
>>> [1]
>>> [INFO]
>>> testIaasProviderAttributes(org.apache.stratos.integration.tests.iaas.IaasProviderAttributeTestCase)
>>> Time elapsed: 85.196 sec <<< FAILURE!
>>> [INFO] org.testng.internal.thread.ThreadTimeoutException: Method
>>> org.testng.internal.TestNGMethod.testIaasProviderAttributes() didn't finish
>>> within the time-out 300000
>>> [INFO] at java.lang.Thread.sleep(Native Method)
>>> [INFO] at
>>> org.apache.stratos.integration.common.TopologyHandler.assertApplicationStatus(TopologyHandler.java:240)
>>> [INFO] at
>>> org.apache.stratos.integration.tests.iaas.IaasProviderAttributeTestCase.testIaasProviderAttributes(IaasProviderAttributeTestCase.java:133)
>>> [INFO]
>>> [INFO]
>>> [INFO] Results :
>>> [INFO]
>>> [INFO] Failed tests:
>>> [INFO] IaasProviderAttributeTestCase.testIaasProviderAttributes:133 »
>>> ThreadTimeout M...
>>>
>>> Thanks,
>>> Gayan
>>>
>>>
>>>
>>> --
>>>
>>> Gayan Gunarathne
>>> Technical Lead, WSO2 Inc. (http://wso2.com)
>>> Committer & PMC Member, Apache Stratos
>>> email : gayang@wso2.com | mobile : +94 775030545 <%2B94%20766819985>
>>>
>>>
>>>
>>
>>
>>
>> --
>> Akila Ravihansa Perera
>> WSO2 Inc.; http://wso2.com/
>>
>> Blog:
>> http://ravihansa3000.blogspot.com
>>
>> --
>> <http://ravihansa3000.blogspot.com>
>> <http://ravihansa3000.blogspot.com>
>> Thanks and Regards,
>>
>> Isuru H.
>> <http://ravihansa3000.blogspot.com>
>> +94 716 358 048 <http://ravihansa3000.blogspot.com>* <http://wso2.com/>*
>>
>>
>> * <http://wso2.com/>*
>>
>>
>>
--
Rajkumar Rajaratnam
Committer & PMC Member, Apache Stratos
Software Engineer, WSO2
Mobile : +94777568639
Blog : rajkumarr.com
Re: Integration test timeout issue
Posted by Isuru Haththotuwa <is...@apache.org>.
I see that we have upgraded TestNG version [1]. Do we still have the issue
where the tests are run in parallel, even if we specify not to?
[1]. https://issues.apache.org/jira/browse/STRATOS-1627
On Wed, Nov 25, 2015 at 4:23 PM, Akila Ravihansa Perera <ra...@wso2.com>
wrote:
> Hi Gayan,
>
> I don't think timeout value is the root cause here. It already has a
> timeout value of 300000 (5mins). Can you check the
> stratos-integration.log and see whether Stratos restart test case was
> executed in middle of this test case?
>
> I noticed that Stratos restart test case only depends on
> "stratos.application.deployment" test group. But there are bunch of other
> test cases in the suite (including IaasProviderAttributeTestCase) that
> belongs to different test groups. We might need to add dependencies to
> those as well. I'm working on this now.
>
Without defining dependencies between test cases, if we specifically list
the classes in the test-suite-all.xml, it should run the test sequentially
AFAIU. Currently we have only specified the package.
>
> Thanks.
>
> On Wed, Nov 25, 2015 at 3:20 PM, Gayan Gunarathne <ga...@wso2.com> wrote:
>
>>
>> With the release build I found that IaasProviderAttributeTestCase failed
>> due to the time-out[1]. Seems this is intermittent. I have increased the
>> timeout of that test case by introducing separate timeout value for that
>> test case.I will trigger another build with that.
>>
>>
>> [1]
>> [INFO]
>> testIaasProviderAttributes(org.apache.stratos.integration.tests.iaas.IaasProviderAttributeTestCase)
>> Time elapsed: 85.196 sec <<< FAILURE!
>> [INFO] org.testng.internal.thread.ThreadTimeoutException: Method
>> org.testng.internal.TestNGMethod.testIaasProviderAttributes() didn't finish
>> within the time-out 300000
>> [INFO] at java.lang.Thread.sleep(Native Method)
>> [INFO] at
>> org.apache.stratos.integration.common.TopologyHandler.assertApplicationStatus(TopologyHandler.java:240)
>> [INFO] at
>> org.apache.stratos.integration.tests.iaas.IaasProviderAttributeTestCase.testIaasProviderAttributes(IaasProviderAttributeTestCase.java:133)
>> [INFO]
>> [INFO]
>> [INFO] Results :
>> [INFO]
>> [INFO] Failed tests:
>> [INFO] IaasProviderAttributeTestCase.testIaasProviderAttributes:133 »
>> ThreadTimeout M...
>>
>> Thanks,
>> Gayan
>>
>>
>>
>> --
>>
>> Gayan Gunarathne
>> Technical Lead, WSO2 Inc. (http://wso2.com)
>> Committer & PMC Member, Apache Stratos
>> email : gayang@wso2.com | mobile : +94 775030545 <%2B94%20766819985>
>>
>>
>>
>
>
>
> --
> Akila Ravihansa Perera
> WSO2 Inc.; http://wso2.com/
>
> Blog: http://ravihansa3000.blogspot.com
>
> --
> <http://ravihansa3000.blogspot.com>
> <http://ravihansa3000.blogspot.com>
> Thanks and Regards,
>
> Isuru H.
> <http://ravihansa3000.blogspot.com>
> +94 716 358 048 <http://ravihansa3000.blogspot.com>* <http://wso2.com/>*
>
>
> * <http://wso2.com/>*
>
>
>
Re: Integration test timeout issue
Posted by Akila Ravihansa Perera <ra...@wso2.com>.
Hi Gayan,
I don't think timeout value is the root cause here. It already has a
timeout value of 300000 (5mins). Can you check the stratos-integration.log
and see whether Stratos restart test case was executed in middle of this
test case?
I noticed that Stratos restart test case only depends on
"stratos.application.deployment" test group. But there are bunch of other
test cases in the suite (including IaasProviderAttributeTestCase) that
belongs to different test groups. We might need to add dependencies to
those as well. I'm working on this now.
Thanks.
On Wed, Nov 25, 2015 at 3:20 PM, Gayan Gunarathne <ga...@wso2.com> wrote:
>
> With the release build I found that IaasProviderAttributeTestCase failed
> due to the time-out[1]. Seems this is intermittent. I have increased the
> timeout of that test case by introducing separate timeout value for that
> test case.I will trigger another build with that.
>
>
> [1]
> [INFO]
> testIaasProviderAttributes(org.apache.stratos.integration.tests.iaas.IaasProviderAttributeTestCase)
> Time elapsed: 85.196 sec <<< FAILURE!
> [INFO] org.testng.internal.thread.ThreadTimeoutException: Method
> org.testng.internal.TestNGMethod.testIaasProviderAttributes() didn't finish
> within the time-out 300000
> [INFO] at java.lang.Thread.sleep(Native Method)
> [INFO] at
> org.apache.stratos.integration.common.TopologyHandler.assertApplicationStatus(TopologyHandler.java:240)
> [INFO] at
> org.apache.stratos.integration.tests.iaas.IaasProviderAttributeTestCase.testIaasProviderAttributes(IaasProviderAttributeTestCase.java:133)
> [INFO]
> [INFO]
> [INFO] Results :
> [INFO]
> [INFO] Failed tests:
> [INFO] IaasProviderAttributeTestCase.testIaasProviderAttributes:133 »
> ThreadTimeout M...
>
> Thanks,
> Gayan
>
>
>
> --
>
> Gayan Gunarathne
> Technical Lead, WSO2 Inc. (http://wso2.com)
> Committer & PMC Member, Apache Stratos
> email : gayang@wso2.com | mobile : +94 775030545 <%2B94%20766819985>
>
>
>
--
Akila Ravihansa Perera
WSO2 Inc.; http://wso2.com/
Blog: http://ravihansa3000.blogspot.com