You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@jclouds.apache.org by Adrian Cole <no...@github.com> on 2014/11/17 04:18:45 UTC
[jclouds] Remove OS support tests from TemplateBuilderLiveTest, as
people are ignoring failures (#603)
Developers don't seem to be running live tests much anymore. Prime example is [even amazon ec2 is now rusting](https://issues.apache.org/jira/browse/JCLOUDS-775)!
Part of this is likely that they fail for trivial reasons due to a somewhat obsessive test approach I introduced many years ago. Basically, we defined the "supported operating system" list in subtypes of BaseTemplateBuilderLiveTest. We once thought that we would test multiple os families, but we've never done that. In the mean time, the supported operating system part of the test is just something that breaks every 3 months when a provider adds or removes an OS version. Since there's no action we can take on that, and this drowns out real issues in the provider, let's remove it.
What we retain are more important aspects, such as properties apply, or that the default template resolves to known values. Since all our tests work off the resolved template, we do need to be aware when that changes.
https://issues.apache.org/jira/browse/JCLOUDS-778
You can merge this Pull Request by running:
git pull https://github.com/adriancole/jclouds adrian.less-live-work
Or you can view, comment on it, or merge it online at:
https://github.com/jclouds/jclouds/pull/603
-- Commit Summary --
* JCLOUDS-778 Remove OS support tests from TemplateBuilderLiveTest, as people are ignoring failures,
-- File Changes --
M compute/src/test/java/org/jclouds/compute/StubTemplateBuilderIntegrationTest.java (2)
M compute/src/test/java/org/jclouds/compute/internal/BaseTemplateBuilderLiveTest.java (79)
M providers/aws-ec2/src/test/java/org/jclouds/aws/ec2/compute/AWSEC2TemplateBuilderLiveTest.java (32)
M providers/cloudservers-uk/src/test/java/org/jclouds/rackspace/cloudservers/compute/CloudServersUKTemplateBuilderLiveTest.java (29)
M providers/cloudservers-us/src/test/java/org/jclouds/rackspace/cloudservers/compute/CloudServersUSTemplateBuilderLiveTest.java (31)
M providers/elastichosts-ams-e/src/test/java/org/jclouds/elastichosts/compute/ElasticHostsAmsterdamTemplateBuilderLiveTest.java (29)
M providers/elastichosts-hkg-e/src/test/java/org/jclouds/elastichosts/compute/ElasticHostsHongKongTemplateBuilderLiveTest.java (31)
M providers/elastichosts-lax-p/src/test/java/org/jclouds/elastichosts/compute/ElasticHostsPeer1LosAngelesTemplateBuilderLiveTest.java (31)
M providers/elastichosts-lon-b/src/test/java/org/jclouds/elastichosts/compute/ElasticHostsBlueSquareLondonTemplateBuilderLiveTest.java (29)
M providers/elastichosts-lon-p/src/test/java/org/jclouds/elastichosts/compute/ElasticHostsPeer1LondonTemplateBuilderLiveTest.java (31)
M providers/elastichosts-sat-p/src/test/java/org/jclouds/elastichosts/compute/ElasticHostsPeer1SanAntonioTemplateBuilderLiveTest.java (31)
M providers/elastichosts-sjc-c/src/test/java/org/jclouds/elastichosts/compute/ElasticHostsSanJoseTemplateBuilderLiveTest.java (29)
M providers/elastichosts-syd-v/src/test/java/org/jclouds/elastichosts/compute/ElasticHostsSydneyTemplateBuilderLiveTest.java (29)
M providers/elastichosts-tor-p/src/test/java/org/jclouds/elastichosts/compute/ElasticHostsPeer1TorontoTemplateBuilderLiveTest.java (29)
M providers/glesys/src/test/java/org/jclouds/glesys/compute/GleSYSTemplateBuilderLiveTest.java (37)
M providers/go2cloud-jhb1/src/test/java/org/jclouds/go2cloud/compute/Go2CloudJohannesburg1TemplateBuilderLiveTest.java (26)
M providers/gogrid/src/test/java/org/jclouds/gogrid/compute/GoGridTemplateBuilderLiveTest.java (28)
M providers/hpcloud-compute/src/test/java/org/jclouds/hpcloud/compute/compute/HPCloudComputeTemplateBuilderLiveTest.java (28)
M providers/openhosting-east1/src/test/java/org/jclouds/openhosting/compute/OpenHostingEast1TemplateBuilderLiveTest.java (28)
M providers/rackspace-cloudservers-uk/src/test/java/org/jclouds/rackspace/cloudservers/uk/compute/CloudServersUKTemplateBuilderLiveTest.java (28)
M providers/rackspace-cloudservers-us/src/test/java/org/jclouds/rackspace/cloudservers/us/compute/CloudServersUSTemplateBuilderLiveTest.java (30)
M providers/serverlove-z1-man/src/test/java/org/jclouds/serverlove/compute/ServerloveManchesterTemplateBuilderLiveTest.java (28)
M providers/skalicloud-sdg-my/src/test/java/org/jclouds/skalicloud/compute/SkaliCloudMalaysiaTemplateBuilderLiveTest.java (33)
M providers/softlayer/src/test/java/org/jclouds/softlayer/compute/SoftLayerTemplateBuilderLiveTest.java (58)
-- Patch Links --
https://github.com/jclouds/jclouds/pull/603.patch
https://github.com/jclouds/jclouds/pull/603.diff
---
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds/pull/603
Re: [jclouds] Remove OS support tests from TemplateBuilderLiveTest,
as people are ignoring failures (#603)
Posted by CloudBees pull request builder plugin <no...@github.com>.
[jclouds-pull-requests #1387](https://jclouds.ci.cloudbees.com/job/jclouds-pull-requests/1387/) SUCCESS
This pull request looks good
---
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds/pull/603#issuecomment-63256651
Re: [jclouds] Remove OS support tests from TemplateBuilderLiveTest,
as people are ignoring failures (#603)
Posted by BuildHive <no...@github.com>.
[jclouds ยป jclouds #1918](https://buildhive.cloudbees.com/job/jclouds/job/jclouds/1918/) SUCCESS
This pull request looks good
[(what's this?)](https://www.cloudbees.com/what-is-buildhive)
---
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds/pull/603#issuecomment-63264896
Re: [jclouds] Remove OS support tests from TemplateBuilderLiveTest,
as people are ignoring failures (#603)
Posted by CloudBees pull request builder plugin <no...@github.com>.
[jclouds-pull-requests-java-7 #298](https://jclouds.ci.cloudbees.com/job/jclouds-pull-requests-java-7/298/) SUCCESS
This pull request looks good
---
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds/pull/603#issuecomment-63256246
Re: [jclouds] Remove OS support tests from TemplateBuilderLiveTest,
as people are ignoring failures (#603)
Posted by Adrian Cole <no...@github.com>.
If you look at AWSEC2TemplateBuilderLiveTest, you'll see sometimes we are
testing cloud-specific templates, not just the default ones. These tests
will be retained.
This change adds focus by removing the confusing concept of supported
operating systems which we were never able to back up. Also, it removes
maintenance.
The cost of removing is that image parsers may have glitches that go
unnoticed. Ex. The code accidentally represents an Imagr with a naming
convention like foo-linux-ubuntu-12.04 as linux and not getting the version
right. However, this sort of thing is better handled in mock tests anyway,
as delaying parse problems until live tests may mean they go unnoticed.
Bottom-line, removing this high maintenance and runtime are of live tests
has a small impact on quality, if they were run. Since they haven't been,
increasing the signal to noise and reducing runtime makes this a more
approachable test, which we can hopefully start paying attention to again.
---
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds/pull/603#issuecomment-63338528
Re: [jclouds] Remove OS support tests from TemplateBuilderLiveTest,
as people are ignoring failures (#603)
Posted by Adrian Cole <no...@github.com>.
merged to master, 1.8.x
---
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds/pull/603#issuecomment-63343724
Re: [jclouds] Remove OS support tests from TemplateBuilderLiveTest,
as people are ignoring failures (#603)
Posted by Andrew Phillips <no...@github.com>.
Just to be clear that the strategy moving forward is to define a single (?) target template for live tests in the test properties.
Here, we're dropping some tests that try to find a match for a whole range of templates that were never actually used in tests in practice?
---
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds/pull/603#issuecomment-63335475
Re: [jclouds] Remove OS support tests from TemplateBuilderLiveTest,
as people are ignoring failures (#603)
Posted by Andrew Phillips <no...@github.com>.
> However, this sort of thing is better handled in mock tests anyway, as delaying parse problems until
> live tests may mean they go unnoticed.
Especially if, indeed, the live tests are hardly ever run.
Thanks for the explanation. +1 from me!
---
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds/pull/603#issuecomment-63355478