You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@rave.apache.org by Marlon Pierce <ma...@iu.edu> on 2012/04/06 20:53:18 UTC
Quality assurance steps for Rave
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
I'd like to propose the following.
* Develop a list of Rave features that should be tested, put this on the wiki, and update it every time a new feature is added. I'm happy to get this started. Recommendations for tools to automate some of this are welcome.
* Regular testing of the above feature list (not just in preparation for releases). We would need a record of this (who tested, when, etc). If we do this manually, we could keep records by updating the wiki, posting to the dev list, etc. Recommendations for better tools are of course welcome.
* All features should be tested before release by at least one person. The Release Manager should coordinate this. We could continue to do this as part of the current Release Candidate process, but it may be better to have an intermediate SVN tag that can go through QA before we start an official vote. This would allow us to keep the trunk open for commits while we test and avoid canceled releases.
I'm not a QA expert, so comments welcome.
Marlon
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.16 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iQEcBAEBAgAGBQJPfzueAAoJEEfVXEODPFIDPb0IAJT5Op6iaQbKgO5Jp9wBjsCp
oC2Jxk5HrGolbavE8yi0LDAGv8x52mo1U8+m7PfRT6yhQ38ciMMkm8I7W2BYZ2Ei
WWX1kerVriICKV4LSfZ3u503OJ77WYlhoOsJwvvZEvxYndKuhroYyxDIZ0PdTsGI
GRp7Iu4r9My4AOz+bIh4EaAWE+VKHhS9INoHvfsm6Fb8vxAyvSNfklgtOiZoTSlg
g4BxvEPlkxs5Xv3jTFaX/rgGWoKU31LkL8mSLIeW64BWXBqXMscZSTSW/l5Brx/8
dJLnuHUmmSJLjKhzELNaDH68Rntb1WFV+oWsWf8xBqawYKXkVElqNywTh5yOxb8=
=1anU
-----END PGP SIGNATURE-----
Re: Quality assurance steps for Rave
Posted by Jasha Joachimsthal <j....@onehippo.com>.
On 13 April 2012 13:52, Franklin, Matthew B. <mf...@mitre.org> wrote:
> >-----Original Message-----
> >From: Jasha Joachimsthal [mailto:j.joachimsthal@onehippo.com]
> >Sent: Friday, April 13, 2012 7:43 AM
> >To: dev@rave.apache.org
> >Subject: Re: Quality assurance steps for Rave
> >
> >Jasha Joachimsthal
> >
> >Europe - Amsterdam - Oosteinde 11, 1017 WT Amsterdam - +31(0)20 522 4466
> >US - Boston - 1 Broadway, Cambridge, MA 02142 - +1 877 414 4776 (toll
> free)
> >
> >www.onehippo.com
> >
> >
> >On 12 April 2012 03:59, Franklin, Matthew B. <mf...@mitre.org> wrote:
> >
> >> >-----Original Message-----
> >> >From: Marlon Pierce [mailto:marpierc@iu.edu]
> >> >Sent: Tuesday, April 10, 2012 12:01 PM
> >> >To: dev@rave.apache.org
> >> >Subject: Re: Quality assurance steps for Rave
> >> >
> >> >-----BEGIN PGP SIGNED MESSAGE-----
> >> >Hash: SHA1
> >> >
> >> >I started a checklist of tests at
> >> >http://wiki.apache.org/rave/ReleaseManagement/QualityAssurance, as
> >you
> >> >have seen from the [Rave Wiki] messages.
> >>
> >> Good set of tests. I had been going through most of these on my own
> >> before each release, but that will become impossible to keep up with as
> the
> >> functionality grows.
> >>
> >> >
> >> >* Assuming the user interface tests are done manually, updating the
> wiki
> >> >every time would be cruddy. Is there good test/assurance process
> >> >management software out there? Should we do this with an online
> >(Google)
> >> >spreadsheet?
> >>
> >> I say create a blocking Pre-release test task in Jira and have subtasks
> >> for each "section" of the tests. (we can use copy so it isn't such a
> manual
> >> task)
> >>
> >> >
> >> >* Do we want instead to actively maintain selenium tests? I started
> this
> >> but
> >> >let it drop because these only worked in Firefox, and Firefox's
> frequent
> >> >updates broke the tests. The tests themselves would have to be
> actively
> >> >maintained--if you change or add a user interface feature, you need to
> >> also
> >> >update the test.
> >>
> >> I think if we develop the simplest possible tests to protect against
> >> regression, it could work without too much effort. We need to make it
> an
> >> explicit part of the process though. It has also been my experience
> that
> >> recorded test cases don't work without some modification. I have also
> had
> >> decent success in keeping code-based (not selenese) tests working....
> >>
> >> >
> >> >* Are there other recommendations beside Selenium?
> >>
> >
> >I've just created a test project [1] with JBehave [2] which uses Selenium
> >under the hood.
>
> +1 for JBehave
>
> >
> >Scenario: User creates a new account and logs in into the portal
> >When I go to "http://localhost:8080/portal"
> >Then I see the login page
> >When I follow the new account link
> >Then I get the new account form
> >When I fill in the form with username "newuser" password "password"
> >confirmpassword "password" email "newuser@example.com"
> >And I submit the new account form
> >Then I see the login page
> >And A message appears "Account successfully created"
> >When I fill in the login form with username "newuser" password "password"
> >Then I see my portal page with the add new widgets box
> >
> >
> >you can run it with mvn clean install or run the NewUserStories class from
> >your IDE.
>
> I would think we would want to do this from a different profile, right?
>
We need to think about the right setup of the integration tests. It would
be good if an experienced tester could help with that. I just wanted to
show an option how to test the portal without clicking through it manually.
>
> >It does not clean up the newly created user so you cannot run it twice
> >successfully without cleaning up the user.
> >
> >[1] https://github.com/jashaj/PortalTests
> >[2] http://jbehave.org/
> >
> > >
> >> >
> >> >Marlon
> >> >
> >> >
> >> >On 4/6/12 3:32 PM, Raminderjeet Singh wrote:
> >> >> +1 with Marlon on having a checklist of features. I know going
> forward
> >> this
> >> >list can grow and we may need more people to verify the build.
> >> >>
> >> >> Based on my experience doing 0.10 release, Matt and others have done
> >a
> >> >great job putting all the steps together into scripts. Thanks!
> >> >>
> >> >> After the code freeze announcement, Release manager can tag the
> >current
> >> >code and verification can be done on that particular tag. One developer
> >> >(release manager also) can verify the code tag for the feature list and
> >> if very
> >> >thing looks good can do the release based on the tag. Matt can comment
> >> >more as i have to still understand when the pom versions are updated.
> >> >>
> >> >> Thanks
> >> >> Raminder
> >> >>
> >> >> On Apr 6, 2012, at 2:53 PM, Marlon Pierce wrote:
> >> >>
> >> >> I'd like to propose the following.
> >> >>
> >> >> * Develop a list of Rave features that should be tested, put this on
> >> the wiki,
> >> >and update it every time a new feature is added. I'm happy to get this
> >> >started. Recommendations for tools to automate some of this are
> >welcome.
> >> >>
> >> >>
> >> >> * Regular testing of the above feature list (not just in preparation
> for
> >> >releases). We would need a record of this (who tested, when, etc). If
> >> we do
> >> >this manually, we could keep records by updating the wiki, posting to
> the
> >> dev
> >> >list, etc. Recommendations for better tools are of course welcome.
> >> >>
> >> >>
> >> >> * All features should be tested before release by at least one
> person.
> >> The
> >> >Release Manager should coordinate this. We could continue to do this as
> >> part
> >> >of the current Release Candidate process, but it may be better to have
> an
> >> >intermediate SVN tag that can go through QA before we start an official
> >> vote.
> >> >This would allow us to keep the trunk open for commits while we test
> and
> >> >avoid canceled releases.
> >> >>
> >> >>
> >> >> I'm not a QA expert, so comments welcome.
> >> >>
> >> >>
> >> >>
> >> >> Marlon
> >> >>
> >> >>
> >> >-----BEGIN PGP SIGNATURE-----
> >> >Version: GnuPG/MacGPG2 v2.0.16 (Darwin)
> >> >Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
> >> >
> >>
> >>iQEcBAEBAgAGBQJPhFlJAAoJEEfVXEODPFIDPZ4H/2FM9G6b6mJuJgkong4cA
> >W
> >> >wg
> >>
> >>TMj8Vks5ykOsRspkqIff4Q8ItzkVrrq0hKb+1p2gaaGj8nG6/BhKnh+CaSsuwfhC
> >>
> >>QqeucJdj4/0RreCx+eQ9M0A17Y9O21yZxKuKQaiPHHWbv32NZWHsuycUPDP
> >+
> >> >MmvI
> >>
> >>mC5/w9+N2jQf2O7y6X0fR6ydfyggJrhBbM0PNY3yp6O7dHb91KqUz4Vm5giXl1
> >> >H/
> >>
> >>OzG0NEDv/EweIbTCeEJWOs2oDZGZgkRCT7z0LoJXXWNHifUSSJMuUDwPu0
> >M
> >> >5CIIJ
> >>
> >>tX16D/JdC2ik9FtarCHeQV101uEJYrC3Mx4F9Lb/PO0Hj+ePOBrha9rMRnSOcK0
> >=
> >> >=06BC
> >> >-----END PGP SIGNATURE-----
> >>
>
RE: Quality assurance steps for Rave
Posted by "Franklin, Matthew B." <mf...@mitre.org>.
>-----Original Message-----
>From: Jasha Joachimsthal [mailto:j.joachimsthal@onehippo.com]
>Sent: Friday, April 13, 2012 7:43 AM
>To: dev@rave.apache.org
>Subject: Re: Quality assurance steps for Rave
>
>Jasha Joachimsthal
>
>Europe - Amsterdam - Oosteinde 11, 1017 WT Amsterdam - +31(0)20 522 4466
>US - Boston - 1 Broadway, Cambridge, MA 02142 - +1 877 414 4776 (toll free)
>
>www.onehippo.com
>
>
>On 12 April 2012 03:59, Franklin, Matthew B. <mf...@mitre.org> wrote:
>
>> >-----Original Message-----
>> >From: Marlon Pierce [mailto:marpierc@iu.edu]
>> >Sent: Tuesday, April 10, 2012 12:01 PM
>> >To: dev@rave.apache.org
>> >Subject: Re: Quality assurance steps for Rave
>> >
>> >-----BEGIN PGP SIGNED MESSAGE-----
>> >Hash: SHA1
>> >
>> >I started a checklist of tests at
>> >http://wiki.apache.org/rave/ReleaseManagement/QualityAssurance, as
>you
>> >have seen from the [Rave Wiki] messages.
>>
>> Good set of tests. I had been going through most of these on my own
>> before each release, but that will become impossible to keep up with as the
>> functionality grows.
>>
>> >
>> >* Assuming the user interface tests are done manually, updating the wiki
>> >every time would be cruddy. Is there good test/assurance process
>> >management software out there? Should we do this with an online
>(Google)
>> >spreadsheet?
>>
>> I say create a blocking Pre-release test task in Jira and have subtasks
>> for each "section" of the tests. (we can use copy so it isn't such a manual
>> task)
>>
>> >
>> >* Do we want instead to actively maintain selenium tests? I started this
>> but
>> >let it drop because these only worked in Firefox, and Firefox's frequent
>> >updates broke the tests. The tests themselves would have to be actively
>> >maintained--if you change or add a user interface feature, you need to
>> also
>> >update the test.
>>
>> I think if we develop the simplest possible tests to protect against
>> regression, it could work without too much effort. We need to make it an
>> explicit part of the process though. It has also been my experience that
>> recorded test cases don't work without some modification. I have also had
>> decent success in keeping code-based (not selenese) tests working....
>>
>> >
>> >* Are there other recommendations beside Selenium?
>>
>
>I've just created a test project [1] with JBehave [2] which uses Selenium
>under the hood.
+1 for JBehave
>
>Scenario: User creates a new account and logs in into the portal
>When I go to "http://localhost:8080/portal"
>Then I see the login page
>When I follow the new account link
>Then I get the new account form
>When I fill in the form with username "newuser" password "password"
>confirmpassword "password" email "newuser@example.com"
>And I submit the new account form
>Then I see the login page
>And A message appears "Account successfully created"
>When I fill in the login form with username "newuser" password "password"
>Then I see my portal page with the add new widgets box
>
>
>you can run it with mvn clean install or run the NewUserStories class from
>your IDE.
I would think we would want to do this from a different profile, right?
>It does not clean up the newly created user so you cannot run it twice
>successfully without cleaning up the user.
>
>[1] https://github.com/jashaj/PortalTests
>[2] http://jbehave.org/
>
> >
>> >
>> >Marlon
>> >
>> >
>> >On 4/6/12 3:32 PM, Raminderjeet Singh wrote:
>> >> +1 with Marlon on having a checklist of features. I know going forward
>> this
>> >list can grow and we may need more people to verify the build.
>> >>
>> >> Based on my experience doing 0.10 release, Matt and others have done
>a
>> >great job putting all the steps together into scripts. Thanks!
>> >>
>> >> After the code freeze announcement, Release manager can tag the
>current
>> >code and verification can be done on that particular tag. One developer
>> >(release manager also) can verify the code tag for the feature list and
>> if very
>> >thing looks good can do the release based on the tag. Matt can comment
>> >more as i have to still understand when the pom versions are updated.
>> >>
>> >> Thanks
>> >> Raminder
>> >>
>> >> On Apr 6, 2012, at 2:53 PM, Marlon Pierce wrote:
>> >>
>> >> I'd like to propose the following.
>> >>
>> >> * Develop a list of Rave features that should be tested, put this on
>> the wiki,
>> >and update it every time a new feature is added. I'm happy to get this
>> >started. Recommendations for tools to automate some of this are
>welcome.
>> >>
>> >>
>> >> * Regular testing of the above feature list (not just in preparation for
>> >releases). We would need a record of this (who tested, when, etc). If
>> we do
>> >this manually, we could keep records by updating the wiki, posting to the
>> dev
>> >list, etc. Recommendations for better tools are of course welcome.
>> >>
>> >>
>> >> * All features should be tested before release by at least one person.
>> The
>> >Release Manager should coordinate this. We could continue to do this as
>> part
>> >of the current Release Candidate process, but it may be better to have an
>> >intermediate SVN tag that can go through QA before we start an official
>> vote.
>> >This would allow us to keep the trunk open for commits while we test and
>> >avoid canceled releases.
>> >>
>> >>
>> >> I'm not a QA expert, so comments welcome.
>> >>
>> >>
>> >>
>> >> Marlon
>> >>
>> >>
>> >-----BEGIN PGP SIGNATURE-----
>> >Version: GnuPG/MacGPG2 v2.0.16 (Darwin)
>> >Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>> >
>>
>>iQEcBAEBAgAGBQJPhFlJAAoJEEfVXEODPFIDPZ4H/2FM9G6b6mJuJgkong4cA
>W
>> >wg
>>
>>TMj8Vks5ykOsRspkqIff4Q8ItzkVrrq0hKb+1p2gaaGj8nG6/BhKnh+CaSsuwfhC
>>
>>QqeucJdj4/0RreCx+eQ9M0A17Y9O21yZxKuKQaiPHHWbv32NZWHsuycUPDP
>+
>> >MmvI
>>
>>mC5/w9+N2jQf2O7y6X0fR6ydfyggJrhBbM0PNY3yp6O7dHb91KqUz4Vm5giXl1
>> >H/
>>
>>OzG0NEDv/EweIbTCeEJWOs2oDZGZgkRCT7z0LoJXXWNHifUSSJMuUDwPu0
>M
>> >5CIIJ
>>
>>tX16D/JdC2ik9FtarCHeQV101uEJYrC3Mx4F9Lb/PO0Hj+ePOBrha9rMRnSOcK0
>=
>> >=06BC
>> >-----END PGP SIGNATURE-----
>>
Re: Quality assurance steps for Rave
Posted by Marlon Pierce <ma...@iu.edu>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Great, I will try it out--
Marlon
On 4/13/12 7:43 AM, Jasha Joachimsthal wrote:
> Jasha Joachimsthal
>
> Europe - Amsterdam - Oosteinde 11, 1017 WT Amsterdam - +31(0)20 522 4466
> US - Boston - 1 Broadway, Cambridge, MA 02142 - +1 877 414 4776 (toll free)
>
> www.onehippo.com
>
>
> On 12 April 2012 03:59, Franklin, Matthew B. <mf...@mitre.org> wrote:
>
>>> -----Original Message-----
>>> From: Marlon Pierce [mailto:marpierc@iu.edu]
>>> Sent: Tuesday, April 10, 2012 12:01 PM
>>> To: dev@rave.apache.org
>>> Subject: Re: Quality assurance steps for Rave
>>>
> I started a checklist of tests at
> http://wiki.apache.org/rave/ReleaseManagement/QualityAssurance, as you
> have seen from the [Rave Wiki] messages.
>>>
>>> Good set of tests. I had been going through most of these on my own
>>> before each release, but that will become impossible to keep up with as the
>>> functionality grows.
>>>
>
> * Assuming the user interface tests are done manually, updating the wiki
> every time would be cruddy. Is there good test/assurance process
> management software out there? Should we do this with an online (Google)
> spreadsheet?
>>>
>>> I say create a blocking Pre-release test task in Jira and have subtasks
>>> for each "section" of the tests. (we can use copy so it isn't such a manual
>>> task)
>>>
>
> * Do we want instead to actively maintain selenium tests? I started this
>>> but
> let it drop because these only worked in Firefox, and Firefox's frequent
> updates broke the tests. The tests themselves would have to be actively
> maintained--if you change or add a user interface feature, you need to
>>> also
> update the test.
>>>
>>> I think if we develop the simplest possible tests to protect against
>>> regression, it could work without too much effort. We need to make it an
>>> explicit part of the process though. It has also been my experience that
>>> recorded test cases don't work without some modification. I have also had
>>> decent success in keeping code-based (not selenese) tests working....
>>>
>
> * Are there other recommendations beside Selenium?
>>>
>
>> I've just created a test project [1] with JBehave [2] which uses Selenium
>> under the hood.
>
>> Scenario: User creates a new account and logs in into the portal
>> When I go to "http://localhost:8080/portal"
>> Then I see the login page
>> When I follow the new account link
>> Then I get the new account form
>> When I fill in the form with username "newuser" password "password"
>> confirmpassword "password" email "newuser@example.com"
>> And I submit the new account form
>> Then I see the login page
>> And A message appears "Account successfully created"
>> When I fill in the login form with username "newuser" password "password"
>> Then I see my portal page with the add new widgets box
>
>
>> you can run it with mvn clean install or run the NewUserStories class from
>> your IDE.
>> It does not clean up the newly created user so you cannot run it twice
>> successfully without cleaning up the user.
>
>> [1] https://github.com/jashaj/PortalTests
>> [2] http://jbehave.org/
>
>> >
>
> Marlon
>
>
> On 4/6/12 3:32 PM, Raminderjeet Singh wrote:
>>>>> +1 with Marlon on having a checklist of features. I know going forward
>>> this
> list can grow and we may need more people to verify the build.
>>>>>
>>>>> Based on my experience doing 0.10 release, Matt and others have done a
> great job putting all the steps together into scripts. Thanks!
>>>>>
>>>>> After the code freeze announcement, Release manager can tag the current
> code and verification can be done on that particular tag. One developer
> (release manager also) can verify the code tag for the feature list and
>>> if very
> thing looks good can do the release based on the tag. Matt can comment
> more as i have to still understand when the pom versions are updated.
>>>>>
>>>>> Thanks
>>>>> Raminder
>>>>>
>>>>> On Apr 6, 2012, at 2:53 PM, Marlon Pierce wrote:
>>>>>
>>>>> I'd like to propose the following.
>>>>>
>>>>> * Develop a list of Rave features that should be tested, put this on
>>> the wiki,
> and update it every time a new feature is added. I'm happy to get this
> started. Recommendations for tools to automate some of this are welcome.
>>>>>
>>>>>
>>>>> * Regular testing of the above feature list (not just in preparation for
> releases). We would need a record of this (who tested, when, etc). If
>>> we do
> this manually, we could keep records by updating the wiki, posting to the
>>> dev
> list, etc. Recommendations for better tools are of course welcome.
>>>>>
>>>>>
>>>>> * All features should be tested before release by at least one person.
>>> The
> Release Manager should coordinate this. We could continue to do this as
>>> part
> of the current Release Candidate process, but it may be better to have an
> intermediate SVN tag that can go through QA before we start an official
>>> vote.
> This would allow us to keep the trunk open for commits while we test and
> avoid canceled releases.
>>>>>
>>>>>
>>>>> I'm not a QA expert, so comments welcome.
>>>>>
>>>>>
>>>>>
>>>>> Marlon
>>>>>
>>>>>
>>
>
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.16 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iQEcBAEBAgAGBQJPiBmZAAoJEEfVXEODPFIDLmsIAJhErl0l1Ryyv5RJSCaa6vt9
VvlII/FpaRjYAlAr/NRnvUhgRrY78YSBjKsJqapTOLEbLDqkbBSozo1VPoESPX21
T1ztNQQlhn572PjLSlvhwWHFDWh/esNGZ1Zga8g2lTEo8kfNfPVqZVue3cLi5b1N
lxK+4aOqTR2L57w8xg4q5DmclxwwKR62RkVR8POW+kUF2efp5GyIbqCXy/dAmnrj
9uGU71gUexN5+cRueKvfUUa5FVXLEtJPnuLr9o8HJsLXDxokmqL+8VVZdoDoFbbR
b6RI7KUIfSZVA6gWMXuQrOwRn1r4D1ZZIHMHO+8ZIgW3Xauk2CSYpSPlESdwhGo=
=3M7b
-----END PGP SIGNATURE-----
Re: Quality assurance steps for Rave
Posted by Marlon Pierce <ma...@iu.edu>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Can we put these in Rave as a module executed by an optional profile?
Marlon
On 4/19/12 2:49 AM, Jasha Joachimsthal wrote:
> On 18 April 2012 23:17, Ate Douma <at...@douma.nu> wrote:
>
>> On 04/18/2012 10:33 PM, Marlon Pierce wrote:
>>
> I'm finally looking at Jasha's example in detail, but the tests don't
> work (Firefox doesn't start). The JBehave tutorial (
> https://github.com/jbehave/**jbehave-tutorial<https://github.com/jbehave/jbehave-tutorial>)
> works fine.
>
>
> Any idea what the problem is?
>
>>> Seems like Jasha is developing on a case-insensitive file system :)
>>>
>
>> Okay tricky thing I learned for git: It "works on my machine" because I
>> _had_ renamed the files to lowercase. But in .git/config "ignorecase" was
>> set to "true" so git did not see that as a change that needed to be
>> committed. I changed the setting and now the files are lowercase, so Ate's
>> patch is no longer necessary (where was the pull request ;)).
>
>
>>> If you apply below patch it should run, it then works for me:
>>>
>
>
>
>
>>>
>>> diff --git a/src/main/java/eu/jasha/**portaltests/stories/**NewUserStories.java
>>> b/src/main/java/eu/jasha/**portaltests/stories/**NewUserStories.java
>>> index 14257ce..1c8271d 100644
>>> --- a/src/main/java/eu/jasha/**portaltests/stories/**NewUserStories.java
>>> +++ b/src/main/java/eu/jasha/**portaltests/stories/**NewUserStories.java
>>> @@ -66,13 +66,13 @@ public class NewUserStories extends JUnitStories {
>>> .useStoryReporterBuilder(**reporterBuilder);
>>> useConfiguration(**configuration);
>>>
>>> - ApplicationContext context = new SpringApplicationContextFactor**
>>> y("newuser-steps.xml").**createApplicationContext();
>>> + ApplicationContext context = new SpringApplicationContextFactor**
>>> y("newUser-steps.xml").**createApplicationContext();
>>> useStepsFactory(new SpringStepsFactory(**configuration, context));
>>>
>>> }
>>>
>>> @Override
>>> protected List<String> storyPaths() {
>>> - return new StoryFinder().findPaths(**codeLocationFromClass(this.*
>>> *getClass()).getFile(), asList("**/newuser.story"), null);
>>> + return new StoryFinder().findPaths(**codeLocationFromClass(this.*
>>> *getClass()).getFile(), asList("**/newUser.story"), null);
>>>
>>> }
>>> }
>>>
>>>
>>>
>
> Marlon
>
>
> On 4/13/12 7:43 AM, Jasha Joachimsthal wrote:
>
>>>>> Jasha Joachimsthal
>>>>>
>>>>> Europe - Amsterdam - Oosteinde 11, 1017 WT Amsterdam - +31(0)20 522 4466
>>>>> US - Boston - 1 Broadway, Cambridge, MA 02142 - +1 877 414 4776 (toll
>>>>> free)
>>>>>
>>>>> www.onehippo.com
>>>>>
>>>>>
>>>>> On 12 April 2012 03:59, Franklin, Matthew B.<mf...@mitre.org>
>>>>> wrote:
>>>>>
>>>>> -----Original Message-----
>>>>>>> From: Marlon Pierce [mailto:marpierc@iu.edu]
>>>>>>> Sent: Tuesday, April 10, 2012 12:01 PM
>>>>>>> To: dev@rave.apache.org
>>>>>>> Subject: Re: Quality assurance steps for Rave
>>>>>>>
>>>>>>> I started a checklist of tests at
>>>>> http://wiki.apache.org/rave/**ReleaseManagement/**QualityAssurance<http://wiki.apache.org/rave/ReleaseManagement/QualityAssurance>,
>>>>> as you
>>>>> have seen from the [Rave Wiki] messages.
>>>>>
>>>>>>
>>>>>>> Good set of tests. I had been going through most of these on my own
>>>>>>> before each release, but that will become impossible to keep up with
>>>>>>> as the
>>>>>>> functionality grows.
>>>>>>>
>>>>>>>
>>>>> * Assuming the user interface tests are done manually, updating the wiki
>>>>> every time would be cruddy. Is there good test/assurance process
>>>>> management software out there? Should we do this with an online (Google)
>>>>> spreadsheet?
>>>>>
>>>>>>
>>>>>>> I say create a blocking Pre-release test task in Jira and have subtasks
>>>>>>> for each "section" of the tests. (we can use copy so it isn't such a
>>>>>>> manual
>>>>>>> task)
>>>>>>>
>>>>>>>
>>>>> * Do we want instead to actively maintain selenium tests? I started this
>>>>>
>>>>>> but
>>>>>>>
>>>>>> let it drop because these only worked in Firefox, and Firefox's frequent
>>>>> updates broke the tests. The tests themselves would have to be actively
>>>>> maintained--if you change or add a user interface feature, you need to
>>>>>
>>>>>> also
>>>>>>>
>>>>>> update the test.
>>>>>
>>>>>>
>>>>>>> I think if we develop the simplest possible tests to protect against
>>>>>>> regression, it could work without too much effort. We need to make it
>>>>>>> an
>>>>>>> explicit part of the process though. It has also been my experience
>>>>>>> that
>>>>>>> recorded test cases don't work without some modification. I have also
>>>>>>> had
>>>>>>> decent success in keeping code-based (not selenese) tests working....
>>>>>>>
>>>>>>>
>>>>> * Are there other recommendations beside Selenium?
>>>>>
>>>>>>
>>>>>>>
>>>>> I've just created a test project [1] with JBehave [2] which uses
>>>>>> Selenium
>>>>>> under the hood.
>>>>>>
>>>>>
>>>>> Scenario: User creates a new account and logs in into the portal
>>>>>> When I go to "http://localhost:8080/portal"
>>>>>> Then I see the login page
>>>>>> When I follow the new account link
>>>>>> Then I get the new account form
>>>>>> When I fill in the form with username "newuser" password "password"
>>>>>> confirmpassword "password" email "newuser@example.com"
>>>>>> And I submit the new account form
>>>>>> Then I see the login page
>>>>>> And A message appears "Account successfully created"
>>>>>> When I fill in the login form with username "newuser" password
>>>>>> "password"
>>>>>> Then I see my portal page with the add new widgets box
>>>>>>
>>>>>
>>>>>
>>>>> you can run it with mvn clean install or run the NewUserStories class
>>>>>> from
>>>>>> your IDE.
>>>>>> It does not clean up the newly created user so you cannot run it twice
>>>>>> successfully without cleaning up the user.
>>>>>>
>>>>>
>>>>> [1] https://github.com/jashaj/**PortalTests<https://github.com/jashaj/PortalTests>
>>>>>> [2] http://jbehave.org/
>>>>>>
>>>>>
>>>>> >
>>>>>>
>>>>>
>>>>> Marlon
>>>>>
>>>>>
>>>>> On 4/6/12 3:32 PM, Raminderjeet Singh wrote:
>>>>>
>>>>>> +1 with Marlon on having a checklist of features. I know going forward
>>>>>>>>>
>>>>>>>> this
>>>>>>>
>>>>>> list can grow and we may need more people to verify the build.
>>>>>
>>>>>>
>>>>>>>>> Based on my experience doing 0.10 release, Matt and others have done
>>>>>>>>> a
>>>>>>>>>
>>>>>>>> great job putting all the steps together into scripts. Thanks!
>>>>>
>>>>>>
>>>>>>>>> After the code freeze announcement, Release manager can tag the
>>>>>>>>> current
>>>>>>>>>
>>>>>>>> code and verification can be done on that particular tag. One
>>>>> developer
>>>>> (release manager also) can verify the code tag for the feature list and
>>>>>
>>>>>> if very
>>>>>>>
>>>>>> thing looks good can do the release based on the tag. Matt can comment
>>>>> more as i have to still understand when the pom versions are updated.
>>>>>
>>>>>>
>>>>>>>>> Thanks
>>>>>>>>> Raminder
>>>>>>>>>
>>>>>>>>> On Apr 6, 2012, at 2:53 PM, Marlon Pierce wrote:
>>>>>>>>>
>>>>>>>>> I'd like to propose the following.
>>>>>>>>>
>>>>>>>>> * Develop a list of Rave features that should be tested, put this on
>>>>>>>>>
>>>>>>>> the wiki,
>>>>>>>
>>>>>> and update it every time a new feature is added. I'm happy to get this
>>>>> started. Recommendations for tools to automate some of this are welcome.
>>>>>
>>>>>>
>>>>>>>>>
>>>>>>>>> * Regular testing of the above feature list (not just in preparation
>>>>>>>>> for
>>>>>>>>>
>>>>>>>> releases). We would need a record of this (who tested, when, etc).
>>>>> If
>>>>>
>>>>>> we do
>>>>>>>
>>>>>> this manually, we could keep records by updating the wiki, posting to
>>>>> the
>>>>>
>>>>>> dev
>>>>>>>
>>>>>> list, etc. Recommendations for better tools are of course welcome.
>>>>>
>>>>>>
>>>>>>>>>
>>>>>>>>> * All features should be tested before release by at least one
>>>>>>>>> person.
>>>>>>>>>
>>>>>>>> The
>>>>>>>
>>>>>> Release Manager should coordinate this. We could continue to do this as
>>>>>
>>>>>> part
>>>>>>>
>>>>>> of the current Release Candidate process, but it may be better to have
>>>>> an
>>>>> intermediate SVN tag that can go through QA before we start an official
>>>>>
>>>>>> vote.
>>>>>>>
>>>>>> This would allow us to keep the trunk open for commits while we test and
>>>>> avoid canceled releases.
>>>>>
>>>>>>
>>>>>>>>>
>>>>>>>>> I'm not a QA expert, so comments welcome.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Marlon
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>
>>>>>
>>>
>>
>>
>
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.16 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iQEcBAEBAgAGBQJPkCbBAAoJEEfVXEODPFIDZy4H/0WxFNONvQhDVEkizujqtVSR
aHTJBhZFCcTA0mSuFzMFQcnfREgEFCZzv4sTggL+icLSPQ8jczpZAtq6BQbrrakq
AwdrGQbzdawbdvtFdb669b/t7IJM4lW5dheElyw4V/R4v2TQltY3cBl9btM75zJQ
NlpqJjOmJopRJzLaEz43Xsf+xUUSs6WHL2hXs4hNMPstk9aUlcCAtzpqlpD2KRr7
x+yGeqeTMCgTGxaaZ2gJIGve8KTcx0DQAWbZvbEIbs5duTwPTqQS2kQJTB/FR8tF
KqCFme/2nTVpXoPTa+GLkhPkS/urDsFzXV7N8wtdUDnw3QBZ2yhBUvKrCDI21sQ=
=KDyj
-----END PGP SIGNATURE-----
Re: Quality assurance steps for Rave
Posted by Jasha Joachimsthal <j....@onehippo.com>.
On 18 April 2012 23:17, Ate Douma <at...@douma.nu> wrote:
> On 04/18/2012 10:33 PM, Marlon Pierce wrote:
>
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> I'm finally looking at Jasha's example in detail, but the tests don't
>> work (Firefox doesn't start). The JBehave tutorial (
>> https://github.com/jbehave/**jbehave-tutorial<https://github.com/jbehave/jbehave-tutorial>)
>> works fine.
>>
>>
>> Any idea what the problem is?
>>
> Seems like Jasha is developing on a case-insensitive file system :)
>
Okay tricky thing I learned for git: It "works on my machine" because I
_had_ renamed the files to lowercase. But in .git/config "ignorecase" was
set to "true" so git did not see that as a change that needed to be
committed. I changed the setting and now the files are lowercase, so Ate's
patch is no longer necessary (where was the pull request ;)).
> If you apply below patch it should run, it then works for me:
>
>
> diff --git a/src/main/java/eu/jasha/**portaltests/stories/**NewUserStories.java
> b/src/main/java/eu/jasha/**portaltests/stories/**NewUserStories.java
> index 14257ce..1c8271d 100644
> --- a/src/main/java/eu/jasha/**portaltests/stories/**NewUserStories.java
> +++ b/src/main/java/eu/jasha/**portaltests/stories/**NewUserStories.java
> @@ -66,13 +66,13 @@ public class NewUserStories extends JUnitStories {
> .useStoryReporterBuilder(**reporterBuilder);
> useConfiguration(**configuration);
>
> - ApplicationContext context = new SpringApplicationContextFactor**
> y("newuser-steps.xml").**createApplicationContext();
> + ApplicationContext context = new SpringApplicationContextFactor**
> y("newUser-steps.xml").**createApplicationContext();
> useStepsFactory(new SpringStepsFactory(**configuration, context));
>
> }
>
> @Override
> protected List<String> storyPaths() {
> - return new StoryFinder().findPaths(**codeLocationFromClass(this.*
> *getClass()).getFile(), asList("**/newuser.story"), null);
> + return new StoryFinder().findPaths(**codeLocationFromClass(this.*
> *getClass()).getFile(), asList("**/newUser.story"), null);
>
> }
> }
>
>
>
>>
>> Marlon
>>
>>
>> On 4/13/12 7:43 AM, Jasha Joachimsthal wrote:
>>
>>> Jasha Joachimsthal
>>>
>>> Europe - Amsterdam - Oosteinde 11, 1017 WT Amsterdam - +31(0)20 522 4466
>>> US - Boston - 1 Broadway, Cambridge, MA 02142 - +1 877 414 4776 (toll
>>> free)
>>>
>>> www.onehippo.com
>>>
>>>
>>> On 12 April 2012 03:59, Franklin, Matthew B.<mf...@mitre.org>
>>> wrote:
>>>
>>> -----Original Message-----
>>>>> From: Marlon Pierce [mailto:marpierc@iu.edu]
>>>>> Sent: Tuesday, April 10, 2012 12:01 PM
>>>>> To: dev@rave.apache.org
>>>>> Subject: Re: Quality assurance steps for Rave
>>>>>
>>>>> I started a checklist of tests at
>>> http://wiki.apache.org/rave/**ReleaseManagement/**QualityAssurance<http://wiki.apache.org/rave/ReleaseManagement/QualityAssurance>,
>>> as you
>>> have seen from the [Rave Wiki] messages.
>>>
>>>>
>>>>> Good set of tests. I had been going through most of these on my own
>>>>> before each release, but that will become impossible to keep up with
>>>>> as the
>>>>> functionality grows.
>>>>>
>>>>>
>>> * Assuming the user interface tests are done manually, updating the wiki
>>> every time would be cruddy. Is there good test/assurance process
>>> management software out there? Should we do this with an online (Google)
>>> spreadsheet?
>>>
>>>>
>>>>> I say create a blocking Pre-release test task in Jira and have subtasks
>>>>> for each "section" of the tests. (we can use copy so it isn't such a
>>>>> manual
>>>>> task)
>>>>>
>>>>>
>>> * Do we want instead to actively maintain selenium tests? I started this
>>>
>>>> but
>>>>>
>>>> let it drop because these only worked in Firefox, and Firefox's frequent
>>> updates broke the tests. The tests themselves would have to be actively
>>> maintained--if you change or add a user interface feature, you need to
>>>
>>>> also
>>>>>
>>>> update the test.
>>>
>>>>
>>>>> I think if we develop the simplest possible tests to protect against
>>>>> regression, it could work without too much effort. We need to make it
>>>>> an
>>>>> explicit part of the process though. It has also been my experience
>>>>> that
>>>>> recorded test cases don't work without some modification. I have also
>>>>> had
>>>>> decent success in keeping code-based (not selenese) tests working....
>>>>>
>>>>>
>>> * Are there other recommendations beside Selenium?
>>>
>>>>
>>>>>
>>> I've just created a test project [1] with JBehave [2] which uses
>>>> Selenium
>>>> under the hood.
>>>>
>>>
>>> Scenario: User creates a new account and logs in into the portal
>>>> When I go to "http://localhost:8080/portal"
>>>> Then I see the login page
>>>> When I follow the new account link
>>>> Then I get the new account form
>>>> When I fill in the form with username "newuser" password "password"
>>>> confirmpassword "password" email "newuser@example.com"
>>>> And I submit the new account form
>>>> Then I see the login page
>>>> And A message appears "Account successfully created"
>>>> When I fill in the login form with username "newuser" password
>>>> "password"
>>>> Then I see my portal page with the add new widgets box
>>>>
>>>
>>>
>>> you can run it with mvn clean install or run the NewUserStories class
>>>> from
>>>> your IDE.
>>>> It does not clean up the newly created user so you cannot run it twice
>>>> successfully without cleaning up the user.
>>>>
>>>
>>> [1] https://github.com/jashaj/**PortalTests<https://github.com/jashaj/PortalTests>
>>>> [2] http://jbehave.org/
>>>>
>>>
>>> >
>>>>
>>>
>>> Marlon
>>>
>>>
>>> On 4/6/12 3:32 PM, Raminderjeet Singh wrote:
>>>
>>>> +1 with Marlon on having a checklist of features. I know going forward
>>>>>>>
>>>>>> this
>>>>>
>>>> list can grow and we may need more people to verify the build.
>>>
>>>>
>>>>>>> Based on my experience doing 0.10 release, Matt and others have done
>>>>>>> a
>>>>>>>
>>>>>> great job putting all the steps together into scripts. Thanks!
>>>
>>>>
>>>>>>> After the code freeze announcement, Release manager can tag the
>>>>>>> current
>>>>>>>
>>>>>> code and verification can be done on that particular tag. One
>>> developer
>>> (release manager also) can verify the code tag for the feature list and
>>>
>>>> if very
>>>>>
>>>> thing looks good can do the release based on the tag. Matt can comment
>>> more as i have to still understand when the pom versions are updated.
>>>
>>>>
>>>>>>> Thanks
>>>>>>> Raminder
>>>>>>>
>>>>>>> On Apr 6, 2012, at 2:53 PM, Marlon Pierce wrote:
>>>>>>>
>>>>>>> I'd like to propose the following.
>>>>>>>
>>>>>>> * Develop a list of Rave features that should be tested, put this on
>>>>>>>
>>>>>> the wiki,
>>>>>
>>>> and update it every time a new feature is added. I'm happy to get this
>>> started. Recommendations for tools to automate some of this are welcome.
>>>
>>>>
>>>>>>>
>>>>>>> * Regular testing of the above feature list (not just in preparation
>>>>>>> for
>>>>>>>
>>>>>> releases). We would need a record of this (who tested, when, etc).
>>> If
>>>
>>>> we do
>>>>>
>>>> this manually, we could keep records by updating the wiki, posting to
>>> the
>>>
>>>> dev
>>>>>
>>>> list, etc. Recommendations for better tools are of course welcome.
>>>
>>>>
>>>>>>>
>>>>>>> * All features should be tested before release by at least one
>>>>>>> person.
>>>>>>>
>>>>>> The
>>>>>
>>>> Release Manager should coordinate this. We could continue to do this as
>>>
>>>> part
>>>>>
>>>> of the current Release Candidate process, but it may be better to have
>>> an
>>> intermediate SVN tag that can go through QA before we start an official
>>>
>>>> vote.
>>>>>
>>>> This would allow us to keep the trunk open for commits while we test and
>>> avoid canceled releases.
>>>
>>>>
>>>>>>>
>>>>>>> I'm not a QA expert, so comments welcome.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Marlon
>>>>>>>
>>>>>>>
>>>>>>>
>>>>
>>> -----BEGIN PGP SIGNATURE-----
>> Version: GnuPG/MacGPG2 v2.0.16 (Darwin)
>> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>>
>> iQEcBAEBAgAGBQJPjyUfAAoJEEfVXE**ODPFIDMIwH/**2yeikOdkp5XeYbGJTTOBf+v
>> R/4uXMQ/tZnX9Of7/**efjTuIS5I6faPpRTVIwYEQFaY9yJZe**++0H7l2KCwayfJPPk
>> Fhi4TIkp8PEhjXPGnTcDzVqpxftMkW**m2bJTqxDZGTnRT3uQF9y+**MeKwtzjZVnrhs
>> pixgRSSanFsEco7XFckfu7qfTseqKV**xyTO/UKv43lozzcub41OlYgM/**CJWCOYfcd
>> awenewyLjCrBJlOegBgl/GJhFr+**ITBVtMfFZpqY6e0xOg3myTy+**ekpQU3+t4jiuY
>> rxYtmCyGoElAJuAQVA1uCv8tFpnzvk**ySxxWIQhhYlQo2grnHJbbgR6Z4Efh/**4yU=
>> =QFxm
>> -----END PGP SIGNATURE-----
>>
>
>
Re: Quality assurance steps for Rave
Posted by Marlon Pierce <ma...@iu.edu>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Thanks, Ate--I had nearly completed converting the JBehave tutorial to work with Rave and was about to set up a rival github site....
Marlon
On 4/18/12 5:17 PM, Ate Douma wrote:
> On 04/18/2012 10:33 PM, Marlon Pierce wrote:
> I'm finally looking at Jasha's example in detail, but the tests don't work (Firefox doesn't start). The JBehave tutorial (https://github.com/jbehave/jbehave-tutorial) works fine.
>
>
> Any idea what the problem is?
>> Seems like Jasha is developing on a case-insensitive file system :)
>
>> If you apply below patch it should run, it then works for me:
>
>> diff --git a/src/main/java/eu/jasha/portaltests/stories/NewUserStories.java b/src/main/java/eu/jasha/portaltests/stories/NewUserStories.java
>> index 14257ce..1c8271d 100644
>> --- a/src/main/java/eu/jasha/portaltests/stories/NewUserStories.java
>> +++ b/src/main/java/eu/jasha/portaltests/stories/NewUserStories.java
>> @@ -66,13 +66,13 @@ public class NewUserStories extends JUnitStories {
>> .useStoryReporterBuilder(reporterBuilder);
>> useConfiguration(configuration);
>
>> - ApplicationContext context = new SpringApplicationContextFactory("newuser-steps.xml").createApplicationContext();
>> + ApplicationContext context = new SpringApplicationContextFactory("newUser-steps.xml").createApplicationContext();
>> useStepsFactory(new SpringStepsFactory(configuration, context));
>
>> }
>
>> @Override
>> protected List<String> storyPaths() {
>> - return new StoryFinder().findPaths(codeLocationFromClass(this.getClass()).getFile(), asList("**/newuser.story"), null);
>> + return new StoryFinder().findPaths(codeLocationFromClass(this.getClass()).getFile(), asList("**/newUser.story"), null);
>> }
>> }
>
>
>
>
> Marlon
>
>
> On 4/13/12 7:43 AM, Jasha Joachimsthal wrote:
>>>> Jasha Joachimsthal
>>>>
>>>> Europe - Amsterdam - Oosteinde 11, 1017 WT Amsterdam - +31(0)20 522 4466
>>>> US - Boston - 1 Broadway, Cambridge, MA 02142 - +1 877 414 4776 (toll free)
>>>>
>>>> www.onehippo.com
>>>>
>>>>
>>>> On 12 April 2012 03:59, Franklin, Matthew B.<mf...@mitre.org> wrote:
>>>>
>>>>>> -----Original Message-----
>>>>>> From: Marlon Pierce [mailto:marpierc@iu.edu]
>>>>>> Sent: Tuesday, April 10, 2012 12:01 PM
>>>>>> To: dev@rave.apache.org
>>>>>> Subject: Re: Quality assurance steps for Rave
>>>>>>
>>>> I started a checklist of tests at
>>>> http://wiki.apache.org/rave/ReleaseManagement/QualityAssurance, as you
>>>> have seen from the [Rave Wiki] messages.
>>>>>>
>>>>>> Good set of tests. I had been going through most of these on my own
>>>>>> before each release, but that will become impossible to keep up with as the
>>>>>> functionality grows.
>>>>>>
>>>>
>>>> * Assuming the user interface tests are done manually, updating the wiki
>>>> every time would be cruddy. Is there good test/assurance process
>>>> management software out there? Should we do this with an online (Google)
>>>> spreadsheet?
>>>>>>
>>>>>> I say create a blocking Pre-release test task in Jira and have subtasks
>>>>>> for each "section" of the tests. (we can use copy so it isn't such a manual
>>>>>> task)
>>>>>>
>>>>
>>>> * Do we want instead to actively maintain selenium tests? I started this
>>>>>> but
>>>> let it drop because these only worked in Firefox, and Firefox's frequent
>>>> updates broke the tests. The tests themselves would have to be actively
>>>> maintained--if you change or add a user interface feature, you need to
>>>>>> also
>>>> update the test.
>>>>>>
>>>>>> I think if we develop the simplest possible tests to protect against
>>>>>> regression, it could work without too much effort. We need to make it an
>>>>>> explicit part of the process though. It has also been my experience that
>>>>>> recorded test cases don't work without some modification. I have also had
>>>>>> decent success in keeping code-based (not selenese) tests working....
>>>>>>
>>>>
>>>> * Are there other recommendations beside Selenium?
>>>>>>
>>>>
>>>>> I've just created a test project [1] with JBehave [2] which uses Selenium
>>>>> under the hood.
>>>>
>>>>> Scenario: User creates a new account and logs in into the portal
>>>>> When I go to "http://localhost:8080/portal"
>>>>> Then I see the login page
>>>>> When I follow the new account link
>>>>> Then I get the new account form
>>>>> When I fill in the form with username "newuser" password "password"
>>>>> confirmpassword "password" email "newuser@example.com"
>>>>> And I submit the new account form
>>>>> Then I see the login page
>>>>> And A message appears "Account successfully created"
>>>>> When I fill in the login form with username "newuser" password "password"
>>>>> Then I see my portal page with the add new widgets box
>>>>
>>>>
>>>>> you can run it with mvn clean install or run the NewUserStories class from
>>>>> your IDE.
>>>>> It does not clean up the newly created user so you cannot run it twice
>>>>> successfully without cleaning up the user.
>>>>
>>>>> [1] https://github.com/jashaj/PortalTests
>>>>> [2] http://jbehave.org/
>>>>
>>>>> >
>>>>
>>>> Marlon
>>>>
>>>>
>>>> On 4/6/12 3:32 PM, Raminderjeet Singh wrote:
>>>>>>>> +1 with Marlon on having a checklist of features. I know going forward
>>>>>> this
>>>> list can grow and we may need more people to verify the build.
>>>>>>>>
>>>>>>>> Based on my experience doing 0.10 release, Matt and others have done a
>>>> great job putting all the steps together into scripts. Thanks!
>>>>>>>>
>>>>>>>> After the code freeze announcement, Release manager can tag the current
>>>> code and verification can be done on that particular tag. One developer
>>>> (release manager also) can verify the code tag for the feature list and
>>>>>> if very
>>>> thing looks good can do the release based on the tag. Matt can comment
>>>> more as i have to still understand when the pom versions are updated.
>>>>>>>>
>>>>>>>> Thanks
>>>>>>>> Raminder
>>>>>>>>
>>>>>>>> On Apr 6, 2012, at 2:53 PM, Marlon Pierce wrote:
>>>>>>>>
>>>>>>>> I'd like to propose the following.
>>>>>>>>
>>>>>>>> * Develop a list of Rave features that should be tested, put this on
>>>>>> the wiki,
>>>> and update it every time a new feature is added. I'm happy to get this
>>>> started. Recommendations for tools to automate some of this are welcome.
>>>>>>>>
>>>>>>>>
>>>>>>>> * Regular testing of the above feature list (not just in preparation for
>>>> releases). We would need a record of this (who tested, when, etc). If
>>>>>> we do
>>>> this manually, we could keep records by updating the wiki, posting to the
>>>>>> dev
>>>> list, etc. Recommendations for better tools are of course welcome.
>>>>>>>>
>>>>>>>>
>>>>>>>> * All features should be tested before release by at least one person.
>>>>>> The
>>>> Release Manager should coordinate this. We could continue to do this as
>>>>>> part
>>>> of the current Release Candidate process, but it may be better to have an
>>>> intermediate SVN tag that can go through QA before we start an official
>>>>>> vote.
>>>> This would allow us to keep the trunk open for commits while we test and
>>>> avoid canceled releases.
>>>>>>>>
>>>>>>>>
>>>>>>>> I'm not a QA expert, so comments welcome.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Marlon
>>>>>>>>
>>>>>>>>
>>>>>
>>>>
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.16 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iQEcBAEBAgAGBQJPjzCnAAoJEEfVXEODPFIDM7gH/RrbDcWfscY/3FE5lkC+Eym5
MZh+qVts7WCJz5hdXqRnOxBRpOwr9bIJWqI0yrP3C2rinI+ue23Do+wrJTCFyqyE
7VZaNSIBlzY6O4Mvqwug4n0nfk6znAR/Wf0FbDVHDnL6sT1fB1P47cDFz96dTy51
Yy/PfaAEvIa7nTJmbJiuEQfV1gV9nbfWLmXLqNzsiJCrJ71tZR9hMmDxwlcIdrSB
bkOkzkt1f5G9NL6rUNkbAoH8hMW17L5wArs0zjMThwtK9ZzuZS/f76Z0y9pyTUUn
9rWhh0jBeQjNjK1Ru8EvZZweeg5npeoa5PPKaRNGl+SFEMwg4l71PmrWWFRPjnM=
=EM/D
-----END PGP SIGNATURE-----
Re: Quality assurance steps for Rave
Posted by Ate Douma <at...@douma.nu>.
On 04/18/2012 10:33 PM, Marlon Pierce wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> I'm finally looking at Jasha's example in detail, but the tests don't work (Firefox doesn't start). The JBehave tutorial (https://github.com/jbehave/jbehave-tutorial) works fine.
>
>
> Any idea what the problem is?
Seems like Jasha is developing on a case-insensitive file system :)
If you apply below patch it should run, it then works for me:
diff --git a/src/main/java/eu/jasha/portaltests/stories/NewUserStories.java
b/src/main/java/eu/jasha/portaltests/stories/NewUserStories.java
index 14257ce..1c8271d 100644
--- a/src/main/java/eu/jasha/portaltests/stories/NewUserStories.java
+++ b/src/main/java/eu/jasha/portaltests/stories/NewUserStories.java
@@ -66,13 +66,13 @@ public class NewUserStories extends JUnitStories {
.useStoryReporterBuilder(reporterBuilder);
useConfiguration(configuration);
- ApplicationContext context = new
SpringApplicationContextFactory("newuser-steps.xml").createApplicationContext();
+ ApplicationContext context = new
SpringApplicationContextFactory("newUser-steps.xml").createApplicationContext();
useStepsFactory(new SpringStepsFactory(configuration, context));
}
@Override
protected List<String> storyPaths() {
- return new
StoryFinder().findPaths(codeLocationFromClass(this.getClass()).getFile(),
asList("**/newuser.story"), null);
+ return new
StoryFinder().findPaths(codeLocationFromClass(this.getClass()).getFile(),
asList("**/newUser.story"), null);
}
}
>
>
> Marlon
>
>
> On 4/13/12 7:43 AM, Jasha Joachimsthal wrote:
>> Jasha Joachimsthal
>>
>> Europe - Amsterdam - Oosteinde 11, 1017 WT Amsterdam - +31(0)20 522 4466
>> US - Boston - 1 Broadway, Cambridge, MA 02142 - +1 877 414 4776 (toll free)
>>
>> www.onehippo.com
>>
>>
>> On 12 April 2012 03:59, Franklin, Matthew B.<mf...@mitre.org> wrote:
>>
>>>> -----Original Message-----
>>>> From: Marlon Pierce [mailto:marpierc@iu.edu]
>>>> Sent: Tuesday, April 10, 2012 12:01 PM
>>>> To: dev@rave.apache.org
>>>> Subject: Re: Quality assurance steps for Rave
>>>>
>> I started a checklist of tests at
>> http://wiki.apache.org/rave/ReleaseManagement/QualityAssurance, as you
>> have seen from the [Rave Wiki] messages.
>>>>
>>>> Good set of tests. I had been going through most of these on my own
>>>> before each release, but that will become impossible to keep up with as the
>>>> functionality grows.
>>>>
>>
>> * Assuming the user interface tests are done manually, updating the wiki
>> every time would be cruddy. Is there good test/assurance process
>> management software out there? Should we do this with an online (Google)
>> spreadsheet?
>>>>
>>>> I say create a blocking Pre-release test task in Jira and have subtasks
>>>> for each "section" of the tests. (we can use copy so it isn't such a manual
>>>> task)
>>>>
>>
>> * Do we want instead to actively maintain selenium tests? I started this
>>>> but
>> let it drop because these only worked in Firefox, and Firefox's frequent
>> updates broke the tests. The tests themselves would have to be actively
>> maintained--if you change or add a user interface feature, you need to
>>>> also
>> update the test.
>>>>
>>>> I think if we develop the simplest possible tests to protect against
>>>> regression, it could work without too much effort. We need to make it an
>>>> explicit part of the process though. It has also been my experience that
>>>> recorded test cases don't work without some modification. I have also had
>>>> decent success in keeping code-based (not selenese) tests working....
>>>>
>>
>> * Are there other recommendations beside Selenium?
>>>>
>>
>>> I've just created a test project [1] with JBehave [2] which uses Selenium
>>> under the hood.
>>
>>> Scenario: User creates a new account and logs in into the portal
>>> When I go to "http://localhost:8080/portal"
>>> Then I see the login page
>>> When I follow the new account link
>>> Then I get the new account form
>>> When I fill in the form with username "newuser" password "password"
>>> confirmpassword "password" email "newuser@example.com"
>>> And I submit the new account form
>>> Then I see the login page
>>> And A message appears "Account successfully created"
>>> When I fill in the login form with username "newuser" password "password"
>>> Then I see my portal page with the add new widgets box
>>
>>
>>> you can run it with mvn clean install or run the NewUserStories class from
>>> your IDE.
>>> It does not clean up the newly created user so you cannot run it twice
>>> successfully without cleaning up the user.
>>
>>> [1] https://github.com/jashaj/PortalTests
>>> [2] http://jbehave.org/
>>
>>> >
>>
>> Marlon
>>
>>
>> On 4/6/12 3:32 PM, Raminderjeet Singh wrote:
>>>>>> +1 with Marlon on having a checklist of features. I know going forward
>>>> this
>> list can grow and we may need more people to verify the build.
>>>>>>
>>>>>> Based on my experience doing 0.10 release, Matt and others have done a
>> great job putting all the steps together into scripts. Thanks!
>>>>>>
>>>>>> After the code freeze announcement, Release manager can tag the current
>> code and verification can be done on that particular tag. One developer
>> (release manager also) can verify the code tag for the feature list and
>>>> if very
>> thing looks good can do the release based on the tag. Matt can comment
>> more as i have to still understand when the pom versions are updated.
>>>>>>
>>>>>> Thanks
>>>>>> Raminder
>>>>>>
>>>>>> On Apr 6, 2012, at 2:53 PM, Marlon Pierce wrote:
>>>>>>
>>>>>> I'd like to propose the following.
>>>>>>
>>>>>> * Develop a list of Rave features that should be tested, put this on
>>>> the wiki,
>> and update it every time a new feature is added. I'm happy to get this
>> started. Recommendations for tools to automate some of this are welcome.
>>>>>>
>>>>>>
>>>>>> * Regular testing of the above feature list (not just in preparation for
>> releases). We would need a record of this (who tested, when, etc). If
>>>> we do
>> this manually, we could keep records by updating the wiki, posting to the
>>>> dev
>> list, etc. Recommendations for better tools are of course welcome.
>>>>>>
>>>>>>
>>>>>> * All features should be tested before release by at least one person.
>>>> The
>> Release Manager should coordinate this. We could continue to do this as
>>>> part
>> of the current Release Candidate process, but it may be better to have an
>> intermediate SVN tag that can go through QA before we start an official
>>>> vote.
>> This would allow us to keep the trunk open for commits while we test and
>> avoid canceled releases.
>>>>>>
>>>>>>
>>>>>> I'm not a QA expert, so comments welcome.
>>>>>>
>>>>>>
>>>>>>
>>>>>> Marlon
>>>>>>
>>>>>>
>>>
>>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG/MacGPG2 v2.0.16 (Darwin)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>
> iQEcBAEBAgAGBQJPjyUfAAoJEEfVXEODPFIDMIwH/2yeikOdkp5XeYbGJTTOBf+v
> R/4uXMQ/tZnX9Of7/efjTuIS5I6faPpRTVIwYEQFaY9yJZe++0H7l2KCwayfJPPk
> Fhi4TIkp8PEhjXPGnTcDzVqpxftMkWm2bJTqxDZGTnRT3uQF9y+MeKwtzjZVnrhs
> pixgRSSanFsEco7XFckfu7qfTseqKVxyTO/UKv43lozzcub41OlYgM/CJWCOYfcd
> awenewyLjCrBJlOegBgl/GJhFr+ITBVtMfFZpqY6e0xOg3myTy+ekpQU3+t4jiuY
> rxYtmCyGoElAJuAQVA1uCv8tFpnzvkySxxWIQhhYlQo2grnHJbbgR6Z4Efh/4yU=
> =QFxm
> -----END PGP SIGNATURE-----
Re: Quality assurance steps for Rave
Posted by Marlon Pierce <ma...@iu.edu>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
I'm finally looking at Jasha's example in detail, but the tests don't work (Firefox doesn't start). The JBehave tutorial (https://github.com/jbehave/jbehave-tutorial) works fine.
Any idea what the problem is?
Marlon
On 4/13/12 7:43 AM, Jasha Joachimsthal wrote:
> Jasha Joachimsthal
>
> Europe - Amsterdam - Oosteinde 11, 1017 WT Amsterdam - +31(0)20 522 4466
> US - Boston - 1 Broadway, Cambridge, MA 02142 - +1 877 414 4776 (toll free)
>
> www.onehippo.com
>
>
> On 12 April 2012 03:59, Franklin, Matthew B. <mf...@mitre.org> wrote:
>
>>> -----Original Message-----
>>> From: Marlon Pierce [mailto:marpierc@iu.edu]
>>> Sent: Tuesday, April 10, 2012 12:01 PM
>>> To: dev@rave.apache.org
>>> Subject: Re: Quality assurance steps for Rave
>>>
> I started a checklist of tests at
> http://wiki.apache.org/rave/ReleaseManagement/QualityAssurance, as you
> have seen from the [Rave Wiki] messages.
>>>
>>> Good set of tests. I had been going through most of these on my own
>>> before each release, but that will become impossible to keep up with as the
>>> functionality grows.
>>>
>
> * Assuming the user interface tests are done manually, updating the wiki
> every time would be cruddy. Is there good test/assurance process
> management software out there? Should we do this with an online (Google)
> spreadsheet?
>>>
>>> I say create a blocking Pre-release test task in Jira and have subtasks
>>> for each "section" of the tests. (we can use copy so it isn't such a manual
>>> task)
>>>
>
> * Do we want instead to actively maintain selenium tests? I started this
>>> but
> let it drop because these only worked in Firefox, and Firefox's frequent
> updates broke the tests. The tests themselves would have to be actively
> maintained--if you change or add a user interface feature, you need to
>>> also
> update the test.
>>>
>>> I think if we develop the simplest possible tests to protect against
>>> regression, it could work without too much effort. We need to make it an
>>> explicit part of the process though. It has also been my experience that
>>> recorded test cases don't work without some modification. I have also had
>>> decent success in keeping code-based (not selenese) tests working....
>>>
>
> * Are there other recommendations beside Selenium?
>>>
>
>> I've just created a test project [1] with JBehave [2] which uses Selenium
>> under the hood.
>
>> Scenario: User creates a new account and logs in into the portal
>> When I go to "http://localhost:8080/portal"
>> Then I see the login page
>> When I follow the new account link
>> Then I get the new account form
>> When I fill in the form with username "newuser" password "password"
>> confirmpassword "password" email "newuser@example.com"
>> And I submit the new account form
>> Then I see the login page
>> And A message appears "Account successfully created"
>> When I fill in the login form with username "newuser" password "password"
>> Then I see my portal page with the add new widgets box
>
>
>> you can run it with mvn clean install or run the NewUserStories class from
>> your IDE.
>> It does not clean up the newly created user so you cannot run it twice
>> successfully without cleaning up the user.
>
>> [1] https://github.com/jashaj/PortalTests
>> [2] http://jbehave.org/
>
>> >
>
> Marlon
>
>
> On 4/6/12 3:32 PM, Raminderjeet Singh wrote:
>>>>> +1 with Marlon on having a checklist of features. I know going forward
>>> this
> list can grow and we may need more people to verify the build.
>>>>>
>>>>> Based on my experience doing 0.10 release, Matt and others have done a
> great job putting all the steps together into scripts. Thanks!
>>>>>
>>>>> After the code freeze announcement, Release manager can tag the current
> code and verification can be done on that particular tag. One developer
> (release manager also) can verify the code tag for the feature list and
>>> if very
> thing looks good can do the release based on the tag. Matt can comment
> more as i have to still understand when the pom versions are updated.
>>>>>
>>>>> Thanks
>>>>> Raminder
>>>>>
>>>>> On Apr 6, 2012, at 2:53 PM, Marlon Pierce wrote:
>>>>>
>>>>> I'd like to propose the following.
>>>>>
>>>>> * Develop a list of Rave features that should be tested, put this on
>>> the wiki,
> and update it every time a new feature is added. I'm happy to get this
> started. Recommendations for tools to automate some of this are welcome.
>>>>>
>>>>>
>>>>> * Regular testing of the above feature list (not just in preparation for
> releases). We would need a record of this (who tested, when, etc). If
>>> we do
> this manually, we could keep records by updating the wiki, posting to the
>>> dev
> list, etc. Recommendations for better tools are of course welcome.
>>>>>
>>>>>
>>>>> * All features should be tested before release by at least one person.
>>> The
> Release Manager should coordinate this. We could continue to do this as
>>> part
> of the current Release Candidate process, but it may be better to have an
> intermediate SVN tag that can go through QA before we start an official
>>> vote.
> This would allow us to keep the trunk open for commits while we test and
> avoid canceled releases.
>>>>>
>>>>>
>>>>> I'm not a QA expert, so comments welcome.
>>>>>
>>>>>
>>>>>
>>>>> Marlon
>>>>>
>>>>>
>>
>
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.16 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iQEcBAEBAgAGBQJPjyUfAAoJEEfVXEODPFIDMIwH/2yeikOdkp5XeYbGJTTOBf+v
R/4uXMQ/tZnX9Of7/efjTuIS5I6faPpRTVIwYEQFaY9yJZe++0H7l2KCwayfJPPk
Fhi4TIkp8PEhjXPGnTcDzVqpxftMkWm2bJTqxDZGTnRT3uQF9y+MeKwtzjZVnrhs
pixgRSSanFsEco7XFckfu7qfTseqKVxyTO/UKv43lozzcub41OlYgM/CJWCOYfcd
awenewyLjCrBJlOegBgl/GJhFr+ITBVtMfFZpqY6e0xOg3myTy+ekpQU3+t4jiuY
rxYtmCyGoElAJuAQVA1uCv8tFpnzvkySxxWIQhhYlQo2grnHJbbgR6Z4Efh/4yU=
=QFxm
-----END PGP SIGNATURE-----
Re: Quality assurance steps for Rave
Posted by Jasha Joachimsthal <j....@onehippo.com>.
Jasha Joachimsthal
Europe - Amsterdam - Oosteinde 11, 1017 WT Amsterdam - +31(0)20 522 4466
US - Boston - 1 Broadway, Cambridge, MA 02142 - +1 877 414 4776 (toll free)
www.onehippo.com
On 12 April 2012 03:59, Franklin, Matthew B. <mf...@mitre.org> wrote:
> >-----Original Message-----
> >From: Marlon Pierce [mailto:marpierc@iu.edu]
> >Sent: Tuesday, April 10, 2012 12:01 PM
> >To: dev@rave.apache.org
> >Subject: Re: Quality assurance steps for Rave
> >
> >-----BEGIN PGP SIGNED MESSAGE-----
> >Hash: SHA1
> >
> >I started a checklist of tests at
> >http://wiki.apache.org/rave/ReleaseManagement/QualityAssurance, as you
> >have seen from the [Rave Wiki] messages.
>
> Good set of tests. I had been going through most of these on my own
> before each release, but that will become impossible to keep up with as the
> functionality grows.
>
> >
> >* Assuming the user interface tests are done manually, updating the wiki
> >every time would be cruddy. Is there good test/assurance process
> >management software out there? Should we do this with an online (Google)
> >spreadsheet?
>
> I say create a blocking Pre-release test task in Jira and have subtasks
> for each "section" of the tests. (we can use copy so it isn't such a manual
> task)
>
> >
> >* Do we want instead to actively maintain selenium tests? I started this
> but
> >let it drop because these only worked in Firefox, and Firefox's frequent
> >updates broke the tests. The tests themselves would have to be actively
> >maintained--if you change or add a user interface feature, you need to
> also
> >update the test.
>
> I think if we develop the simplest possible tests to protect against
> regression, it could work without too much effort. We need to make it an
> explicit part of the process though. It has also been my experience that
> recorded test cases don't work without some modification. I have also had
> decent success in keeping code-based (not selenese) tests working....
>
> >
> >* Are there other recommendations beside Selenium?
>
I've just created a test project [1] with JBehave [2] which uses Selenium
under the hood.
Scenario: User creates a new account and logs in into the portal
When I go to "http://localhost:8080/portal"
Then I see the login page
When I follow the new account link
Then I get the new account form
When I fill in the form with username "newuser" password "password"
confirmpassword "password" email "newuser@example.com"
And I submit the new account form
Then I see the login page
And A message appears "Account successfully created"
When I fill in the login form with username "newuser" password "password"
Then I see my portal page with the add new widgets box
you can run it with mvn clean install or run the NewUserStories class from
your IDE.
It does not clean up the newly created user so you cannot run it twice
successfully without cleaning up the user.
[1] https://github.com/jashaj/PortalTests
[2] http://jbehave.org/
>
> >
> >Marlon
> >
> >
> >On 4/6/12 3:32 PM, Raminderjeet Singh wrote:
> >> +1 with Marlon on having a checklist of features. I know going forward
> this
> >list can grow and we may need more people to verify the build.
> >>
> >> Based on my experience doing 0.10 release, Matt and others have done a
> >great job putting all the steps together into scripts. Thanks!
> >>
> >> After the code freeze announcement, Release manager can tag the current
> >code and verification can be done on that particular tag. One developer
> >(release manager also) can verify the code tag for the feature list and
> if very
> >thing looks good can do the release based on the tag. Matt can comment
> >more as i have to still understand when the pom versions are updated.
> >>
> >> Thanks
> >> Raminder
> >>
> >> On Apr 6, 2012, at 2:53 PM, Marlon Pierce wrote:
> >>
> >> I'd like to propose the following.
> >>
> >> * Develop a list of Rave features that should be tested, put this on
> the wiki,
> >and update it every time a new feature is added. I'm happy to get this
> >started. Recommendations for tools to automate some of this are welcome.
> >>
> >>
> >> * Regular testing of the above feature list (not just in preparation for
> >releases). We would need a record of this (who tested, when, etc). If
> we do
> >this manually, we could keep records by updating the wiki, posting to the
> dev
> >list, etc. Recommendations for better tools are of course welcome.
> >>
> >>
> >> * All features should be tested before release by at least one person.
> The
> >Release Manager should coordinate this. We could continue to do this as
> part
> >of the current Release Candidate process, but it may be better to have an
> >intermediate SVN tag that can go through QA before we start an official
> vote.
> >This would allow us to keep the trunk open for commits while we test and
> >avoid canceled releases.
> >>
> >>
> >> I'm not a QA expert, so comments welcome.
> >>
> >>
> >>
> >> Marlon
> >>
> >>
> >-----BEGIN PGP SIGNATURE-----
> >Version: GnuPG/MacGPG2 v2.0.16 (Darwin)
> >Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
> >
> >iQEcBAEBAgAGBQJPhFlJAAoJEEfVXEODPFIDPZ4H/2FM9G6b6mJuJgkong4cAW
> >wg
> >TMj8Vks5ykOsRspkqIff4Q8ItzkVrrq0hKb+1p2gaaGj8nG6/BhKnh+CaSsuwfhC
> >QqeucJdj4/0RreCx+eQ9M0A17Y9O21yZxKuKQaiPHHWbv32NZWHsuycUPDP+
> >MmvI
> >mC5/w9+N2jQf2O7y6X0fR6ydfyggJrhBbM0PNY3yp6O7dHb91KqUz4Vm5giXl1
> >H/
> >OzG0NEDv/EweIbTCeEJWOs2oDZGZgkRCT7z0LoJXXWNHifUSSJMuUDwPu0M
> >5CIIJ
> >tX16D/JdC2ik9FtarCHeQV101uEJYrC3Mx4F9Lb/PO0Hj+ePOBrha9rMRnSOcK0=
> >=06BC
> >-----END PGP SIGNATURE-----
>
RE: Quality assurance steps for Rave
Posted by "Franklin, Matthew B." <mf...@mitre.org>.
>-----Original Message-----
>From: Marlon Pierce [mailto:marpierc@iu.edu]
>Sent: Tuesday, April 10, 2012 12:01 PM
>To: dev@rave.apache.org
>Subject: Re: Quality assurance steps for Rave
>
>-----BEGIN PGP SIGNED MESSAGE-----
>Hash: SHA1
>
>I started a checklist of tests at
>http://wiki.apache.org/rave/ReleaseManagement/QualityAssurance, as you
>have seen from the [Rave Wiki] messages.
Good set of tests. I had been going through most of these on my own before each release, but that will become impossible to keep up with as the functionality grows.
>
>* Assuming the user interface tests are done manually, updating the wiki
>every time would be cruddy. Is there good test/assurance process
>management software out there? Should we do this with an online (Google)
>spreadsheet?
I say create a blocking Pre-release test task in Jira and have subtasks for each "section" of the tests. (we can use copy so it isn't such a manual task)
>
>* Do we want instead to actively maintain selenium tests? I started this but
>let it drop because these only worked in Firefox, and Firefox's frequent
>updates broke the tests. The tests themselves would have to be actively
>maintained--if you change or add a user interface feature, you need to also
>update the test.
I think if we develop the simplest possible tests to protect against regression, it could work without too much effort. We need to make it an explicit part of the process though. It has also been my experience that recorded test cases don't work without some modification. I have also had decent success in keeping code-based (not selenese) tests working....
>
>* Are there other recommendations beside Selenium?
>
>
>Marlon
>
>
>On 4/6/12 3:32 PM, Raminderjeet Singh wrote:
>> +1 with Marlon on having a checklist of features. I know going forward this
>list can grow and we may need more people to verify the build.
>>
>> Based on my experience doing 0.10 release, Matt and others have done a
>great job putting all the steps together into scripts. Thanks!
>>
>> After the code freeze announcement, Release manager can tag the current
>code and verification can be done on that particular tag. One developer
>(release manager also) can verify the code tag for the feature list and if very
>thing looks good can do the release based on the tag. Matt can comment
>more as i have to still understand when the pom versions are updated.
>>
>> Thanks
>> Raminder
>>
>> On Apr 6, 2012, at 2:53 PM, Marlon Pierce wrote:
>>
>> I'd like to propose the following.
>>
>> * Develop a list of Rave features that should be tested, put this on the wiki,
>and update it every time a new feature is added. I'm happy to get this
>started. Recommendations for tools to automate some of this are welcome.
>>
>>
>> * Regular testing of the above feature list (not just in preparation for
>releases). We would need a record of this (who tested, when, etc). If we do
>this manually, we could keep records by updating the wiki, posting to the dev
>list, etc. Recommendations for better tools are of course welcome.
>>
>>
>> * All features should be tested before release by at least one person. The
>Release Manager should coordinate this. We could continue to do this as part
>of the current Release Candidate process, but it may be better to have an
>intermediate SVN tag that can go through QA before we start an official vote.
>This would allow us to keep the trunk open for commits while we test and
>avoid canceled releases.
>>
>>
>> I'm not a QA expert, so comments welcome.
>>
>>
>>
>> Marlon
>>
>>
>-----BEGIN PGP SIGNATURE-----
>Version: GnuPG/MacGPG2 v2.0.16 (Darwin)
>Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>
>iQEcBAEBAgAGBQJPhFlJAAoJEEfVXEODPFIDPZ4H/2FM9G6b6mJuJgkong4cAW
>wg
>TMj8Vks5ykOsRspkqIff4Q8ItzkVrrq0hKb+1p2gaaGj8nG6/BhKnh+CaSsuwfhC
>QqeucJdj4/0RreCx+eQ9M0A17Y9O21yZxKuKQaiPHHWbv32NZWHsuycUPDP+
>MmvI
>mC5/w9+N2jQf2O7y6X0fR6ydfyggJrhBbM0PNY3yp6O7dHb91KqUz4Vm5giXl1
>H/
>OzG0NEDv/EweIbTCeEJWOs2oDZGZgkRCT7z0LoJXXWNHifUSSJMuUDwPu0M
>5CIIJ
>tX16D/JdC2ik9FtarCHeQV101uEJYrC3Mx4F9Lb/PO0Hj+ePOBrha9rMRnSOcK0=
>=06BC
>-----END PGP SIGNATURE-----
Re: Quality assurance steps for Rave
Posted by Marlon Pierce <ma...@iu.edu>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
I started a checklist of tests at http://wiki.apache.org/rave/ReleaseManagement/QualityAssurance, as you have seen from the [Rave Wiki] messages.
* Assuming the user interface tests are done manually, updating the wiki every time would be cruddy. Is there good test/assurance process management software out there? Should we do this with an online (Google) spreadsheet?
* Do we want instead to actively maintain selenium tests? I started this but let it drop because these only worked in Firefox, and Firefox's frequent updates broke the tests. The tests themselves would have to be actively maintained--if you change or add a user interface feature, you need to also update the test.
* Are there other recommendations beside Selenium?
Marlon
On 4/6/12 3:32 PM, Raminderjeet Singh wrote:
> +1 with Marlon on having a checklist of features. I know going forward this list can grow and we may need more people to verify the build.
>
> Based on my experience doing 0.10 release, Matt and others have done a great job putting all the steps together into scripts. Thanks!
>
> After the code freeze announcement, Release manager can tag the current code and verification can be done on that particular tag. One developer (release manager also) can verify the code tag for the feature list and if very thing looks good can do the release based on the tag. Matt can comment more as i have to still understand when the pom versions are updated.
>
> Thanks
> Raminder
>
> On Apr 6, 2012, at 2:53 PM, Marlon Pierce wrote:
>
> I'd like to propose the following.
>
> * Develop a list of Rave features that should be tested, put this on the wiki, and update it every time a new feature is added. I'm happy to get this started. Recommendations for tools to automate some of this are welcome.
>
>
> * Regular testing of the above feature list (not just in preparation for releases). We would need a record of this (who tested, when, etc). If we do this manually, we could keep records by updating the wiki, posting to the dev list, etc. Recommendations for better tools are of course welcome.
>
>
> * All features should be tested before release by at least one person. The Release Manager should coordinate this. We could continue to do this as part of the current Release Candidate process, but it may be better to have an intermediate SVN tag that can go through QA before we start an official vote. This would allow us to keep the trunk open for commits while we test and avoid canceled releases.
>
>
> I'm not a QA expert, so comments welcome.
>
>
>
> Marlon
>
>
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.16 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iQEcBAEBAgAGBQJPhFlJAAoJEEfVXEODPFIDPZ4H/2FM9G6b6mJuJgkong4cAWwg
TMj8Vks5ykOsRspkqIff4Q8ItzkVrrq0hKb+1p2gaaGj8nG6/BhKnh+CaSsuwfhC
QqeucJdj4/0RreCx+eQ9M0A17Y9O21yZxKuKQaiPHHWbv32NZWHsuycUPDP+MmvI
mC5/w9+N2jQf2O7y6X0fR6ydfyggJrhBbM0PNY3yp6O7dHb91KqUz4Vm5giXl1H/
OzG0NEDv/EweIbTCeEJWOs2oDZGZgkRCT7z0LoJXXWNHifUSSJMuUDwPu0M5CIIJ
tX16D/JdC2ik9FtarCHeQV101uEJYrC3Mx4F9Lb/PO0Hj+ePOBrha9rMRnSOcK0=
=06BC
-----END PGP SIGNATURE-----
Re: Quality assurance steps for Rave
Posted by Raminderjeet Singh <ra...@gmail.com>.
+1 with Marlon on having a checklist of features. I know going forward this list can grow and we may need more people to verify the build.
Based on my experience doing 0.10 release, Matt and others have done a great job putting all the steps together into scripts. Thanks!
After the code freeze announcement, Release manager can tag the current code and verification can be done on that particular tag. One developer (release manager also) can verify the code tag for the feature list and if very thing looks good can do the release based on the tag. Matt can comment more as i have to still understand when the pom versions are updated.
Thanks
Raminder
On Apr 6, 2012, at 2:53 PM, Marlon Pierce wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> I'd like to propose the following.
>
> * Develop a list of Rave features that should be tested, put this on the wiki, and update it every time a new feature is added. I'm happy to get this started. Recommendations for tools to automate some of this are welcome.
>
>
> * Regular testing of the above feature list (not just in preparation for releases). We would need a record of this (who tested, when, etc). If we do this manually, we could keep records by updating the wiki, posting to the dev list, etc. Recommendations for better tools are of course welcome.
>
>
> * All features should be tested before release by at least one person. The Release Manager should coordinate this. We could continue to do this as part of the current Release Candidate process, but it may be better to have an intermediate SVN tag that can go through QA before we start an official vote. This would allow us to keep the trunk open for commits while we test and avoid canceled releases.
>
>
> I'm not a QA expert, so comments welcome.
>
>
>
> Marlon
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG/MacGPG2 v2.0.16 (Darwin)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>
> iQEcBAEBAgAGBQJPfzueAAoJEEfVXEODPFIDPb0IAJT5Op6iaQbKgO5Jp9wBjsCp
> oC2Jxk5HrGolbavE8yi0LDAGv8x52mo1U8+m7PfRT6yhQ38ciMMkm8I7W2BYZ2Ei
> WWX1kerVriICKV4LSfZ3u503OJ77WYlhoOsJwvvZEvxYndKuhroYyxDIZ0PdTsGI
> GRp7Iu4r9My4AOz+bIh4EaAWE+VKHhS9INoHvfsm6Fb8vxAyvSNfklgtOiZoTSlg
> g4BxvEPlkxs5Xv3jTFaX/rgGWoKU31LkL8mSLIeW64BWXBqXMscZSTSW/l5Brx/8
> dJLnuHUmmSJLjKhzELNaDH68Rntb1WFV+oWsWf8xBqawYKXkVElqNywTh5yOxb8=
> =1anU
> -----END PGP SIGNATURE-----