You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by Alexey Melezhik <me...@gmail.com> on 2016/01/27 21:53:48 UTC
collaboration request - apache server automation testing with swat tool
Hi! I am the author of swat - https://github.com/melezhik/swat -
perl/curl based DSL for web services test automation.
I'd like to contribute in at apache server automated testing. Please
take a look at _simple_ example of swat test suite for apache web
server - https://sparrowhub.org/info/swat-apache?v=0.000004
Some swat features ( more info could be found at
https://github.com/melezhik/swat ) which possibly could be of
interested for you:
* swat is DSL for rapid test scenarios development for various web
services and application
* swat is written on Perl and could be extended by Perl if you need
something more complicated when DSL is not enough
* swat has minimal dependencies ( mostly core perl modules plus few
CPAN ones ) and easy to install
* swat relies on perl prove as internal test runner and so yield test
reports in portable TAP format
* swat relies on curl to make http requests against tested
application, so there is almost no limit for you - sending, saving
cookies, complicated http headers, make html form uploads, etc
* swat allow both simple (smoke) tests development and complicated
integration tests - with sequential, coupled http requests, saving
intermediate state, code reuse, etc. This is classical program API for
complex things combined with intuitive simple DSL ( for simple and
medium things )
* not only httpd server but other a Apache Foundation products could
be tested with swat ( tomcat, ... ) - take a look at
https://sparrowhub.org/ - central swat test suite repository to get
more examples.
I'd be glad to get feedback from you guys.
Feel free to ask any further questions.
Alexey Melezhik - the author of swat and sparrow.
Re: collaboration request - apache server automation testing with
swat tool
Posted by Alexey Melezhik <me...@gmail.com>.
Hi Bill! Thanks a lot for your answer. Quick remarks.
> IMHO more suites are
always a benefit, not a competition between test tools.
Completely agree with you
> that it will be a challenge to
bring the resource to bear to replicate what is being done right now
I see. I am not going to replace your current framework by mine, sure.
But what I can suggest. There are some gaps I may be covered by swat.
Let me say a couple:
1) consider INTEGRATION TESTING under various platforms and
environments ( debian, ubuntu, centos, gentoo ... ). Chef folks use
tool called test-kitchen which is simple way to get desired OS and
install something there and then run some tests against installed
product. They use this extensively in chef cookbook development.
Indeed the same workflow could be used for testing apache. Consider
this:
- install apache ( via vendored package ) on specific platform
(centos, ubuntu, fedora, suse slackware - whatever ) - and the run
some (smoke? sanity ) integration tests so to discovery environment
specific gray zones / issues with apache - which of course might arise
!
Having a extensive experiences in devops,chef, configuration
management I may give you a good help in writing such tests (using
swat) and automating tests environments ( using chef, test kitchen,
travis, vagrant )
2) Cover unresolved issues or test cases ( listed at "Tests to be
written:" http://svn.apache.org/repos/asf/httpd/test/framework/trunk/STATUS
) using swat.
Sometimes there is no much time to write a tests, is it ? (: Swat test
development in _most cases_ take a minimal amount of time. What if I
try to "prove" it? ((: What about writing tests from "Tests to be
written:" section ( with some moderate help of apache gurus on apache
specific details if need ):
* t/apache
- simulations of network failures (incomplete POST bodies,
chunked and unchunked; missing POST bodies; slooow
client connexions, such as taking 1 minute to send
1KiB; ...)
* t/modules/autoindex
- something seems possibly broken with inheritance on 2.0
* t/ssl
- SSLPassPhraseDialog exec:
- SSLRandomSeed exec:
That's it. Please follow up.
Thanks
2016-01-28 5:33 GMT+03:00 William A Rowe Jr <wr...@rowe-clan.net>:
> On Wed, Jan 27, 2016 at 2:53 PM, Alexey Melezhik <me...@gmail.com> wrote:
>>
>> Hi! I am the author of swat - https://github.com/melezhik/swat -
>> perl/curl based DSL for web services test automation.
>> I'd like to contribute in at apache server automated testing. Please
>> take a look at _simple_ example of swat test suite for apache web
>> server - https://sparrowhub.org/info/swat-apache?v=0.000004
>>
>> Some swat features ( more info could be found at
>> https://github.com/melezhik/swat ) which possibly could be of
>> interested for you:
>>
>> * swat is DSL for rapid test scenarios development for various web
>> services and application
>> * swat is written on Perl and could be extended by Perl if you need
>> something more complicated when DSL is not enough
>> * swat has minimal dependencies ( mostly core perl modules plus few
>> CPAN ones ) and easy to install
>> * swat relies on perl prove as internal test runner and so yield test
>> reports in portable TAP format
>> * swat relies on curl to make http requests against tested
>> application, so there is almost no limit for you - sending, saving
>> cookies, complicated http headers, make html form uploads, etc
>> * swat allow both simple (smoke) tests development and complicated
>> integration tests - with sequential, coupled http requests, saving
>> intermediate state, code reuse, etc. This is classical program API for
>> complex things combined with intuitive simple DSL ( for simple and
>> medium things )
>>
>> * not only httpd server but other a Apache Foundation products could
>> be tested with swat ( tomcat, ... ) - take a look at
>> https://sparrowhub.org/ - central swat test suite repository to get
>> more examples.
>>
>> I'd be glad to get feedback from you guys.
>>
>> Feel free to ask any further questions.
>>
>> Alexey Melezhik - the author of swat and sparrow.
>
>
> Thanks for the pointers and your efforts. IMHO more suites are
> always a benefit, not a competition between test tools.
>
> For a sense of what we actively review today, please do take a quick
> look at http://svn.apache.org/repos/asf/httpd/test/framework/trunk/
> to understand the scope of the current framework.
>
> I personally don't care precisely which tool or framework I use, as long
> as it is comprehensive. I'm looking forward to looking at what you have
> put together, but wanted you to be aware that it will be a challenge to
> bring the resource to bear to replicate what is being done right now
> under our existing framework.
>
> Props and cheers,
>
> Bill
>
Re: collaboration request - apache server automation testing with
swat tool
Posted by Alexey Melezhik <me...@gmail.com>.
Hi ! I have got a few comments , questions concerning this topic, thanks
for everybody! though still have idea if Apache devs are interested in new
testing framework ? It is hard to keep moving doubting if such tool could
be of interest for someone else beside me ) , so give me a sign please :)
If I don't waist my time.
Thanks.
пятница, 5 февраля 2016 г. пользователь Jim Jagielski написал:
> Personally, I like the idea of having another framework;
> the current one is OK but somewhat "painful" to update.
>
> I wonder how possible it would be to transcode the old tests
> to Swat? We could then provide for 2 testing frameworks, one
> developed by the ASF and the other external and 3rd party.
>
>
> > On Feb 1, 2016, at 5:23 AM, Alexey Melezhik <melezhik@gmail.com
> <javascript:;>> wrote:
> >
> > Here the list of existed issues I was able to automate tests for:
> >
> > vagrant@Debian-jessie-amd64-netboot:~/my/apache-swat$ ls -1 | grep -P
> '\d'
> > 44221
> > 46751
> > 58789
> > 58828
> > 58854
> >
> >
> > I have informed developers at https://bz.apache.org/bugzilla/ about
> > these, so they rely upon them. Or you have some pre-release testing,
> > so run all of these in a whole chunk?
> >
> > Regards
> >
> > Alexey
> >
> > PS chaining issues / tests listing as always -
> > https://github.com/melezhik/apache-swat
> >
> > PS2 it'd be could to verify tests as well against "bleeding edge"
> > apache version gets installed from SCM, but I don't know how to do
> > this.
> > Currently I use http://httpd.apache.org/download.cgi ( last stable
> release )
> >
> >
> >
> > 2016-01-30 21:36 GMT+03:00 Alexey Melezhik <melezhik@gmail.com
> <javascript:;>>:
> >> Hi Bill!
> >>
> >> I have started to assemble swat tests for apache2 issues coming from
> >> https://bz.apache.org/bugzilla
> >>
> >> And this the first one -
> https://bz.apache.org/bugzilla/show_bug.cgi?id=44221 ,
> >>
> >> Test suite is failed for the moment :
> >>
> >>
> >> vagrant@Debian-jessie-amd64-netboot:~/my/apache-swat$ swat -t 44221
> >> /home/vagrant/.swat/.cache/384/prove/44221/foo.baz/00.GET.t ...
> >> ok 1 - GET 127.0.0.1/44221/foo.baz succeeded
> >> # http headers saved to
> /home/vagrant/.swat/.cache/384/prove/52jLAo76YZ.hdr
> >> # body saved to /home/vagrant/.swat/.cache/384/prove/52jLAo76YZ
> >> ok 2 - output match '200 OK'
> >> 1..2
> >> ok
> >> /home/vagrant/.swat/.cache/384/prove/44221/FOO.bar/00.GET.t ...
> >> ok 1 - GET 127.0.0.1/44221/FOO.bar succeeded
> >> # http headers saved to
> /home/vagrant/.swat/.cache/384/prove/DvEZnO6ALe.hdr
> >> # body saved to /home/vagrant/.swat/.cache/384/prove/DvEZnO6ALe
> >> ok 2 - output match '200 OK'
> >> ok 3 - output match /Location: \S+/
> >> ok 4 - 'Location: http://127.0.0.1/44221/foo.bar' match 'foo.bar'
> >> 1..4
> >> ok
> >> /home/vagrant/.swat/.cache/384/prove/44221/foo.BAR/00.GET.t ...
> >> ok 1 - GET 127.0.0.1/44221/foo.BAR succeeded
> >> # http headers saved to
> /home/vagrant/.swat/.cache/384/prove/bWQ1Sbl0YZ.hdr
> >> # body saved to /home/vagrant/.swat/.cache/384/prove/bWQ1Sbl0YZ
> >> ok 2 - output match '200 OK'
> >> ok 3 - output match /Location: \S+/
> >> ok 4 - 'Location: http://127.0.0.1/44221/foo.bar' match 'foo.bar'
> >> 1..4
> >> ok
> >> /home/vagrant/.swat/.cache/384/prove/44221/foo.bar/00.GET.t ...
> >> ok 1 - GET 127.0.0.1/44221/foo.bar succeeded
> >> # http headers saved to
> /home/vagrant/.swat/.cache/384/prove/Jd_hlo7Qwv.hdr
> >> # body saved to /home/vagrant/.swat/.cache/384/prove/Jd_hlo7Qwv
> >> ok 2 - output match '200 OK'
> >> 1..2
> >> ok
> >> /home/vagrant/.swat/.cache/384/prove/44221/foo.html/00.GET.t ..
> >> ok 1 - GET 127.0.0.1/44221/foo.html succeeded
> >> # http headers saved to
> /home/vagrant/.swat/.cache/384/prove/tW2L511eym.hdr
> >> # body saved to /home/vagrant/.swat/.cache/384/prove/tW2L511eym
> >> ok 2 - output match /HTTP\/(\S+) (\d+) \S+/
> >> not ok 3 - 'HTTP/1.1 300 Multiple Choices' match /404 /
> >>
> >> # Failed test ''HTTP/1.1 300 Multiple Choices' match /404 /'
> >> # at /usr/local/share/perl/5.20.2/swat.pm line 218.
> >> not ok 4 - output match 'Not Found'
> >>
> >> # Failed test 'output match 'Not Found''
> >> # at /usr/local/share/perl/5.20.2/swat.pm line 218.
> >> 1..4
> >> # Looks like you failed 2 tests of 4.
> >> Dubious, test returned 2 (wstat 512, 0x200)
> >> Failed 2/4 subtests
> >>
> >> Test Summary Report
> >> -------------------
> >> /home/vagrant/.swat/.cache/384/prove/44221/foo.html/00.GET.t (Wstat:
> >> 512 Tests: 4 Failed: 2)
> >> Failed tests: 3-4
> >> Non-zero exit status: 2
> >> Files=5, Tests=16, 0 wallclock secs ( 0.03 usr 0.00 sys + 0.27 cusr
> >> 0.00 csys = 0.30 CPU)
> >> Result: FAIL
> >>
> >> ...
> >>
> >> Ok, this is just the beginning ;-))) , watch soon updates at -
> >> https://github.com/melezhik/apache-swat
> >>
> >> 2016-01-29 20:44 GMT+03:00 William A Rowe Jr <wrowe@rowe-clan.net
> <javascript:;>>:
> >>> On Fri, Jan 29, 2016 at 6:35 AM, Alexey Melezhik <melezhik@gmail.com
> <javascript:;>> wrote:
> >>>>
> >>>> Hi Bill!
> >>>>
> >>>> Any news? ( Please see my previous reply ...)
> >>>> Intr
> >>>
> >>>
> >>> Intrigued :) But my responses will be delayed, I personally
> >>> won't have time to look further myself until other backlogged
> >>> commitments to httpd are caught up some more.
> >>>
> >>> Be aware many of us have only a few hours to participate in the
> >>> httpd project itself, so it will take several days or more to gather
> >>> feedback on your interest and perhaps a champion to drive such
> >>> an effort if it is actively driven from this side. Of course that
> >>> should not stop you from assembling some ideas at the vcs of
> >>> your choice, and sharing a link for those who want to look at an
> >>> early draft of such an effort!
> >>>
> >>> Cheers,
> >>>
> >>> Bill
>
>
Re: collaboration request - apache server automation testing with
swat tool
Posted by Alexey Melezhik <me...@gmail.com>.
Hi Yann!
* per issues apache configurations *
https://github.com/entimelezhik/apache-swat/tree/master/htdocs is
document root , every issue thus is represented by directory inside
htdocs , named by issue number f.e.
- htdocs/44221
- htdocs/58777
If it's necessary per issue configuration defined by .htaccess files
placed at issues directories for example:
- htdocs/58777/.htaccess
All other static files are thus kept under issue directory , for example:
- htdocs/58777/data/foo.data
* global apache configuration ( httpd.conf )*
httpd.conf is generated by template file -
https://github.com/melezhik/apache-swat/blob/master/httpd.conf.tmpl -
this is vanilla apcahe2 config file taken from standard apach2e
installation ( https://httpd.apache.org/docs/2.2/en/install.html )
with some `LoadModule' lines uncommented to make it possible to run
tests for some none default enabled apache modules ( mod_rewrite ,
mod_include , ... )
* Initial test configuration *
Initial test configuration is trivial :
* install apache2 to /some/prefix/directory
$ git clone https://github.com/melezhik/apache-swat.git
$ cd apache-swat
# generate httpd.conf for your apache installation prefix
$ ./configure.bash /some/prefix/directory
# run apache with generated httpd.conf at CWD
$ sudo /some/prefix/directory/bin/apachectl -f `pwd`/httpd.conf
then you may choose to:
# run all tests
$ swat
# or run per-issue test
$ swat -t 58777
To summarize:
* per issues apache configurations based on related directories/htcaccess files
* one httpd.conf for all issues
* technically speaking it is possible to have multiple issues related
httpd.conf files ( one per issue ) , but this means:
- necessity of apache2 server restart at the beginning of every
issue test which has time impact ( or have many apache2 instances with
different httpd.conf which is hard to maintain )
- in many cases per issues apache2 configuration could be expressed
via directories/htaccess files which is much easier to maintain
Hope, this helps.
Regards
Alexey
2016-02-06 19:33 GMT+03:00 Yann Ylavic <yl...@gmail.com>:
> Hi Alexey,
>
> On Sat, Feb 6, 2016 at 5:03 PM, Alexey Melezhik <me...@gmail.com> wrote:
>> Sure, then it's probably won't be big deal to port existed test code base.
>>
>> Here I want to highlight some swat features which make it more then
>> just engine to make http requests and verify an output.
>
> How could we manage the httpd onfiguration files associated with each test?
>
> Regards,
> Yann.
Re: collaboration request - apache server automation testing with
swat tool
Posted by Yann Ylavic <yl...@gmail.com>.
Hi Alexey,
On Sat, Feb 6, 2016 at 5:03 PM, Alexey Melezhik <me...@gmail.com> wrote:
> Sure, then it's probably won't be big deal to port existed test code base.
>
> Here I want to highlight some swat features which make it more then
> just engine to make http requests and verify an output.
How could we manage the httpd onfiguration files associated with each test?
Regards,
Yann.
Re: collaboration request - apache server automation testing with
swat tool
Posted by Alexey Melezhik <me...@gmail.com>.
Sure, then it's probably won't be big deal to port existed test code base.
Here I want to highlight some swat features which make it more then
just engine to make http requests and verify an output.
Swat adds some features could be of interest in httpd2 testing, to
describe briefly a few:
- *Curl centric design *
Swat relies on curl heavily, constructing all http requests in a terms
of curl command line utility and then passing them to curl to make
http request. This a single swat test unit is based on more or many
http request made by curl. IMHO using curl has some essential benefits
for httpd2 testing, to list a few:
- often apache server testing implies low level http context
which is out of the box when using cur. Consider a few cases:
- making http2 protocol requests ( curl does! )
- setting explicit timeouts ( per initial connection and per
whole requests )
- issuing "raw" http requests with chunked/broken data to
tests how apache reacts on invalid requests
- construction of various html related entities ( forms ,
cookies , named parameters )
- various types of authentication works out of the box -
--basic, --digest, --ntlm, and --negotiate
- ssl related testing ( not relates to http but ... )
Also please take into account:
- curl is stable and well supported
- no need to install to many perl dependencies for http related stuff
- all the features granted by latest bleeding edge versions of curl
are here to use
- *Simple debugging*.
As swat uses curl to make http request. _basically_ a failing test
mean you may reproduce the same case manually using curl as you have
all you need here ( http request parameters get dumped into console )
In other words swat is much closer to what people usually do when
carry out manual testing - get a grip with curl, send command and
then see the output. Such an approach makes it swat's usage extremely
simple and understandable.
- *Sequential curl requests*.
As curl a request oriented ( only one single request could be made at
time ) it has some limits. As swat just uses curl it makes it possible
to create so called sequential/compound requests. When before one
request other request is issued. As trivial example of such a case is
user authentication. When one needs access for restricted http
resources and a preliminary authentication is required. So first we
need get logged in and only then request a restricted resource. Such a
compound requests on other hand are just a single test story for end
user and are hidden under the hood wheh using swat - see
https://github.com/melezhik/swat#upstream-and-downstream-stories for
details.
- * Text validation DSL *
And finally swat provides rich and simple DSL for text validation.
This simplifies server output verification and _often_ even does not
require a single line of perl code ( as it none perlish DSL ). In
other hand swat is extendable by perl to add desired complexity to
your tests. Take a look at examples - https://github.com/melezhik/swat
and https://github.com/melezhik/swat/tree/master/examples
Regards
Alexey
2016-02-06 17:34 GMT+03:00 Jim Jagielski <ji...@jagunet.com>:
>
>> On Feb 5, 2016, at 9:26 AM, Alexey Melezhik <me...@gmail.com> wrote:
>>
>>> I wonder how possible it would be to transcode the old tests to Swat?
>>
>> Swat has some architectural specifics. As it simply makes http
>> requests ( by curl ) and validate server response ( by swat DSL ) it
>> is considered as black box testing tools , there is no way to interact
>> with httpd internal structure. So probably most of old tests could be
>> easily ported but some probably not ( which implies something more
>> complicated ).
>>
>
> Most of the other tests w/ the old framework is the exact same...
> requests sent and responses verified against what is expected.
>
>> Basically swat could cover simple and average use cases, everything
>> that could expressed in terms of make http requests and then
>> analyzing an output , see examples for some issues I have expressed in
>> swat scenarios at https://github.com/melezhik/apache-swat
>>
>>
>> Also alternatively we may run 2 frameworks in parallel. Apache-swat is
>> intended for developers usage first and is able to run per-issues
>> test.
>>
>> This how I see possible workflow:
>>
>> * developer accept a bug request
>> * developer ( or some else ) create swat test for given bug/feature
>> * so we have `test first` approach - everyone involved in bug/feature
>> resolution has possibility to verify changes by simply running :
>>
>> swat -t $issue-id
>>
>>
>>
>>
>>
>>
>> 2016-02-05 17:08 GMT+03:00 Alexey Melezhik <me...@gmail.com>:
>>> 2016-02-05 17:01 GMT+03:00 Jim Jagielski <ji...@jagunet.com>:
>>>> Personally, I like the idea of having another framework;
>>>> the current one is OK but somewhat "painful" to update.
>>>
>>> Hi! What kind of possible issues you here ? when saying "painful" to
>>> update, please explain.
>>> Thanks.
>>>
>>> Also , please consider this link - https://github.com/melezhik/apache-swat and
>>> this post - http://mail-archives.apache.org/mod_mbox/httpd-dev/201602.mbox/%3CCAL7UUk-zqb1=7wDvCe8E-Ku=U=aY0MdJ_-JgCjVzZdvAranKQg@mail.gmail.com%3E
>>> as latest version of this idea ....
>>>
>>>
>>>
>>>
>>>
>>>>
>>>> I wonder how possible it would be to transcode the old tests
>>>> to Swat? We could then provide for 2 testing frameworks, one
>>>> developed by the ASF and the other external and 3rd party.
>>>>
>>>>
>>>>> On Feb 1, 2016, at 5:23 AM, Alexey Melezhik <me...@gmail.com> wrote:
>>>>>
>>>>> Here the list of existed issues I was able to automate tests for:
>>>>>
>>>>> vagrant@Debian-jessie-amd64-netboot:~/my/apache-swat$ ls -1 | grep -P '\d'
>>>>> 44221
>>>>> 46751
>>>>> 58789
>>>>> 58828
>>>>> 58854
>>>>>
>>>>>
>>>>> I have informed developers at https://bz.apache.org/bugzilla/ about
>>>>> these, so they rely upon them. Or you have some pre-release testing,
>>>>> so run all of these in a whole chunk?
>>>>>
>>>>> Regards
>>>>>
>>>>> Alexey
>>>>>
>>>>> PS chaining issues / tests listing as always -
>>>>> https://github.com/melezhik/apache-swat
>>>>>
>>>>> PS2 it'd be could to verify tests as well against "bleeding edge"
>>>>> apache version gets installed from SCM, but I don't know how to do
>>>>> this.
>>>>> Currently I use http://httpd.apache.org/download.cgi ( last stable release )
>>>>>
>>>>>
>>>>>
>>>>> 2016-01-30 21:36 GMT+03:00 Alexey Melezhik <me...@gmail.com>:
>>>>>> Hi Bill!
>>>>>>
>>>>>> I have started to assemble swat tests for apache2 issues coming from
>>>>>> https://bz.apache.org/bugzilla
>>>>>>
>>>>>> And this the first one - https://bz.apache.org/bugzilla/show_bug.cgi?id=44221 ,
>>>>>>
>>>>>> Test suite is failed for the moment :
>>>>>>
>>>>>>
>>>>>> vagrant@Debian-jessie-amd64-netboot:~/my/apache-swat$ swat -t 44221
>>>>>> /home/vagrant/.swat/.cache/384/prove/44221/foo.baz/00.GET.t ...
>>>>>> ok 1 - GET 127.0.0.1/44221/foo.baz succeeded
>>>>>> # http headers saved to /home/vagrant/.swat/.cache/384/prove/52jLAo76YZ.hdr
>>>>>> # body saved to /home/vagrant/.swat/.cache/384/prove/52jLAo76YZ
>>>>>> ok 2 - output match '200 OK'
>>>>>> 1..2
>>>>>> ok
>>>>>> /home/vagrant/.swat/.cache/384/prove/44221/FOO.bar/00.GET.t ...
>>>>>> ok 1 - GET 127.0.0.1/44221/FOO.bar succeeded
>>>>>> # http headers saved to /home/vagrant/.swat/.cache/384/prove/DvEZnO6ALe.hdr
>>>>>> # body saved to /home/vagrant/.swat/.cache/384/prove/DvEZnO6ALe
>>>>>> ok 2 - output match '200 OK'
>>>>>> ok 3 - output match /Location: \S+/
>>>>>> ok 4 - 'Location: http://127.0.0.1/44221/foo.bar' match 'foo.bar'
>>>>>> 1..4
>>>>>> ok
>>>>>> /home/vagrant/.swat/.cache/384/prove/44221/foo.BAR/00.GET.t ...
>>>>>> ok 1 - GET 127.0.0.1/44221/foo.BAR succeeded
>>>>>> # http headers saved to /home/vagrant/.swat/.cache/384/prove/bWQ1Sbl0YZ.hdr
>>>>>> # body saved to /home/vagrant/.swat/.cache/384/prove/bWQ1Sbl0YZ
>>>>>> ok 2 - output match '200 OK'
>>>>>> ok 3 - output match /Location: \S+/
>>>>>> ok 4 - 'Location: http://127.0.0.1/44221/foo.bar' match 'foo.bar'
>>>>>> 1..4
>>>>>> ok
>>>>>> /home/vagrant/.swat/.cache/384/prove/44221/foo.bar/00.GET.t ...
>>>>>> ok 1 - GET 127.0.0.1/44221/foo.bar succeeded
>>>>>> # http headers saved to /home/vagrant/.swat/.cache/384/prove/Jd_hlo7Qwv.hdr
>>>>>> # body saved to /home/vagrant/.swat/.cache/384/prove/Jd_hlo7Qwv
>>>>>> ok 2 - output match '200 OK'
>>>>>> 1..2
>>>>>> ok
>>>>>> /home/vagrant/.swat/.cache/384/prove/44221/foo.html/00.GET.t ..
>>>>>> ok 1 - GET 127.0.0.1/44221/foo.html succeeded
>>>>>> # http headers saved to /home/vagrant/.swat/.cache/384/prove/tW2L511eym.hdr
>>>>>> # body saved to /home/vagrant/.swat/.cache/384/prove/tW2L511eym
>>>>>> ok 2 - output match /HTTP\/(\S+) (\d+) \S+/
>>>>>> not ok 3 - 'HTTP/1.1 300 Multiple Choices' match /404 /
>>>>>>
>>>>>> # Failed test ''HTTP/1.1 300 Multiple Choices' match /404 /'
>>>>>> # at /usr/local/share/perl/5.20.2/swat.pm line 218.
>>>>>> not ok 4 - output match 'Not Found'
>>>>>>
>>>>>> # Failed test 'output match 'Not Found''
>>>>>> # at /usr/local/share/perl/5.20.2/swat.pm line 218.
>>>>>> 1..4
>>>>>> # Looks like you failed 2 tests of 4.
>>>>>> Dubious, test returned 2 (wstat 512, 0x200)
>>>>>> Failed 2/4 subtests
>>>>>>
>>>>>> Test Summary Report
>>>>>> -------------------
>>>>>> /home/vagrant/.swat/.cache/384/prove/44221/foo.html/00.GET.t (Wstat:
>>>>>> 512 Tests: 4 Failed: 2)
>>>>>> Failed tests: 3-4
>>>>>> Non-zero exit status: 2
>>>>>> Files=5, Tests=16, 0 wallclock secs ( 0.03 usr 0.00 sys + 0.27 cusr
>>>>>> 0.00 csys = 0.30 CPU)
>>>>>> Result: FAIL
>>>>>>
>>>>>> ...
>>>>>>
>>>>>> Ok, this is just the beginning ;-))) , watch soon updates at -
>>>>>> https://github.com/melezhik/apache-swat
>>>>>>
>>>>>> 2016-01-29 20:44 GMT+03:00 William A Rowe Jr <wr...@rowe-clan.net>:
>>>>>>> On Fri, Jan 29, 2016 at 6:35 AM, Alexey Melezhik <me...@gmail.com> wrote:
>>>>>>>>
>>>>>>>> Hi Bill!
>>>>>>>>
>>>>>>>> Any news? ( Please see my previous reply ...)
>>>>>>>> Intr
>>>>>>>
>>>>>>>
>>>>>>> Intrigued :) But my responses will be delayed, I personally
>>>>>>> won't have time to look further myself until other backlogged
>>>>>>> commitments to httpd are caught up some more.
>>>>>>>
>>>>>>> Be aware many of us have only a few hours to participate in the
>>>>>>> httpd project itself, so it will take several days or more to gather
>>>>>>> feedback on your interest and perhaps a champion to drive such
>>>>>>> an effort if it is actively driven from this side. Of course that
>>>>>>> should not stop you from assembling some ideas at the vcs of
>>>>>>> your choice, and sharing a link for those who want to look at an
>>>>>>> early draft of such an effort!
>>>>>>>
>>>>>>> Cheers,
>>>>>>>
>>>>>>> Bill
>>>>
>
Re: collaboration request - apache server automation testing with swat tool
Posted by Jim Jagielski <ji...@jaguNET.com>.
> On Feb 5, 2016, at 9:26 AM, Alexey Melezhik <me...@gmail.com> wrote:
>
>> I wonder how possible it would be to transcode the old tests to Swat?
>
> Swat has some architectural specifics. As it simply makes http
> requests ( by curl ) and validate server response ( by swat DSL ) it
> is considered as black box testing tools , there is no way to interact
> with httpd internal structure. So probably most of old tests could be
> easily ported but some probably not ( which implies something more
> complicated ).
>
Most of the other tests w/ the old framework is the exact same...
requests sent and responses verified against what is expected.
> Basically swat could cover simple and average use cases, everything
> that could expressed in terms of make http requests and then
> analyzing an output , see examples for some issues I have expressed in
> swat scenarios at https://github.com/melezhik/apache-swat
>
>
> Also alternatively we may run 2 frameworks in parallel. Apache-swat is
> intended for developers usage first and is able to run per-issues
> test.
>
> This how I see possible workflow:
>
> * developer accept a bug request
> * developer ( or some else ) create swat test for given bug/feature
> * so we have `test first` approach - everyone involved in bug/feature
> resolution has possibility to verify changes by simply running :
>
> swat -t $issue-id
>
>
>
>
>
>
> 2016-02-05 17:08 GMT+03:00 Alexey Melezhik <me...@gmail.com>:
>> 2016-02-05 17:01 GMT+03:00 Jim Jagielski <ji...@jagunet.com>:
>>> Personally, I like the idea of having another framework;
>>> the current one is OK but somewhat "painful" to update.
>>
>> Hi! What kind of possible issues you here ? when saying "painful" to
>> update, please explain.
>> Thanks.
>>
>> Also , please consider this link - https://github.com/melezhik/apache-swat and
>> this post - http://mail-archives.apache.org/mod_mbox/httpd-dev/201602.mbox/%3CCAL7UUk-zqb1=7wDvCe8E-Ku=U=aY0MdJ_-JgCjVzZdvAranKQg@mail.gmail.com%3E
>> as latest version of this idea ....
>>
>>
>>
>>
>>
>>>
>>> I wonder how possible it would be to transcode the old tests
>>> to Swat? We could then provide for 2 testing frameworks, one
>>> developed by the ASF and the other external and 3rd party.
>>>
>>>
>>>> On Feb 1, 2016, at 5:23 AM, Alexey Melezhik <me...@gmail.com> wrote:
>>>>
>>>> Here the list of existed issues I was able to automate tests for:
>>>>
>>>> vagrant@Debian-jessie-amd64-netboot:~/my/apache-swat$ ls -1 | grep -P '\d'
>>>> 44221
>>>> 46751
>>>> 58789
>>>> 58828
>>>> 58854
>>>>
>>>>
>>>> I have informed developers at https://bz.apache.org/bugzilla/ about
>>>> these, so they rely upon them. Or you have some pre-release testing,
>>>> so run all of these in a whole chunk?
>>>>
>>>> Regards
>>>>
>>>> Alexey
>>>>
>>>> PS chaining issues / tests listing as always -
>>>> https://github.com/melezhik/apache-swat
>>>>
>>>> PS2 it'd be could to verify tests as well against "bleeding edge"
>>>> apache version gets installed from SCM, but I don't know how to do
>>>> this.
>>>> Currently I use http://httpd.apache.org/download.cgi ( last stable release )
>>>>
>>>>
>>>>
>>>> 2016-01-30 21:36 GMT+03:00 Alexey Melezhik <me...@gmail.com>:
>>>>> Hi Bill!
>>>>>
>>>>> I have started to assemble swat tests for apache2 issues coming from
>>>>> https://bz.apache.org/bugzilla
>>>>>
>>>>> And this the first one - https://bz.apache.org/bugzilla/show_bug.cgi?id=44221 ,
>>>>>
>>>>> Test suite is failed for the moment :
>>>>>
>>>>>
>>>>> vagrant@Debian-jessie-amd64-netboot:~/my/apache-swat$ swat -t 44221
>>>>> /home/vagrant/.swat/.cache/384/prove/44221/foo.baz/00.GET.t ...
>>>>> ok 1 - GET 127.0.0.1/44221/foo.baz succeeded
>>>>> # http headers saved to /home/vagrant/.swat/.cache/384/prove/52jLAo76YZ.hdr
>>>>> # body saved to /home/vagrant/.swat/.cache/384/prove/52jLAo76YZ
>>>>> ok 2 - output match '200 OK'
>>>>> 1..2
>>>>> ok
>>>>> /home/vagrant/.swat/.cache/384/prove/44221/FOO.bar/00.GET.t ...
>>>>> ok 1 - GET 127.0.0.1/44221/FOO.bar succeeded
>>>>> # http headers saved to /home/vagrant/.swat/.cache/384/prove/DvEZnO6ALe.hdr
>>>>> # body saved to /home/vagrant/.swat/.cache/384/prove/DvEZnO6ALe
>>>>> ok 2 - output match '200 OK'
>>>>> ok 3 - output match /Location: \S+/
>>>>> ok 4 - 'Location: http://127.0.0.1/44221/foo.bar' match 'foo.bar'
>>>>> 1..4
>>>>> ok
>>>>> /home/vagrant/.swat/.cache/384/prove/44221/foo.BAR/00.GET.t ...
>>>>> ok 1 - GET 127.0.0.1/44221/foo.BAR succeeded
>>>>> # http headers saved to /home/vagrant/.swat/.cache/384/prove/bWQ1Sbl0YZ.hdr
>>>>> # body saved to /home/vagrant/.swat/.cache/384/prove/bWQ1Sbl0YZ
>>>>> ok 2 - output match '200 OK'
>>>>> ok 3 - output match /Location: \S+/
>>>>> ok 4 - 'Location: http://127.0.0.1/44221/foo.bar' match 'foo.bar'
>>>>> 1..4
>>>>> ok
>>>>> /home/vagrant/.swat/.cache/384/prove/44221/foo.bar/00.GET.t ...
>>>>> ok 1 - GET 127.0.0.1/44221/foo.bar succeeded
>>>>> # http headers saved to /home/vagrant/.swat/.cache/384/prove/Jd_hlo7Qwv.hdr
>>>>> # body saved to /home/vagrant/.swat/.cache/384/prove/Jd_hlo7Qwv
>>>>> ok 2 - output match '200 OK'
>>>>> 1..2
>>>>> ok
>>>>> /home/vagrant/.swat/.cache/384/prove/44221/foo.html/00.GET.t ..
>>>>> ok 1 - GET 127.0.0.1/44221/foo.html succeeded
>>>>> # http headers saved to /home/vagrant/.swat/.cache/384/prove/tW2L511eym.hdr
>>>>> # body saved to /home/vagrant/.swat/.cache/384/prove/tW2L511eym
>>>>> ok 2 - output match /HTTP\/(\S+) (\d+) \S+/
>>>>> not ok 3 - 'HTTP/1.1 300 Multiple Choices' match /404 /
>>>>>
>>>>> # Failed test ''HTTP/1.1 300 Multiple Choices' match /404 /'
>>>>> # at /usr/local/share/perl/5.20.2/swat.pm line 218.
>>>>> not ok 4 - output match 'Not Found'
>>>>>
>>>>> # Failed test 'output match 'Not Found''
>>>>> # at /usr/local/share/perl/5.20.2/swat.pm line 218.
>>>>> 1..4
>>>>> # Looks like you failed 2 tests of 4.
>>>>> Dubious, test returned 2 (wstat 512, 0x200)
>>>>> Failed 2/4 subtests
>>>>>
>>>>> Test Summary Report
>>>>> -------------------
>>>>> /home/vagrant/.swat/.cache/384/prove/44221/foo.html/00.GET.t (Wstat:
>>>>> 512 Tests: 4 Failed: 2)
>>>>> Failed tests: 3-4
>>>>> Non-zero exit status: 2
>>>>> Files=5, Tests=16, 0 wallclock secs ( 0.03 usr 0.00 sys + 0.27 cusr
>>>>> 0.00 csys = 0.30 CPU)
>>>>> Result: FAIL
>>>>>
>>>>> ...
>>>>>
>>>>> Ok, this is just the beginning ;-))) , watch soon updates at -
>>>>> https://github.com/melezhik/apache-swat
>>>>>
>>>>> 2016-01-29 20:44 GMT+03:00 William A Rowe Jr <wr...@rowe-clan.net>:
>>>>>> On Fri, Jan 29, 2016 at 6:35 AM, Alexey Melezhik <me...@gmail.com> wrote:
>>>>>>>
>>>>>>> Hi Bill!
>>>>>>>
>>>>>>> Any news? ( Please see my previous reply ...)
>>>>>>> Intr
>>>>>>
>>>>>>
>>>>>> Intrigued :) But my responses will be delayed, I personally
>>>>>> won't have time to look further myself until other backlogged
>>>>>> commitments to httpd are caught up some more.
>>>>>>
>>>>>> Be aware many of us have only a few hours to participate in the
>>>>>> httpd project itself, so it will take several days or more to gather
>>>>>> feedback on your interest and perhaps a champion to drive such
>>>>>> an effort if it is actively driven from this side. Of course that
>>>>>> should not stop you from assembling some ideas at the vcs of
>>>>>> your choice, and sharing a link for those who want to look at an
>>>>>> early draft of such an effort!
>>>>>>
>>>>>> Cheers,
>>>>>>
>>>>>> Bill
>>>
Re: collaboration request - apache server automation testing with
swat tool
Posted by Alexey Melezhik <me...@gmail.com>.
> I wonder how possible it would be to transcode the old tests to Swat?
Swat has some architectural specifics. As it simply makes http
requests ( by curl ) and validate server response ( by swat DSL ) it
is considered as black box testing tools , there is no way to interact
with httpd internal structure. So probably most of old tests could be
easily ported but some probably not ( which implies something more
complicated ).
Basically swat could cover simple and average use cases, everything
that could expressed in terms of make http requests and then
analyzing an output , see examples for some issues I have expressed in
swat scenarios at https://github.com/melezhik/apache-swat
Also alternatively we may run 2 frameworks in parallel. Apache-swat is
intended for developers usage first and is able to run per-issues
test.
This how I see possible workflow:
* developer accept a bug request
* developer ( or some else ) create swat test for given bug/feature
* so we have `test first` approach - everyone involved in bug/feature
resolution has possibility to verify changes by simply running :
swat -t $issue-id
2016-02-05 17:08 GMT+03:00 Alexey Melezhik <me...@gmail.com>:
> 2016-02-05 17:01 GMT+03:00 Jim Jagielski <ji...@jagunet.com>:
>> Personally, I like the idea of having another framework;
>> the current one is OK but somewhat "painful" to update.
>
> Hi! What kind of possible issues you here ? when saying "painful" to
> update, please explain.
> Thanks.
>
> Also , please consider this link - https://github.com/melezhik/apache-swat and
> this post - http://mail-archives.apache.org/mod_mbox/httpd-dev/201602.mbox/%3CCAL7UUk-zqb1=7wDvCe8E-Ku=U=aY0MdJ_-JgCjVzZdvAranKQg@mail.gmail.com%3E
> as latest version of this idea ....
>
>
>
>
>
>>
>> I wonder how possible it would be to transcode the old tests
>> to Swat? We could then provide for 2 testing frameworks, one
>> developed by the ASF and the other external and 3rd party.
>>
>>
>>> On Feb 1, 2016, at 5:23 AM, Alexey Melezhik <me...@gmail.com> wrote:
>>>
>>> Here the list of existed issues I was able to automate tests for:
>>>
>>> vagrant@Debian-jessie-amd64-netboot:~/my/apache-swat$ ls -1 | grep -P '\d'
>>> 44221
>>> 46751
>>> 58789
>>> 58828
>>> 58854
>>>
>>>
>>> I have informed developers at https://bz.apache.org/bugzilla/ about
>>> these, so they rely upon them. Or you have some pre-release testing,
>>> so run all of these in a whole chunk?
>>>
>>> Regards
>>>
>>> Alexey
>>>
>>> PS chaining issues / tests listing as always -
>>> https://github.com/melezhik/apache-swat
>>>
>>> PS2 it'd be could to verify tests as well against "bleeding edge"
>>> apache version gets installed from SCM, but I don't know how to do
>>> this.
>>> Currently I use http://httpd.apache.org/download.cgi ( last stable release )
>>>
>>>
>>>
>>> 2016-01-30 21:36 GMT+03:00 Alexey Melezhik <me...@gmail.com>:
>>>> Hi Bill!
>>>>
>>>> I have started to assemble swat tests for apache2 issues coming from
>>>> https://bz.apache.org/bugzilla
>>>>
>>>> And this the first one - https://bz.apache.org/bugzilla/show_bug.cgi?id=44221 ,
>>>>
>>>> Test suite is failed for the moment :
>>>>
>>>>
>>>> vagrant@Debian-jessie-amd64-netboot:~/my/apache-swat$ swat -t 44221
>>>> /home/vagrant/.swat/.cache/384/prove/44221/foo.baz/00.GET.t ...
>>>> ok 1 - GET 127.0.0.1/44221/foo.baz succeeded
>>>> # http headers saved to /home/vagrant/.swat/.cache/384/prove/52jLAo76YZ.hdr
>>>> # body saved to /home/vagrant/.swat/.cache/384/prove/52jLAo76YZ
>>>> ok 2 - output match '200 OK'
>>>> 1..2
>>>> ok
>>>> /home/vagrant/.swat/.cache/384/prove/44221/FOO.bar/00.GET.t ...
>>>> ok 1 - GET 127.0.0.1/44221/FOO.bar succeeded
>>>> # http headers saved to /home/vagrant/.swat/.cache/384/prove/DvEZnO6ALe.hdr
>>>> # body saved to /home/vagrant/.swat/.cache/384/prove/DvEZnO6ALe
>>>> ok 2 - output match '200 OK'
>>>> ok 3 - output match /Location: \S+/
>>>> ok 4 - 'Location: http://127.0.0.1/44221/foo.bar' match 'foo.bar'
>>>> 1..4
>>>> ok
>>>> /home/vagrant/.swat/.cache/384/prove/44221/foo.BAR/00.GET.t ...
>>>> ok 1 - GET 127.0.0.1/44221/foo.BAR succeeded
>>>> # http headers saved to /home/vagrant/.swat/.cache/384/prove/bWQ1Sbl0YZ.hdr
>>>> # body saved to /home/vagrant/.swat/.cache/384/prove/bWQ1Sbl0YZ
>>>> ok 2 - output match '200 OK'
>>>> ok 3 - output match /Location: \S+/
>>>> ok 4 - 'Location: http://127.0.0.1/44221/foo.bar' match 'foo.bar'
>>>> 1..4
>>>> ok
>>>> /home/vagrant/.swat/.cache/384/prove/44221/foo.bar/00.GET.t ...
>>>> ok 1 - GET 127.0.0.1/44221/foo.bar succeeded
>>>> # http headers saved to /home/vagrant/.swat/.cache/384/prove/Jd_hlo7Qwv.hdr
>>>> # body saved to /home/vagrant/.swat/.cache/384/prove/Jd_hlo7Qwv
>>>> ok 2 - output match '200 OK'
>>>> 1..2
>>>> ok
>>>> /home/vagrant/.swat/.cache/384/prove/44221/foo.html/00.GET.t ..
>>>> ok 1 - GET 127.0.0.1/44221/foo.html succeeded
>>>> # http headers saved to /home/vagrant/.swat/.cache/384/prove/tW2L511eym.hdr
>>>> # body saved to /home/vagrant/.swat/.cache/384/prove/tW2L511eym
>>>> ok 2 - output match /HTTP\/(\S+) (\d+) \S+/
>>>> not ok 3 - 'HTTP/1.1 300 Multiple Choices' match /404 /
>>>>
>>>> # Failed test ''HTTP/1.1 300 Multiple Choices' match /404 /'
>>>> # at /usr/local/share/perl/5.20.2/swat.pm line 218.
>>>> not ok 4 - output match 'Not Found'
>>>>
>>>> # Failed test 'output match 'Not Found''
>>>> # at /usr/local/share/perl/5.20.2/swat.pm line 218.
>>>> 1..4
>>>> # Looks like you failed 2 tests of 4.
>>>> Dubious, test returned 2 (wstat 512, 0x200)
>>>> Failed 2/4 subtests
>>>>
>>>> Test Summary Report
>>>> -------------------
>>>> /home/vagrant/.swat/.cache/384/prove/44221/foo.html/00.GET.t (Wstat:
>>>> 512 Tests: 4 Failed: 2)
>>>> Failed tests: 3-4
>>>> Non-zero exit status: 2
>>>> Files=5, Tests=16, 0 wallclock secs ( 0.03 usr 0.00 sys + 0.27 cusr
>>>> 0.00 csys = 0.30 CPU)
>>>> Result: FAIL
>>>>
>>>> ...
>>>>
>>>> Ok, this is just the beginning ;-))) , watch soon updates at -
>>>> https://github.com/melezhik/apache-swat
>>>>
>>>> 2016-01-29 20:44 GMT+03:00 William A Rowe Jr <wr...@rowe-clan.net>:
>>>>> On Fri, Jan 29, 2016 at 6:35 AM, Alexey Melezhik <me...@gmail.com> wrote:
>>>>>>
>>>>>> Hi Bill!
>>>>>>
>>>>>> Any news? ( Please see my previous reply ...)
>>>>>> Intr
>>>>>
>>>>>
>>>>> Intrigued :) But my responses will be delayed, I personally
>>>>> won't have time to look further myself until other backlogged
>>>>> commitments to httpd are caught up some more.
>>>>>
>>>>> Be aware many of us have only a few hours to participate in the
>>>>> httpd project itself, so it will take several days or more to gather
>>>>> feedback on your interest and perhaps a champion to drive such
>>>>> an effort if it is actively driven from this side. Of course that
>>>>> should not stop you from assembling some ideas at the vcs of
>>>>> your choice, and sharing a link for those who want to look at an
>>>>> early draft of such an effort!
>>>>>
>>>>> Cheers,
>>>>>
>>>>> Bill
>>
Re: collaboration request - apache server automation testing with
swat tool
Posted by Alexey Melezhik <me...@gmail.com>.
2016-02-05 17:01 GMT+03:00 Jim Jagielski <ji...@jagunet.com>:
> Personally, I like the idea of having another framework;
> the current one is OK but somewhat "painful" to update.
Hi! What kind of possible issues you here ? when saying "painful" to
update, please explain.
Thanks.
Also , please consider this link - https://github.com/melezhik/apache-swat and
this post - http://mail-archives.apache.org/mod_mbox/httpd-dev/201602.mbox/%3CCAL7UUk-zqb1=7wDvCe8E-Ku=U=aY0MdJ_-JgCjVzZdvAranKQg@mail.gmail.com%3E
as latest version of this idea ....
>
> I wonder how possible it would be to transcode the old tests
> to Swat? We could then provide for 2 testing frameworks, one
> developed by the ASF and the other external and 3rd party.
>
>
>> On Feb 1, 2016, at 5:23 AM, Alexey Melezhik <me...@gmail.com> wrote:
>>
>> Here the list of existed issues I was able to automate tests for:
>>
>> vagrant@Debian-jessie-amd64-netboot:~/my/apache-swat$ ls -1 | grep -P '\d'
>> 44221
>> 46751
>> 58789
>> 58828
>> 58854
>>
>>
>> I have informed developers at https://bz.apache.org/bugzilla/ about
>> these, so they rely upon them. Or you have some pre-release testing,
>> so run all of these in a whole chunk?
>>
>> Regards
>>
>> Alexey
>>
>> PS chaining issues / tests listing as always -
>> https://github.com/melezhik/apache-swat
>>
>> PS2 it'd be could to verify tests as well against "bleeding edge"
>> apache version gets installed from SCM, but I don't know how to do
>> this.
>> Currently I use http://httpd.apache.org/download.cgi ( last stable release )
>>
>>
>>
>> 2016-01-30 21:36 GMT+03:00 Alexey Melezhik <me...@gmail.com>:
>>> Hi Bill!
>>>
>>> I have started to assemble swat tests for apache2 issues coming from
>>> https://bz.apache.org/bugzilla
>>>
>>> And this the first one - https://bz.apache.org/bugzilla/show_bug.cgi?id=44221 ,
>>>
>>> Test suite is failed for the moment :
>>>
>>>
>>> vagrant@Debian-jessie-amd64-netboot:~/my/apache-swat$ swat -t 44221
>>> /home/vagrant/.swat/.cache/384/prove/44221/foo.baz/00.GET.t ...
>>> ok 1 - GET 127.0.0.1/44221/foo.baz succeeded
>>> # http headers saved to /home/vagrant/.swat/.cache/384/prove/52jLAo76YZ.hdr
>>> # body saved to /home/vagrant/.swat/.cache/384/prove/52jLAo76YZ
>>> ok 2 - output match '200 OK'
>>> 1..2
>>> ok
>>> /home/vagrant/.swat/.cache/384/prove/44221/FOO.bar/00.GET.t ...
>>> ok 1 - GET 127.0.0.1/44221/FOO.bar succeeded
>>> # http headers saved to /home/vagrant/.swat/.cache/384/prove/DvEZnO6ALe.hdr
>>> # body saved to /home/vagrant/.swat/.cache/384/prove/DvEZnO6ALe
>>> ok 2 - output match '200 OK'
>>> ok 3 - output match /Location: \S+/
>>> ok 4 - 'Location: http://127.0.0.1/44221/foo.bar' match 'foo.bar'
>>> 1..4
>>> ok
>>> /home/vagrant/.swat/.cache/384/prove/44221/foo.BAR/00.GET.t ...
>>> ok 1 - GET 127.0.0.1/44221/foo.BAR succeeded
>>> # http headers saved to /home/vagrant/.swat/.cache/384/prove/bWQ1Sbl0YZ.hdr
>>> # body saved to /home/vagrant/.swat/.cache/384/prove/bWQ1Sbl0YZ
>>> ok 2 - output match '200 OK'
>>> ok 3 - output match /Location: \S+/
>>> ok 4 - 'Location: http://127.0.0.1/44221/foo.bar' match 'foo.bar'
>>> 1..4
>>> ok
>>> /home/vagrant/.swat/.cache/384/prove/44221/foo.bar/00.GET.t ...
>>> ok 1 - GET 127.0.0.1/44221/foo.bar succeeded
>>> # http headers saved to /home/vagrant/.swat/.cache/384/prove/Jd_hlo7Qwv.hdr
>>> # body saved to /home/vagrant/.swat/.cache/384/prove/Jd_hlo7Qwv
>>> ok 2 - output match '200 OK'
>>> 1..2
>>> ok
>>> /home/vagrant/.swat/.cache/384/prove/44221/foo.html/00.GET.t ..
>>> ok 1 - GET 127.0.0.1/44221/foo.html succeeded
>>> # http headers saved to /home/vagrant/.swat/.cache/384/prove/tW2L511eym.hdr
>>> # body saved to /home/vagrant/.swat/.cache/384/prove/tW2L511eym
>>> ok 2 - output match /HTTP\/(\S+) (\d+) \S+/
>>> not ok 3 - 'HTTP/1.1 300 Multiple Choices' match /404 /
>>>
>>> # Failed test ''HTTP/1.1 300 Multiple Choices' match /404 /'
>>> # at /usr/local/share/perl/5.20.2/swat.pm line 218.
>>> not ok 4 - output match 'Not Found'
>>>
>>> # Failed test 'output match 'Not Found''
>>> # at /usr/local/share/perl/5.20.2/swat.pm line 218.
>>> 1..4
>>> # Looks like you failed 2 tests of 4.
>>> Dubious, test returned 2 (wstat 512, 0x200)
>>> Failed 2/4 subtests
>>>
>>> Test Summary Report
>>> -------------------
>>> /home/vagrant/.swat/.cache/384/prove/44221/foo.html/00.GET.t (Wstat:
>>> 512 Tests: 4 Failed: 2)
>>> Failed tests: 3-4
>>> Non-zero exit status: 2
>>> Files=5, Tests=16, 0 wallclock secs ( 0.03 usr 0.00 sys + 0.27 cusr
>>> 0.00 csys = 0.30 CPU)
>>> Result: FAIL
>>>
>>> ...
>>>
>>> Ok, this is just the beginning ;-))) , watch soon updates at -
>>> https://github.com/melezhik/apache-swat
>>>
>>> 2016-01-29 20:44 GMT+03:00 William A Rowe Jr <wr...@rowe-clan.net>:
>>>> On Fri, Jan 29, 2016 at 6:35 AM, Alexey Melezhik <me...@gmail.com> wrote:
>>>>>
>>>>> Hi Bill!
>>>>>
>>>>> Any news? ( Please see my previous reply ...)
>>>>> Intr
>>>>
>>>>
>>>> Intrigued :) But my responses will be delayed, I personally
>>>> won't have time to look further myself until other backlogged
>>>> commitments to httpd are caught up some more.
>>>>
>>>> Be aware many of us have only a few hours to participate in the
>>>> httpd project itself, so it will take several days or more to gather
>>>> feedback on your interest and perhaps a champion to drive such
>>>> an effort if it is actively driven from this side. Of course that
>>>> should not stop you from assembling some ideas at the vcs of
>>>> your choice, and sharing a link for those who want to look at an
>>>> early draft of such an effort!
>>>>
>>>> Cheers,
>>>>
>>>> Bill
>
Re: collaboration request - apache server automation testing with
swat tool
Posted by Alexey Melezhik <me...@gmail.com>.
Personally, I like the idea of having another framework;
the current one is OK but somewhat "painful" to update.
Ah, now I understand you, ))), you are talking about current framework.
пятница, 5 февраля 2016 г. пользователь Jim Jagielski написал:
> Personally, I like the idea of having another framework;
> the current one is OK but somewhat "painful" to update.
>
> I wonder how possible it would be to transcode the old tests
> to Swat? We could then provide for 2 testing frameworks, one
> developed by the ASF and the other external and 3rd party.
>
>
> > On Feb 1, 2016, at 5:23 AM, Alexey Melezhik <melezhik@gmail.com
> <javascript:;>> wrote:
> >
> > Here the list of existed issues I was able to automate tests for:
> >
> > vagrant@Debian-jessie-amd64-netboot:~/my/apache-swat$ ls -1 | grep -P
> '\d'
> > 44221
> > 46751
> > 58789
> > 58828
> > 58854
> >
> >
> > I have informed developers at https://bz.apache.org/bugzilla/ about
> > these, so they rely upon them. Or you have some pre-release testing,
> > so run all of these in a whole chunk?
> >
> > Regards
> >
> > Alexey
> >
> > PS chaining issues / tests listing as always -
> > https://github.com/melezhik/apache-swat
> >
> > PS2 it'd be could to verify tests as well against "bleeding edge"
> > apache version gets installed from SCM, but I don't know how to do
> > this.
> > Currently I use http://httpd.apache.org/download.cgi ( last stable
> release )
> >
> >
> >
> > 2016-01-30 21:36 GMT+03:00 Alexey Melezhik <melezhik@gmail.com
> <javascript:;>>:
> >> Hi Bill!
> >>
> >> I have started to assemble swat tests for apache2 issues coming from
> >> https://bz.apache.org/bugzilla
> >>
> >> And this the first one -
> https://bz.apache.org/bugzilla/show_bug.cgi?id=44221 ,
> >>
> >> Test suite is failed for the moment :
> >>
> >>
> >> vagrant@Debian-jessie-amd64-netboot:~/my/apache-swat$ swat -t 44221
> >> /home/vagrant/.swat/.cache/384/prove/44221/foo.baz/00.GET.t ...
> >> ok 1 - GET 127.0.0.1/44221/foo.baz succeeded
> >> # http headers saved to
> /home/vagrant/.swat/.cache/384/prove/52jLAo76YZ.hdr
> >> # body saved to /home/vagrant/.swat/.cache/384/prove/52jLAo76YZ
> >> ok 2 - output match '200 OK'
> >> 1..2
> >> ok
> >> /home/vagrant/.swat/.cache/384/prove/44221/FOO.bar/00.GET.t ...
> >> ok 1 - GET 127.0.0.1/44221/FOO.bar succeeded
> >> # http headers saved to
> /home/vagrant/.swat/.cache/384/prove/DvEZnO6ALe.hdr
> >> # body saved to /home/vagrant/.swat/.cache/384/prove/DvEZnO6ALe
> >> ok 2 - output match '200 OK'
> >> ok 3 - output match /Location: \S+/
> >> ok 4 - 'Location: http://127.0.0.1/44221/foo.bar' match 'foo.bar'
> >> 1..4
> >> ok
> >> /home/vagrant/.swat/.cache/384/prove/44221/foo.BAR/00.GET.t ...
> >> ok 1 - GET 127.0.0.1/44221/foo.BAR succeeded
> >> # http headers saved to
> /home/vagrant/.swat/.cache/384/prove/bWQ1Sbl0YZ.hdr
> >> # body saved to /home/vagrant/.swat/.cache/384/prove/bWQ1Sbl0YZ
> >> ok 2 - output match '200 OK'
> >> ok 3 - output match /Location: \S+/
> >> ok 4 - 'Location: http://127.0.0.1/44221/foo.bar' match 'foo.bar'
> >> 1..4
> >> ok
> >> /home/vagrant/.swat/.cache/384/prove/44221/foo.bar/00.GET.t ...
> >> ok 1 - GET 127.0.0.1/44221/foo.bar succeeded
> >> # http headers saved to
> /home/vagrant/.swat/.cache/384/prove/Jd_hlo7Qwv.hdr
> >> # body saved to /home/vagrant/.swat/.cache/384/prove/Jd_hlo7Qwv
> >> ok 2 - output match '200 OK'
> >> 1..2
> >> ok
> >> /home/vagrant/.swat/.cache/384/prove/44221/foo.html/00.GET.t ..
> >> ok 1 - GET 127.0.0.1/44221/foo.html succeeded
> >> # http headers saved to
> /home/vagrant/.swat/.cache/384/prove/tW2L511eym.hdr
> >> # body saved to /home/vagrant/.swat/.cache/384/prove/tW2L511eym
> >> ok 2 - output match /HTTP\/(\S+) (\d+) \S+/
> >> not ok 3 - 'HTTP/1.1 300 Multiple Choices' match /404 /
> >>
> >> # Failed test ''HTTP/1.1 300 Multiple Choices' match /404 /'
> >> # at /usr/local/share/perl/5.20.2/swat.pm line 218.
> >> not ok 4 - output match 'Not Found'
> >>
> >> # Failed test 'output match 'Not Found''
> >> # at /usr/local/share/perl/5.20.2/swat.pm line 218.
> >> 1..4
> >> # Looks like you failed 2 tests of 4.
> >> Dubious, test returned 2 (wstat 512, 0x200)
> >> Failed 2/4 subtests
> >>
> >> Test Summary Report
> >> -------------------
> >> /home/vagrant/.swat/.cache/384/prove/44221/foo.html/00.GET.t (Wstat:
> >> 512 Tests: 4 Failed: 2)
> >> Failed tests: 3-4
> >> Non-zero exit status: 2
> >> Files=5, Tests=16, 0 wallclock secs ( 0.03 usr 0.00 sys + 0.27 cusr
> >> 0.00 csys = 0.30 CPU)
> >> Result: FAIL
> >>
> >> ...
> >>
> >> Ok, this is just the beginning ;-))) , watch soon updates at -
> >> https://github.com/melezhik/apache-swat
> >>
> >> 2016-01-29 20:44 GMT+03:00 William A Rowe Jr <wrowe@rowe-clan.net
> <javascript:;>>:
> >>> On Fri, Jan 29, 2016 at 6:35 AM, Alexey Melezhik <melezhik@gmail.com
> <javascript:;>> wrote:
> >>>>
> >>>> Hi Bill!
> >>>>
> >>>> Any news? ( Please see my previous reply ...)
> >>>> Intr
> >>>
> >>>
> >>> Intrigued :) But my responses will be delayed, I personally
> >>> won't have time to look further myself until other backlogged
> >>> commitments to httpd are caught up some more.
> >>>
> >>> Be aware many of us have only a few hours to participate in the
> >>> httpd project itself, so it will take several days or more to gather
> >>> feedback on your interest and perhaps a champion to drive such
> >>> an effort if it is actively driven from this side. Of course that
> >>> should not stop you from assembling some ideas at the vcs of
> >>> your choice, and sharing a link for those who want to look at an
> >>> early draft of such an effort!
> >>>
> >>> Cheers,
> >>>
> >>> Bill
>
>
Re: collaboration request - apache server automation testing with swat tool
Posted by Jim Jagielski <ji...@jaguNET.com>.
Personally, I like the idea of having another framework;
the current one is OK but somewhat "painful" to update.
I wonder how possible it would be to transcode the old tests
to Swat? We could then provide for 2 testing frameworks, one
developed by the ASF and the other external and 3rd party.
> On Feb 1, 2016, at 5:23 AM, Alexey Melezhik <me...@gmail.com> wrote:
>
> Here the list of existed issues I was able to automate tests for:
>
> vagrant@Debian-jessie-amd64-netboot:~/my/apache-swat$ ls -1 | grep -P '\d'
> 44221
> 46751
> 58789
> 58828
> 58854
>
>
> I have informed developers at https://bz.apache.org/bugzilla/ about
> these, so they rely upon them. Or you have some pre-release testing,
> so run all of these in a whole chunk?
>
> Regards
>
> Alexey
>
> PS chaining issues / tests listing as always -
> https://github.com/melezhik/apache-swat
>
> PS2 it'd be could to verify tests as well against "bleeding edge"
> apache version gets installed from SCM, but I don't know how to do
> this.
> Currently I use http://httpd.apache.org/download.cgi ( last stable release )
>
>
>
> 2016-01-30 21:36 GMT+03:00 Alexey Melezhik <me...@gmail.com>:
>> Hi Bill!
>>
>> I have started to assemble swat tests for apache2 issues coming from
>> https://bz.apache.org/bugzilla
>>
>> And this the first one - https://bz.apache.org/bugzilla/show_bug.cgi?id=44221 ,
>>
>> Test suite is failed for the moment :
>>
>>
>> vagrant@Debian-jessie-amd64-netboot:~/my/apache-swat$ swat -t 44221
>> /home/vagrant/.swat/.cache/384/prove/44221/foo.baz/00.GET.t ...
>> ok 1 - GET 127.0.0.1/44221/foo.baz succeeded
>> # http headers saved to /home/vagrant/.swat/.cache/384/prove/52jLAo76YZ.hdr
>> # body saved to /home/vagrant/.swat/.cache/384/prove/52jLAo76YZ
>> ok 2 - output match '200 OK'
>> 1..2
>> ok
>> /home/vagrant/.swat/.cache/384/prove/44221/FOO.bar/00.GET.t ...
>> ok 1 - GET 127.0.0.1/44221/FOO.bar succeeded
>> # http headers saved to /home/vagrant/.swat/.cache/384/prove/DvEZnO6ALe.hdr
>> # body saved to /home/vagrant/.swat/.cache/384/prove/DvEZnO6ALe
>> ok 2 - output match '200 OK'
>> ok 3 - output match /Location: \S+/
>> ok 4 - 'Location: http://127.0.0.1/44221/foo.bar' match 'foo.bar'
>> 1..4
>> ok
>> /home/vagrant/.swat/.cache/384/prove/44221/foo.BAR/00.GET.t ...
>> ok 1 - GET 127.0.0.1/44221/foo.BAR succeeded
>> # http headers saved to /home/vagrant/.swat/.cache/384/prove/bWQ1Sbl0YZ.hdr
>> # body saved to /home/vagrant/.swat/.cache/384/prove/bWQ1Sbl0YZ
>> ok 2 - output match '200 OK'
>> ok 3 - output match /Location: \S+/
>> ok 4 - 'Location: http://127.0.0.1/44221/foo.bar' match 'foo.bar'
>> 1..4
>> ok
>> /home/vagrant/.swat/.cache/384/prove/44221/foo.bar/00.GET.t ...
>> ok 1 - GET 127.0.0.1/44221/foo.bar succeeded
>> # http headers saved to /home/vagrant/.swat/.cache/384/prove/Jd_hlo7Qwv.hdr
>> # body saved to /home/vagrant/.swat/.cache/384/prove/Jd_hlo7Qwv
>> ok 2 - output match '200 OK'
>> 1..2
>> ok
>> /home/vagrant/.swat/.cache/384/prove/44221/foo.html/00.GET.t ..
>> ok 1 - GET 127.0.0.1/44221/foo.html succeeded
>> # http headers saved to /home/vagrant/.swat/.cache/384/prove/tW2L511eym.hdr
>> # body saved to /home/vagrant/.swat/.cache/384/prove/tW2L511eym
>> ok 2 - output match /HTTP\/(\S+) (\d+) \S+/
>> not ok 3 - 'HTTP/1.1 300 Multiple Choices' match /404 /
>>
>> # Failed test ''HTTP/1.1 300 Multiple Choices' match /404 /'
>> # at /usr/local/share/perl/5.20.2/swat.pm line 218.
>> not ok 4 - output match 'Not Found'
>>
>> # Failed test 'output match 'Not Found''
>> # at /usr/local/share/perl/5.20.2/swat.pm line 218.
>> 1..4
>> # Looks like you failed 2 tests of 4.
>> Dubious, test returned 2 (wstat 512, 0x200)
>> Failed 2/4 subtests
>>
>> Test Summary Report
>> -------------------
>> /home/vagrant/.swat/.cache/384/prove/44221/foo.html/00.GET.t (Wstat:
>> 512 Tests: 4 Failed: 2)
>> Failed tests: 3-4
>> Non-zero exit status: 2
>> Files=5, Tests=16, 0 wallclock secs ( 0.03 usr 0.00 sys + 0.27 cusr
>> 0.00 csys = 0.30 CPU)
>> Result: FAIL
>>
>> ...
>>
>> Ok, this is just the beginning ;-))) , watch soon updates at -
>> https://github.com/melezhik/apache-swat
>>
>> 2016-01-29 20:44 GMT+03:00 William A Rowe Jr <wr...@rowe-clan.net>:
>>> On Fri, Jan 29, 2016 at 6:35 AM, Alexey Melezhik <me...@gmail.com> wrote:
>>>>
>>>> Hi Bill!
>>>>
>>>> Any news? ( Please see my previous reply ...)
>>>> Intr
>>>
>>>
>>> Intrigued :) But my responses will be delayed, I personally
>>> won't have time to look further myself until other backlogged
>>> commitments to httpd are caught up some more.
>>>
>>> Be aware many of us have only a few hours to participate in the
>>> httpd project itself, so it will take several days or more to gather
>>> feedback on your interest and perhaps a champion to drive such
>>> an effort if it is actively driven from this side. Of course that
>>> should not stop you from assembling some ideas at the vcs of
>>> your choice, and sharing a link for those who want to look at an
>>> early draft of such an effort!
>>>
>>> Cheers,
>>>
>>> Bill
Re: collaboration request - apache server automation testing with
swat tool
Posted by Alexey Melezhik <me...@gmail.com>.
Here the list of existed issues I was able to automate tests for:
vagrant@Debian-jessie-amd64-netboot:~/my/apache-swat$ ls -1 | grep -P '\d'
44221
46751
58789
58828
58854
I have informed developers at https://bz.apache.org/bugzilla/ about
these, so they rely upon them. Or you have some pre-release testing,
so run all of these in a whole chunk?
Regards
Alexey
PS chaining issues / tests listing as always -
https://github.com/melezhik/apache-swat
PS2 it'd be could to verify tests as well against "bleeding edge"
apache version gets installed from SCM, but I don't know how to do
this.
Currently I use http://httpd.apache.org/download.cgi ( last stable release )
2016-01-30 21:36 GMT+03:00 Alexey Melezhik <me...@gmail.com>:
> Hi Bill!
>
> I have started to assemble swat tests for apache2 issues coming from
> https://bz.apache.org/bugzilla
>
> And this the first one - https://bz.apache.org/bugzilla/show_bug.cgi?id=44221 ,
>
> Test suite is failed for the moment :
>
>
> vagrant@Debian-jessie-amd64-netboot:~/my/apache-swat$ swat -t 44221
> /home/vagrant/.swat/.cache/384/prove/44221/foo.baz/00.GET.t ...
> ok 1 - GET 127.0.0.1/44221/foo.baz succeeded
> # http headers saved to /home/vagrant/.swat/.cache/384/prove/52jLAo76YZ.hdr
> # body saved to /home/vagrant/.swat/.cache/384/prove/52jLAo76YZ
> ok 2 - output match '200 OK'
> 1..2
> ok
> /home/vagrant/.swat/.cache/384/prove/44221/FOO.bar/00.GET.t ...
> ok 1 - GET 127.0.0.1/44221/FOO.bar succeeded
> # http headers saved to /home/vagrant/.swat/.cache/384/prove/DvEZnO6ALe.hdr
> # body saved to /home/vagrant/.swat/.cache/384/prove/DvEZnO6ALe
> ok 2 - output match '200 OK'
> ok 3 - output match /Location: \S+/
> ok 4 - 'Location: http://127.0.0.1/44221/foo.bar' match 'foo.bar'
> 1..4
> ok
> /home/vagrant/.swat/.cache/384/prove/44221/foo.BAR/00.GET.t ...
> ok 1 - GET 127.0.0.1/44221/foo.BAR succeeded
> # http headers saved to /home/vagrant/.swat/.cache/384/prove/bWQ1Sbl0YZ.hdr
> # body saved to /home/vagrant/.swat/.cache/384/prove/bWQ1Sbl0YZ
> ok 2 - output match '200 OK'
> ok 3 - output match /Location: \S+/
> ok 4 - 'Location: http://127.0.0.1/44221/foo.bar' match 'foo.bar'
> 1..4
> ok
> /home/vagrant/.swat/.cache/384/prove/44221/foo.bar/00.GET.t ...
> ok 1 - GET 127.0.0.1/44221/foo.bar succeeded
> # http headers saved to /home/vagrant/.swat/.cache/384/prove/Jd_hlo7Qwv.hdr
> # body saved to /home/vagrant/.swat/.cache/384/prove/Jd_hlo7Qwv
> ok 2 - output match '200 OK'
> 1..2
> ok
> /home/vagrant/.swat/.cache/384/prove/44221/foo.html/00.GET.t ..
> ok 1 - GET 127.0.0.1/44221/foo.html succeeded
> # http headers saved to /home/vagrant/.swat/.cache/384/prove/tW2L511eym.hdr
> # body saved to /home/vagrant/.swat/.cache/384/prove/tW2L511eym
> ok 2 - output match /HTTP\/(\S+) (\d+) \S+/
> not ok 3 - 'HTTP/1.1 300 Multiple Choices' match /404 /
>
> # Failed test ''HTTP/1.1 300 Multiple Choices' match /404 /'
> # at /usr/local/share/perl/5.20.2/swat.pm line 218.
> not ok 4 - output match 'Not Found'
>
> # Failed test 'output match 'Not Found''
> # at /usr/local/share/perl/5.20.2/swat.pm line 218.
> 1..4
> # Looks like you failed 2 tests of 4.
> Dubious, test returned 2 (wstat 512, 0x200)
> Failed 2/4 subtests
>
> Test Summary Report
> -------------------
> /home/vagrant/.swat/.cache/384/prove/44221/foo.html/00.GET.t (Wstat:
> 512 Tests: 4 Failed: 2)
> Failed tests: 3-4
> Non-zero exit status: 2
> Files=5, Tests=16, 0 wallclock secs ( 0.03 usr 0.00 sys + 0.27 cusr
> 0.00 csys = 0.30 CPU)
> Result: FAIL
>
> ...
>
> Ok, this is just the beginning ;-))) , watch soon updates at -
> https://github.com/melezhik/apache-swat
>
> 2016-01-29 20:44 GMT+03:00 William A Rowe Jr <wr...@rowe-clan.net>:
>> On Fri, Jan 29, 2016 at 6:35 AM, Alexey Melezhik <me...@gmail.com> wrote:
>>>
>>> Hi Bill!
>>>
>>> Any news? ( Please see my previous reply ...)
>>> Intr
>>
>>
>> Intrigued :) But my responses will be delayed, I personally
>> won't have time to look further myself until other backlogged
>> commitments to httpd are caught up some more.
>>
>> Be aware many of us have only a few hours to participate in the
>> httpd project itself, so it will take several days or more to gather
>> feedback on your interest and perhaps a champion to drive such
>> an effort if it is actively driven from this side. Of course that
>> should not stop you from assembling some ideas at the vcs of
>> your choice, and sharing a link for those who want to look at an
>> early draft of such an effort!
>>
>> Cheers,
>>
>> Bill
Re: collaboration request - apache server automation testing with
swat tool
Posted by Alexey Melezhik <me...@gmail.com>.
Hi Bill!
I have started to assemble swat tests for apache2 issues coming from
https://bz.apache.org/bugzilla
And this the first one - https://bz.apache.org/bugzilla/show_bug.cgi?id=44221 ,
Test suite is failed for the moment :
vagrant@Debian-jessie-amd64-netboot:~/my/apache-swat$ swat -t 44221
/home/vagrant/.swat/.cache/384/prove/44221/foo.baz/00.GET.t ...
ok 1 - GET 127.0.0.1/44221/foo.baz succeeded
# http headers saved to /home/vagrant/.swat/.cache/384/prove/52jLAo76YZ.hdr
# body saved to /home/vagrant/.swat/.cache/384/prove/52jLAo76YZ
ok 2 - output match '200 OK'
1..2
ok
/home/vagrant/.swat/.cache/384/prove/44221/FOO.bar/00.GET.t ...
ok 1 - GET 127.0.0.1/44221/FOO.bar succeeded
# http headers saved to /home/vagrant/.swat/.cache/384/prove/DvEZnO6ALe.hdr
# body saved to /home/vagrant/.swat/.cache/384/prove/DvEZnO6ALe
ok 2 - output match '200 OK'
ok 3 - output match /Location: \S+/
ok 4 - 'Location: http://127.0.0.1/44221/foo.bar' match 'foo.bar'
1..4
ok
/home/vagrant/.swat/.cache/384/prove/44221/foo.BAR/00.GET.t ...
ok 1 - GET 127.0.0.1/44221/foo.BAR succeeded
# http headers saved to /home/vagrant/.swat/.cache/384/prove/bWQ1Sbl0YZ.hdr
# body saved to /home/vagrant/.swat/.cache/384/prove/bWQ1Sbl0YZ
ok 2 - output match '200 OK'
ok 3 - output match /Location: \S+/
ok 4 - 'Location: http://127.0.0.1/44221/foo.bar' match 'foo.bar'
1..4
ok
/home/vagrant/.swat/.cache/384/prove/44221/foo.bar/00.GET.t ...
ok 1 - GET 127.0.0.1/44221/foo.bar succeeded
# http headers saved to /home/vagrant/.swat/.cache/384/prove/Jd_hlo7Qwv.hdr
# body saved to /home/vagrant/.swat/.cache/384/prove/Jd_hlo7Qwv
ok 2 - output match '200 OK'
1..2
ok
/home/vagrant/.swat/.cache/384/prove/44221/foo.html/00.GET.t ..
ok 1 - GET 127.0.0.1/44221/foo.html succeeded
# http headers saved to /home/vagrant/.swat/.cache/384/prove/tW2L511eym.hdr
# body saved to /home/vagrant/.swat/.cache/384/prove/tW2L511eym
ok 2 - output match /HTTP\/(\S+) (\d+) \S+/
not ok 3 - 'HTTP/1.1 300 Multiple Choices' match /404 /
# Failed test ''HTTP/1.1 300 Multiple Choices' match /404 /'
# at /usr/local/share/perl/5.20.2/swat.pm line 218.
not ok 4 - output match 'Not Found'
# Failed test 'output match 'Not Found''
# at /usr/local/share/perl/5.20.2/swat.pm line 218.
1..4
# Looks like you failed 2 tests of 4.
Dubious, test returned 2 (wstat 512, 0x200)
Failed 2/4 subtests
Test Summary Report
-------------------
/home/vagrant/.swat/.cache/384/prove/44221/foo.html/00.GET.t (Wstat:
512 Tests: 4 Failed: 2)
Failed tests: 3-4
Non-zero exit status: 2
Files=5, Tests=16, 0 wallclock secs ( 0.03 usr 0.00 sys + 0.27 cusr
0.00 csys = 0.30 CPU)
Result: FAIL
...
Ok, this is just the beginning ;-))) , watch soon updates at -
https://github.com/melezhik/apache-swat
2016-01-29 20:44 GMT+03:00 William A Rowe Jr <wr...@rowe-clan.net>:
> On Fri, Jan 29, 2016 at 6:35 AM, Alexey Melezhik <me...@gmail.com> wrote:
>>
>> Hi Bill!
>>
>> Any news? ( Please see my previous reply ...)
>> Intr
>
>
> Intrigued :) But my responses will be delayed, I personally
> won't have time to look further myself until other backlogged
> commitments to httpd are caught up some more.
>
> Be aware many of us have only a few hours to participate in the
> httpd project itself, so it will take several days or more to gather
> feedback on your interest and perhaps a champion to drive such
> an effort if it is actively driven from this side. Of course that
> should not stop you from assembling some ideas at the vcs of
> your choice, and sharing a link for those who want to look at an
> early draft of such an effort!
>
> Cheers,
>
> Bill
Re: collaboration request - apache server automation testing with
swat tool
Posted by William A Rowe Jr <wr...@rowe-clan.net>.
On Fri, Jan 29, 2016 at 6:35 AM, Alexey Melezhik <me...@gmail.com> wrote:
> Hi Bill!
>
> Any news? ( Please see my previous reply ...)
> Intr
Intrigued :) But my responses will be delayed, I personally
won't have time to look further myself until other backlogged
commitments to httpd are caught up some more.
Be aware many of us have only a few hours to participate in the
httpd project itself, so it will take several days or more to gather
feedback on your interest and perhaps a champion to drive such
an effort if it is actively driven from this side. Of course that
should not stop you from assembling some ideas at the vcs of
your choice, and sharing a link for those who want to look at an
early draft of such an effort!
Cheers,
Bill
Re: collaboration request - apache server automation testing with
swat tool
Posted by Alexey Melezhik <me...@gmail.com>.
Hi Bill!
Any news? ( Please see my previous reply ...)
Thanks
2016-01-28 5:33 GMT+03:00 William A Rowe Jr <wr...@rowe-clan.net>:
> On Wed, Jan 27, 2016 at 2:53 PM, Alexey Melezhik <me...@gmail.com> wrote:
>>
>> Hi! I am the author of swat - https://github.com/melezhik/swat -
>> perl/curl based DSL for web services test automation.
>> I'd like to contribute in at apache server automated testing. Please
>> take a look at _simple_ example of swat test suite for apache web
>> server - https://sparrowhub.org/info/swat-apache?v=0.000004
>>
>> Some swat features ( more info could be found at
>> https://github.com/melezhik/swat ) which possibly could be of
>> interested for you:
>>
>> * swat is DSL for rapid test scenarios development for various web
>> services and application
>> * swat is written on Perl and could be extended by Perl if you need
>> something more complicated when DSL is not enough
>> * swat has minimal dependencies ( mostly core perl modules plus few
>> CPAN ones ) and easy to install
>> * swat relies on perl prove as internal test runner and so yield test
>> reports in portable TAP format
>> * swat relies on curl to make http requests against tested
>> application, so there is almost no limit for you - sending, saving
>> cookies, complicated http headers, make html form uploads, etc
>> * swat allow both simple (smoke) tests development and complicated
>> integration tests - with sequential, coupled http requests, saving
>> intermediate state, code reuse, etc. This is classical program API for
>> complex things combined with intuitive simple DSL ( for simple and
>> medium things )
>>
>> * not only httpd server but other a Apache Foundation products could
>> be tested with swat ( tomcat, ... ) - take a look at
>> https://sparrowhub.org/ - central swat test suite repository to get
>> more examples.
>>
>> I'd be glad to get feedback from you guys.
>>
>> Feel free to ask any further questions.
>>
>> Alexey Melezhik - the author of swat and sparrow.
>
>
> Thanks for the pointers and your efforts. IMHO more suites are
> always a benefit, not a competition between test tools.
>
> For a sense of what we actively review today, please do take a quick
> look at http://svn.apache.org/repos/asf/httpd/test/framework/trunk/
> to understand the scope of the current framework.
>
> I personally don't care precisely which tool or framework I use, as long
> as it is comprehensive. I'm looking forward to looking at what you have
> put together, but wanted you to be aware that it will be a challenge to
> bring the resource to bear to replicate what is being done right now
> under our existing framework.
>
> Props and cheers,
>
> Bill
>
Re: collaboration request - apache server automation testing with
swat tool
Posted by William A Rowe Jr <wr...@rowe-clan.net>.
On Wed, Jan 27, 2016 at 2:53 PM, Alexey Melezhik <me...@gmail.com> wrote:
> Hi! I am the author of swat - https://github.com/melezhik/swat -
> perl/curl based DSL for web services test automation.
> I'd like to contribute in at apache server automated testing. Please
> take a look at _simple_ example of swat test suite for apache web
> server - https://sparrowhub.org/info/swat-apache?v=0.000004
>
> Some swat features ( more info could be found at
> https://github.com/melezhik/swat ) which possibly could be of
> interested for you:
>
> * swat is DSL for rapid test scenarios development for various web
> services and application
> * swat is written on Perl and could be extended by Perl if you need
> something more complicated when DSL is not enough
> * swat has minimal dependencies ( mostly core perl modules plus few
> CPAN ones ) and easy to install
> * swat relies on perl prove as internal test runner and so yield test
> reports in portable TAP format
> * swat relies on curl to make http requests against tested
> application, so there is almost no limit for you - sending, saving
> cookies, complicated http headers, make html form uploads, etc
> * swat allow both simple (smoke) tests development and complicated
> integration tests - with sequential, coupled http requests, saving
> intermediate state, code reuse, etc. This is classical program API for
> complex things combined with intuitive simple DSL ( for simple and
> medium things )
>
> * not only httpd server but other a Apache Foundation products could
> be tested with swat ( tomcat, ... ) - take a look at
> https://sparrowhub.org/ - central swat test suite repository to get
> more examples.
>
> I'd be glad to get feedback from you guys.
>
> Feel free to ask any further questions.
>
> Alexey Melezhik - the author of swat and sparrow.
>
Thanks for the pointers and your efforts. IMHO more suites are
always a benefit, not a competition between test tools.
For a sense of what we actively review today, please do take a quick
look at http://svn.apache.org/repos/asf/httpd/test/framework/trunk/
to understand the scope of the current framework.
I personally don't care precisely which tool or framework I use, as long
as it is comprehensive. I'm looking forward to looking at what you have
put together, but wanted you to be aware that it will be a challenge to
bring the resource to bear to replicate what is being done right now
under our existing framework.
Props and cheers,
Bill