etap test suite and OTP module http

[Filipe David Manana <fd...@apache.org>]

Hi all,

The module 'http', which is used by some etap tests, has been
deprecated in the OTP R14 series and will no longer exist in OTP R15,
soon to be released (actually it's a rename to 'httpc').

I started replacing all call to 'http' with the ibrowse equivalent
calls. However, specially for branch 1.1.x, the tests will fail often
with a badmatch exception like this:

[info] [<0.74.0>] 127.0.0.1 - - 'PUT'
/couch_test_atts_compression/doc_att_deflate/readme.txt 415
ok 76  - Couldn't create an already compressed attachment using the
unsupported encoding 'deflate'
# Test died abnormally: {'EXIT',
                       {{badmatch,
                         {error,
                          {'EXIT',
                           {normal,
                            {gen_server,call,
                             [<0.105.0>,
                              {send_req,
                               {{url,

"http://127.0.0.1:64038/couch_test_atts_compression/doc_att_compress/readme.txt",
                                 "127.0.0.1",64038,undefined,undefined,

"/couch_test_atts_compression/doc_att_compress/readme.txt",
                                 http,ipv4_address},
                                [{"Content-Encoding","compress"},
                                 {"Content-Type","text/plain"}],


This is something that can be avoided by retrying the request when
this specific error happens. For example, it's how the new replicator
deals with it:

https://github.com/apache/couchdb/blob/trunk/src/couchdb/couch_api_wrap_httpc.erl#L71

I'm thinking about doing the equivalent in the test suite by adding a
new function to test/etap/test_util.erl that will do this retry logic.
Another alternative is for that new function to use 'httpc' if 'http'
doesn't exists.

There are also a few places in our bundled etap which use 'http'.

Basically, as things are now, it means we won't be able to build couch
(because of the bundled etap) and run the etap tests on R15, which is
supposed to be released by the end of the year or early next year.

For 1.1.x, here's the current patch:

https://github.com/fdmanana/couchdb/commit/22fcbccb275885292ff1d349afbec3c7a543ff73


Anyone has a simpler idea in mind?


-- 
Filipe David Manana,

"Reasonable men adapt themselves to the world.
 Unreasonable men adapt the world to themselves.
 That's why all progress depends on unreasonable men."

Re: etap test suite and OTP module http

[Paul Davis <pa...@gmail.com>]

That sounds like a good plan to me. A better API for HTTP requests in
ETAP would be a welcome addition.

As to etap, its been refactored to be a single module so that you can
just drop it into the test directory and compile along with any
necessary test code. I've been meaning to make this update for awhile.
When I did this update I also ended up removing that http related code
so perhaps we should make the update now to future proof for R15.

On Sat, Sep 17, 2011 at 10:32 PM, Filipe David Manana
<fd...@apache.org> wrote:
> Hi all,
>
> The module 'http', which is used by some etap tests, has been
> deprecated in the OTP R14 series and will no longer exist in OTP R15,
> soon to be released (actually it's a rename to 'httpc').
>
> I started replacing all call to 'http' with the ibrowse equivalent
> calls. However, specially for branch 1.1.x, the tests will fail often
> with a badmatch exception like this:
>
> [info] [<0.74.0>] 127.0.0.1 - - 'PUT'
> /couch_test_atts_compression/doc_att_deflate/readme.txt 415
> ok 76  - Couldn't create an already compressed attachment using the
> unsupported encoding 'deflate'
> # Test died abnormally: {'EXIT',
>                       {{badmatch,
>                         {error,
>                          {'EXIT',
>                           {normal,
>                            {gen_server,call,
>                             [<0.105.0>,
>                              {send_req,
>                               {{url,
>
> "http://127.0.0.1:64038/couch_test_atts_compression/doc_att_compress/readme.txt",
>                                 "127.0.0.1",64038,undefined,undefined,
>
> "/couch_test_atts_compression/doc_att_compress/readme.txt",
>                                 http,ipv4_address},
>                                [{"Content-Encoding","compress"},
>                                 {"Content-Type","text/plain"}],
>
>
> This is something that can be avoided by retrying the request when
> this specific error happens. For example, it's how the new replicator
> deals with it:
>
> https://github.com/apache/couchdb/blob/trunk/src/couchdb/couch_api_wrap_httpc.erl#L71
>
> I'm thinking about doing the equivalent in the test suite by adding a
> new function to test/etap/test_util.erl that will do this retry logic.
> Another alternative is for that new function to use 'httpc' if 'http'
> doesn't exists.
>
> There are also a few places in our bundled etap which use 'http'.
>
> Basically, as things are now, it means we won't be able to build couch
> (because of the bundled etap) and run the etap tests on R15, which is
> supposed to be released by the end of the year or early next year.
>
> For 1.1.x, here's the current patch:
>
> https://github.com/fdmanana/couchdb/commit/22fcbccb275885292ff1d349afbec3c7a543ff73
>
>
> Anyone has a simpler idea in mind?
>
>
> --
> Filipe David Manana,
>
> "Reasonable men adapt themselves to the world.
>  Unreasonable men adapt the world to themselves.
>  That's why all progress depends on unreasonable men."
>