You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@thrift.apache.org by Anthony Molinaro <an...@alumni.caltech.edu> on 2011/06/18 05:34:46 UTC
Use rebar with erlang code?
Hi,
So I was considering removing some of the current erlang makefiles and
replacing them with rebar. Its a standalone escript which makes building
and testing erlang easier. The code and script are Apache 2 license
https://github.com/basho/rebar/blob/master/LICENSE so should be fine to
include inside of another apache license. The plan would be to remove
the Makefile.am in lib/erl/src, instead add rebar to lib/erl and add
the appropriate wrapper commands in the lib/erl/Makefile.am to compile
and run eunit tests. I'd also go ahead and move the test code which is
more unit test like into a test directory under lib/erl so that you
could use rebar to run eunit tests.
Anyone have a problem with this? The impetus is I want to merge in some
of the erlang patches we've received but would like to get some eunit test
for the current modules as well. I could hack more stuff into the current
Makefiles, but switching to rebar makes it much easier.
Thanks,
-Anthony
--
------------------------------------------------------------------------
Anthony Molinaro <an...@alumni.caltech.edu>
Re: Use rebar with erlang code?
Posted by David Nadlinger <co...@klickverbot.at>.
To make easily testing without manual intervention possible, it would be
great if the ThriftTest clients actually checked the results of the RPC
calls, instead of just printing them to the console like C++/Java
currently do.
Also, it would be great if either all the functions from the ThriftTest
interface only returned a copy from their arguments (testMapMap and
testInsanity currently don't), or the return values were the same across
all implementations (e.g. by defining them as constants in
ThriftTest.thrift), so that these checks could also be active when doing
cross-language testing.
David
On 6/20/11 8:18 PM, Roger Meier wrote:
>
>
>> -----Ursprüngliche Nachricht-----
>> Von: Anthony Molinaro [mailto:anthonym@alumni.caltech.edu]
>> Gesendet: Sonntag, 19. Juni 2011 21:17
>> An: dev@thrift.apache.org
>> Cc:<de...@thrift.apache.org>
>> Betreff: Re: AW: AW: Use rebar with erlang code?
>>
>>
>>> From my perspective everything should be moved from test/erl to
>>> lib/erl/test/
>>>
>>> The test directory should only contain test definitions like the
>>> ThriftTest.thrift
>>>>
>>>>>
>>>>
>>
>> Doesn't that sort of make testing harder if the thrift files are in a
> different
>> place from the tests? Also if cross language tests are ever implemented
>> where would you expect those to live?
>> I might suggest that pure unit tests (including thrift files for those
> tests) live
>> in the lib/*/test directories but the test servers and clients for cross
> language
>> testing continue to live in top level test directory (possibly with
> something
>> which runs all combinations against each other).
>>
>> -Anthony
>
> From my perspective test/ is the directory that keeps all IDL files for
> UnitTests, which will be used across languages.
> e.g. the ThriftTest.thrift is used by several unit tests
>
> I think it's much easier to build all the tests when they are part of the
> usual build procedure of a specific language.
> It's probably a bit difficult to keep test under test/ in sync with the
> build process of the library,
> that's why a propose to put all the tests in lib/<lang>/test/
>
> The get a cross language test capability I suggest the following approach:
> - The language specific build procedures should create an executable with
> the name TestClient and TestServer
> - They should be placed under test/<lang>/ or a shell script with the same
> name calls corresponding Client or Server
> - The test client and servers should have a unified set of arguments, so
> that cross language tests could easily be automated, e.g.
> roger@slave:~/software/thrift/thrift-trunk$ ./test/cpp/TestServer -h
> Allowed options:
> -h [ --help ] produce help message
> --port arg (=9090) Port number to listen
> --domain-socket arg Unix Domain Socket (e.g.
> /tmp/ThriftTest.thrift)
> --server-type arg (=simple) type of server, "simple", "thread-pool",
> "threaded", or "nonblocking"
> --transport arg (=buffered) transport: buffered, framed, http
> --protocol arg (=binary) protocol: binary, json
> --ssl Encrypted Transport
> using SSL
> --processor-events processor-events
> -n [ --workers ] arg (=4) Number of thread pools workers. Only
> valid for thread-pool server type
>
> roger@slave:~/software/thrift/thrift-trunk$ ./test/cpp/TestClient -h
> Allowed options:
> -h [ --help ] produce help message
> --host arg (=localhost) Host to connect
> --port arg (=9090) Port number to connect
> --domain-socket arg Domain Socket (e.g.
> /tmp/ThriftTest.thrift), instead of host and port
> --transport arg (=buffered) Transport: buffered, framed, http
> --protocol arg (=binary) Protocol: binary, json
> --ssl Encrypted Transport
> using SSL
> -n [ --testloops ] arg (=1) Number of Tests
>
>
> -Roger
>
>
>
Re: AW: AW: Use rebar with erlang code?
Posted by Anthony Molinaro <an...@alumni.caltech.edu>.
Okay, well I just committed the refactoring reusing THRIFT-892 which was
the original bug I was planning on using for refactoring to use
fw-template-erlang. However rebar is the better choice as it widely
used.
Let me know if anyone has any issues.
-Anthony
On Mon, Jun 20, 2011 at 11:33:46AM -0700, Anthony Molinaro wrote:
>
> On Mon, Jun 20, 2011 at 08:18:11PM +0200, Roger Meier wrote:
> > From my perspective test/ is the directory that keeps all IDL files for
> > UnitTests, which will be used across languages.
> > e.g. the ThriftTest.thrift is used by several unit tests
>
> Okay, well I can use relative paths to get to those share .thrift files,
> but what about .thrift files for specific unit tests. I assume keeping
> those in the lib/<lang>/test directory is fine.
>
> > I think it's much easier to build all the tests when they are part of the
> > usual build procedure of a specific language.
> > It's probably a bit difficult to keep test under test/ in sync with the
> > build process of the library,
> > that's why a propose to put all the tests in lib/<lang>/test/
>
> One thing here, in most cases the tests are run with 'make check' so its
> not always the case that 'make' will build the test code. And I think it
> should stay like that, unless you are actually running 'make check' it
> a bit of overhead to compile more stuff when you don't want it.
>
> > The get a cross language test capability I suggest the following approach:
> > - The language specific build procedures should create an executable with
> > the name TestClient and TestServer
> > - They should be placed under test/<lang>/ or a shell script with the same
> > name calls corresponding Client or Server
> > - The test client and servers should have a unified set of arguments, so
> > that cross language tests could easily be automated, e.g.
> > roger@slave:~/software/thrift/thrift-trunk$ ./test/cpp/TestServer -h
> > Allowed options:
> > -h [ --help ] produce help message
> > --port arg (=9090) Port number to listen
> > --domain-socket arg Unix Domain Socket (e.g.
> > /tmp/ThriftTest.thrift)
> > --server-type arg (=simple) type of server, "simple", "thread-pool",
> > "threaded", or "nonblocking"
> > --transport arg (=buffered) transport: buffered, framed, http
> > --protocol arg (=binary) protocol: binary, json
> > --ssl Encrypted Transport
> > using SSL
> > --processor-events processor-events
> > -n [ --workers ] arg (=4) Number of thread pools workers. Only
> > valid for thread-pool server type
> >
> > roger@slave:~/software/thrift/thrift-trunk$ ./test/cpp/TestClient -h
> > Allowed options:
> > -h [ --help ] produce help message
> > --host arg (=localhost) Host to connect
> > --port arg (=9090) Port number to connect
> > --domain-socket arg Domain Socket (e.g.
> > /tmp/ThriftTest.thrift), instead of host and port
> > --transport arg (=buffered) Transport: buffered, framed, http
> > --protocol arg (=binary) Protocol: binary, json
> > --ssl Encrypted Transport
> > using SSL
> > -n [ --testloops ] arg (=1) Number of Tests
>
> This seems reasonable, because of the ordering of SUBDIRS in the top-level
> Makefile.am, the lib hierarchy is traversed first whn you make check. So
> the code which the scripts in test/<lang>/ can then be run as the code
> they probably rely on will have been built. I'll structure the erlang
> code in this manner when I move it, and maybe try to get to these scripts
> in test/<lang>
>
> -Anthony
>
> --
> ------------------------------------------------------------------------
> Anthony Molinaro <an...@alumni.caltech.edu>
--
------------------------------------------------------------------------
Anthony Molinaro <an...@alumni.caltech.edu>
Re: AW: AW: Use rebar with erlang code?
Posted by Anthony Molinaro <an...@alumni.caltech.edu>.
On Mon, Jun 20, 2011 at 08:18:11PM +0200, Roger Meier wrote:
> From my perspective test/ is the directory that keeps all IDL files for
> UnitTests, which will be used across languages.
> e.g. the ThriftTest.thrift is used by several unit tests
Okay, well I can use relative paths to get to those share .thrift files,
but what about .thrift files for specific unit tests. I assume keeping
those in the lib/<lang>/test directory is fine.
> I think it's much easier to build all the tests when they are part of the
> usual build procedure of a specific language.
> It's probably a bit difficult to keep test under test/ in sync with the
> build process of the library,
> that's why a propose to put all the tests in lib/<lang>/test/
One thing here, in most cases the tests are run with 'make check' so its
not always the case that 'make' will build the test code. And I think it
should stay like that, unless you are actually running 'make check' it
a bit of overhead to compile more stuff when you don't want it.
> The get a cross language test capability I suggest the following approach:
> - The language specific build procedures should create an executable with
> the name TestClient and TestServer
> - They should be placed under test/<lang>/ or a shell script with the same
> name calls corresponding Client or Server
> - The test client and servers should have a unified set of arguments, so
> that cross language tests could easily be automated, e.g.
> roger@slave:~/software/thrift/thrift-trunk$ ./test/cpp/TestServer -h
> Allowed options:
> -h [ --help ] produce help message
> --port arg (=9090) Port number to listen
> --domain-socket arg Unix Domain Socket (e.g.
> /tmp/ThriftTest.thrift)
> --server-type arg (=simple) type of server, "simple", "thread-pool",
> "threaded", or "nonblocking"
> --transport arg (=buffered) transport: buffered, framed, http
> --protocol arg (=binary) protocol: binary, json
> --ssl Encrypted Transport
> using SSL
> --processor-events processor-events
> -n [ --workers ] arg (=4) Number of thread pools workers. Only
> valid for thread-pool server type
>
> roger@slave:~/software/thrift/thrift-trunk$ ./test/cpp/TestClient -h
> Allowed options:
> -h [ --help ] produce help message
> --host arg (=localhost) Host to connect
> --port arg (=9090) Port number to connect
> --domain-socket arg Domain Socket (e.g.
> /tmp/ThriftTest.thrift), instead of host and port
> --transport arg (=buffered) Transport: buffered, framed, http
> --protocol arg (=binary) Protocol: binary, json
> --ssl Encrypted Transport
> using SSL
> -n [ --testloops ] arg (=1) Number of Tests
This seems reasonable, because of the ordering of SUBDIRS in the top-level
Makefile.am, the lib hierarchy is traversed first whn you make check. So
the code which the scripts in test/<lang>/ can then be run as the code
they probably rely on will have been built. I'll structure the erlang
code in this manner when I move it, and maybe try to get to these scripts
in test/<lang>
-Anthony
--
------------------------------------------------------------------------
Anthony Molinaro <an...@alumni.caltech.edu>
AW: AW: AW: Use rebar with erlang code?
Posted by Roger Meier <ro...@bufferoverflow.ch>.
> -----Ursprüngliche Nachricht-----
> Von: Anthony Molinaro [mailto:anthonym@alumni.caltech.edu]
> Gesendet: Sonntag, 19. Juni 2011 21:17
> An: dev@thrift.apache.org
> Cc: <de...@thrift.apache.org>
> Betreff: Re: AW: AW: Use rebar with erlang code?
>
>
> > From my perspective everything should be moved from test/erl to
> > lib/erl/test/
> >
> > The test directory should only contain test definitions like the
> > ThriftTest.thrift
> >>
> >>>
> >>
>
> Doesn't that sort of make testing harder if the thrift files are in a
different
> place from the tests? Also if cross language tests are ever implemented
> where would you expect those to live?
> I might suggest that pure unit tests (including thrift files for those
tests) live
> in the lib/*/test directories but the test servers and clients for cross
language
> testing continue to live in top level test directory (possibly with
something
> which runs all combinations against each other).
>
> -Anthony
>From my perspective test/ is the directory that keeps all IDL files for
UnitTests, which will be used across languages.
e.g. the ThriftTest.thrift is used by several unit tests
I think it's much easier to build all the tests when they are part of the
usual build procedure of a specific language.
It's probably a bit difficult to keep test under test/ in sync with the
build process of the library,
that's why a propose to put all the tests in lib/<lang>/test/
The get a cross language test capability I suggest the following approach:
- The language specific build procedures should create an executable with
the name TestClient and TestServer
- They should be placed under test/<lang>/ or a shell script with the same
name calls corresponding Client or Server
- The test client and servers should have a unified set of arguments, so
that cross language tests could easily be automated, e.g.
roger@slave:~/software/thrift/thrift-trunk$ ./test/cpp/TestServer -h
Allowed options:
-h [ --help ] produce help message
--port arg (=9090) Port number to listen
--domain-socket arg Unix Domain Socket (e.g.
/tmp/ThriftTest.thrift)
--server-type arg (=simple) type of server, "simple", "thread-pool",
"threaded", or "nonblocking"
--transport arg (=buffered) transport: buffered, framed, http
--protocol arg (=binary) protocol: binary, json
--ssl Encrypted Transport
using SSL
--processor-events processor-events
-n [ --workers ] arg (=4) Number of thread pools workers. Only
valid for thread-pool server type
roger@slave:~/software/thrift/thrift-trunk$ ./test/cpp/TestClient -h
Allowed options:
-h [ --help ] produce help message
--host arg (=localhost) Host to connect
--port arg (=9090) Port number to connect
--domain-socket arg Domain Socket (e.g.
/tmp/ThriftTest.thrift), instead of host and port
--transport arg (=buffered) Transport: buffered, framed, http
--protocol arg (=binary) Protocol: binary, json
--ssl Encrypted Transport
using SSL
-n [ --testloops ] arg (=1) Number of Tests
-Roger
Re: AW: AW: Use rebar with erlang code?
Posted by Anthony Molinaro <an...@alumni.caltech.edu>.
> From my perspective everything should be moved from test/erl to
> lib/erl/test/
>
> The test directory should only contain test definitions like the
> ThriftTest.thrift
>>
>>>
>>
Doesn't that sort of make testing harder if the thrift files are in a different place from the tests? Also if cross language tests are ever implemented where would you expect those to live?
I might suggest that pure unit tests (including thrift files for those tests) live in the lib/*/test directories but the test servers and clients for cross language testing continue to live in top level test directory (possibly with something which runs all combinations against each other).
-Anthony
AW: AW: Use rebar with erlang code?
Posted by Roger Meier <ro...@bufferoverflow.ch>.
> -----Ursprüngliche Nachricht-----
> Von: Anthony Molinaro [mailto:anthonym@alumni.caltech.edu]
> Gesendet: Samstag, 18. Juni 2011 18:19
> An: dev@thrift.apache.org
> Cc: <de...@thrift.apache.org>
> Betreff: Re: AW: Use rebar with erlang code?
>
>
>
> On Jun 18, 2011, at 1:44 AM, "Roger Meier" <ro...@bufferoverflow.ch>
> wrote:
>
> > HI Anthony,
> >
> > It absolutely makes sense to use a more suitable tool for a specific
> > language, that's why we also have ant to build the java library.
> >
> > Some questions I have:
> > - How do you bring in the VERSION to the rebar build process? Via
> > Makefile.am as today?
>
> I did the change in my local repo last night and the way this works is
that I
> moved thrift.app.src to thrift.app.src.in so I can substitute VERSION that
way
> (so no weird sed just straight autoconf).
Great!
>
> > - Could you also move the testsuite from test/erl/ to lib/erl/test ?
> > See
> > https://issues.apache.org/jira/browse/THRIFT-35
>
> That was the plan. I was going to assume R13 or greater and convert the
> tests to eunit. Is there a reason to leave some tests in test/erl? It
seemed
> like unit type tests would move (so for instance the membuffer test but
the
> test client/server would not?)
>From my perspective everything should be moved from test/erl to
lib/erl/test/
The test directory should only contain test definitions like the
ThriftTest.thrift
>
> -Anthony
> >
>
Re: AW: Use rebar with erlang code?
Posted by Anthony Molinaro <an...@alumni.caltech.edu>.
On Jun 18, 2011, at 1:44 AM, "Roger Meier" <ro...@bufferoverflow.ch> wrote:
> HI Anthony,
>
> It absolutely makes sense to use a more suitable tool for a specific
> language, that's why we also have ant to build the java library.
>
> Some questions I have:
> - How do you bring in the VERSION to the rebar build process? Via
> Makefile.am as today?
I did the change in my local repo last night and the way this works is that I moved thrift.app.src to thrift.app.src.in so I can substitute VERSION that way (so no weird sed just straight autoconf).
> - Could you also move the testsuite from test/erl/ to lib/erl/test ? See
> https://issues.apache.org/jira/browse/THRIFT-35
That was the plan. I was going to assume R13 or greater and convert the tests to eunit. Is there a reason to leave some tests in test/erl? It seemed like unit type tests would move (so for instance the membuffer test but the test client/server would not?)
-Anthony
>
AW: Use rebar with erlang code?
Posted by Roger Meier <ro...@bufferoverflow.ch>.
HI Anthony,
It absolutely makes sense to use a more suitable tool for a specific
language, that's why we also have ant to build the java library.
Some questions I have:
- How do you bring in the VERSION to the rebar build process? Via
Makefile.am as today?
- Could you also move the testsuite from test/erl/ to lib/erl/test ? See
https://issues.apache.org/jira/browse/THRIFT-35
Regards
Roger
-----Ursprüngliche Nachricht-----
Von: Anthony Molinaro [mailto:anthonym@alumni.caltech.edu]
Gesendet: Samstag, 18. Juni 2011 05:35
An: dev@thrift.apache.org
Betreff: Use rebar with erlang code?
Hi,
So I was considering removing some of the current erlang makefiles and
replacing them with rebar. Its a standalone escript which makes building
and testing erlang easier. The code and script are Apache 2 license
https://github.com/basho/rebar/blob/master/LICENSE so should be fine to
include inside of another apache license. The plan would be to remove the
Makefile.am in lib/erl/src, instead add rebar to lib/erl and add the
appropriate wrapper commands in the lib/erl/Makefile.am to compile and run
eunit tests. I'd also go ahead and move the test code which is more unit
test like into a test directory under lib/erl so that you could use rebar to
run eunit tests.
Anyone have a problem with this? The impetus is I want to merge in some of
the erlang patches we've received but would like to get some eunit test for
the current modules as well. I could hack more stuff into the current
Makefiles, but switching to rebar makes it much easier.
Thanks,
-Anthony
--
------------------------------------------------------------------------
Anthony Molinaro <an...@alumni.caltech.edu>