You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@avro.apache.org by "Thiruvalluvan M. G." <th...@yahoo.com> on 2009/04/11 13:58:38 UTC
Code reorg
Because these are early days of avro, it easier to get these changes
effected now rather than later:
1. Directory structure
There doesn't seem to be a de facto standard directory structure for these
kinds of projects. But many projects keep the main code and test code at the
same level. For example, protocol buffers keeps main and test code for Java
under java/src. On the other hand, thrift keeps them as top level as
compiler, lib and test and has directories under them for each language.
Avro keeps test under src, which is somewhat asymmetric. Is there a good
reason to keep that way? I'd prefer to follow the directory structure that
is friendly to maven, which is similar to what protocol buffers uses.
2. Standardize Java tests
The current code base uses JUnit3. It is preferable to move to either JUnit4
TestNG. I don't have much experience in TestNG. I find JUnit4 much more
convenient and expressive compared to JUnit3; this is particularly true for
parameterized tests. Many think TestNG is better than JUnit4.
If there's agreement that we should change these, I'll file JIRA for these
decide on the way to go and then implement the decision.
Thanks
Thiru
Re: TestNG in Avro [Was: Code reorg]
Posted by Doug Cutting <cu...@apache.org>.
Konstantin Boudnik wrote:
> While I was working on TestNG stuff I couldn't help thinking about getting
> replacing statically added libraries with Ivy dependencies tracking.
>
> Am I running ahead of the horse?
Not at all. It would be good to use Ivy.
Doug
Re: TestNG in Avro [Was: Code reorg]
Posted by Konstantin Boudnik <co...@yahoo-inc.com>.
While I was working on TestNG stuff I couldn't help thinking about getting
replacing statically added libraries with Ivy dependencies tracking.
Am I running ahead of the horse?
Cos
On Thu, Apr 30, 2009 at 09:56AM, Konstantin Boudnik wrote:
> I agree with this: maintaining the list isn't an option. However, for
> now it might work for the number of tests is really small.
>
> Besides, it shouldn't be an issue with 'native' TestNG tests, I believe.
>
> I'll find out what's going on there and report back shortly
> Cos
>
> Nigel Daley wrote:
> > On Apr 29, 2009, at 8:22 PM, Konstantin Boudnik wrote:
> >
> >> Hello all.
> >>
> >> Per Nigel's generous offer, I've spent some time in the attempt to
> >> move Avro's unit testing from JUnit to TestNG. Below is a short
> >> report.
> >>
> >> I've tried a couple of different approached which are listed below:
> >> 1) Complete test conversion using provided JUnitConverter. For a
> >> number of reasons it failed to work. One of the possible reasons my
> >> insufficient TestNG skills. Another possible reason is relative
> >> complexity of the tests, which is above JUnitConverter level of
> >> intellect.
> >
> > I really think option 1 is the only way to go. We need to look at why
> > it doesn't work.
> >
> > Maintaining an xml file of test cases is unwieldily in option 2.
> >
> > Cheers,
> > Nige
Re: TestNG in Avro [Was: Code reorg]
Posted by Konstantin Boudnik <co...@yahoo-inc.com>.
I agree with this: maintaining the list isn't an option. However, for
now it might work for the number of tests is really small.
Besides, it shouldn't be an issue with 'native' TestNG tests, I believe.
I'll find out what's going on there and report back shortly
Cos
Nigel Daley wrote:
> On Apr 29, 2009, at 8:22 PM, Konstantin Boudnik wrote:
>
>> Hello all.
>>
>> Per Nigel's generous offer, I've spent some time in the attempt to
>> move Avro's unit testing from JUnit to TestNG. Below is a short
>> report.
>>
>> I've tried a couple of different approached which are listed below:
>> 1) Complete test conversion using provided JUnitConverter. For a
>> number of reasons it failed to work. One of the possible reasons my
>> insufficient TestNG skills. Another possible reason is relative
>> complexity of the tests, which is above JUnitConverter level of
>> intellect.
>
> I really think option 1 is the only way to go. We need to look at why
> it doesn't work.
>
> Maintaining an xml file of test cases is unwieldily in option 2.
>
> Cheers,
> Nige
Re: TestNG in Avro [Was: Code reorg]
Posted by Nigel Daley <nd...@yahoo-inc.com>.
On Apr 29, 2009, at 8:22 PM, Konstantin Boudnik wrote:
> Hello all.
>
> Per Nigel's generous offer, I've spent some time in the attempt to
> move Avro's unit testing from JUnit to TestNG. Below is a short
> report.
>
> I've tried a couple of different approached which are listed below:
> 1) Complete test conversion using provided JUnitConverter. For a
> number of reasons it failed to work. One of the possible reasons my
> insufficient TestNG skills. Another possible reason is relative
> complexity of the tests, which is above JUnitConverter level of
> intellect.
I really think option 1 is the only way to go. We need to look at why
it doesn't work.
Maintaining an xml file of test cases is unwieldily in option 2.
Cheers,
Nige
Re: TestNG in Avro [Was: Code reorg]
Posted by Konstantin Boudnik <co...@yahoo-inc.com>.
It turns out that the issues were caused by this inheritance
public class TestProtocolReflect extends TestProtocolSpecific
where TestProtocolSpecific was declaring testStartServer() method as
@BeforeMethod and TestProtocolReflect has it has just @Test
When using @Before.../@After annotations the overall number of tests was
reduced to 34 from 42 because none of @Before.../@After accounted as
tests anymore.
Resolved and new patch is submitted. Review is highly welcomed! :-)
Cos
Konstantin Boudnik wrote:
> Thanks for the info, Doug.
>
> The failures were caused by these two issues:
> - the presence of generated Test interface which was clashing with
> org.testng.annotations.Test. To work around the problem I have renamed
> test.js to generated.js and refactored all code, which has been affected
> by this
> - RPC testing sequence as you've described below. After I've realized
> that certain tests require a server to be started upfront the rest of
> exercise was a breeze. I've put inter tests dependencies so the
> testStartServer() would be always called before any of the test methods
> is started.
>
> Both of these modifications are included into the AVRO-26 patch.
>
> Just now I have thought and tried another approached to guarantee
> execution sequence for RPC tests: I've added @BeforeMethod and
> @AfterMethod annotations to testStartServer() and testStopServer()
> respectively and removed hardcoded inter-tests dependencies. This
> approach has worked for at least two complex test classes:
> TestProtocolGeneric and TestFsData. In TestProtocolSpecific class
> this approach failed and I'll take a look to find out why it is so.
>
> I think technically AVRO is ready to be switched to TestNG for all
> current java tests are converted into the new framework.
>
> Cos
>
> Doug Cutting wrote:
>> Konstantin Boudnik wrote:
>>> 1) Complete test conversion using provided JUnitConverter. For a
>>> number of reasons it failed to work. One of the possible reasons my
>>> insufficient TestNG skills. Another possible reason is relative
>>> complexity of the tests, which is above JUnitConverter level of intellect.
>> How did it fail?
>>
>> Most of the tests don't push JUnit very hard and should be trivial. The
>> only one's that I'd expect to be a problem are the RPC tests. I tried
>> to use JUnit's setup/teardown methods to start and stop the daemon, but
>> could not get that to work for some reason. I ended up doing something
>> pretty ugly, which is, in each class, adding an initial test method that
>> starts a daemon, and a final one that kills the daemon, and a bunch in
>> between that use the daemon for tests. So perhaps the problem is
>> related to this, e.g., perhaps JUnitConverter doesn't run tests in the
>> right order or somesuch.
>>
>> Doug
>
Re: TestNG in Avro [Was: Code reorg]
Posted by Konstantin Boudnik <co...@yahoo-inc.com>.
Thanks for the info, Doug.
The failures were caused by these two issues:
- the presence of generated Test interface which was clashing with
org.testng.annotations.Test. To work around the problem I have renamed
test.js to generated.js and refactored all code, which has been affected
by this
- RPC testing sequence as you've described below. After I've realized
that certain tests require a server to be started upfront the rest of
exercise was a breeze. I've put inter tests dependencies so the
testStartServer() would be always called before any of the test methods
is started.
Both of these modifications are included into the AVRO-26 patch.
Just now I have thought and tried another approached to guarantee
execution sequence for RPC tests: I've added @BeforeMethod and
@AfterMethod annotations to testStartServer() and testStopServer()
respectively and removed hardcoded inter-tests dependencies. This
approach has worked for at least two complex test classes:
TestProtocolGeneric and TestFsData. In TestProtocolSpecific class
this approach failed and I'll take a look to find out why it is so.
I think technically AVRO is ready to be switched to TestNG for all
current java tests are converted into the new framework.
Cos
Doug Cutting wrote:
> Konstantin Boudnik wrote:
>> 1) Complete test conversion using provided JUnitConverter. For a
>> number of reasons it failed to work. One of the possible reasons my
>> insufficient TestNG skills. Another possible reason is relative
>> complexity of the tests, which is above JUnitConverter level of intellect.
>
> How did it fail?
>
> Most of the tests don't push JUnit very hard and should be trivial. The
> only one's that I'd expect to be a problem are the RPC tests. I tried
> to use JUnit's setup/teardown methods to start and stop the daemon, but
> could not get that to work for some reason. I ended up doing something
> pretty ugly, which is, in each class, adding an initial test method that
> starts a daemon, and a final one that kills the daemon, and a bunch in
> between that use the daemon for tests. So perhaps the problem is
> related to this, e.g., perhaps JUnitConverter doesn't run tests in the
> right order or somesuch.
>
> Doug
Re: TestNG in Avro [Was: Code reorg]
Posted by Doug Cutting <cu...@apache.org>.
Konstantin Boudnik wrote:
> 1) Complete test conversion using provided JUnitConverter. For a
> number of reasons it failed to work. One of the possible reasons my
> insufficient TestNG skills. Another possible reason is relative
> complexity of the tests, which is above JUnitConverter level of intellect.
How did it fail?
Most of the tests don't push JUnit very hard and should be trivial. The
only one's that I'd expect to be a problem are the RPC tests. I tried
to use JUnit's setup/teardown methods to start and stop the daemon, but
could not get that to work for some reason. I ended up doing something
pretty ugly, which is, in each class, adding an initial test method that
starts a daemon, and a final one that kills the daemon, and a bunch in
between that use the daemon for tests. So perhaps the problem is
related to this, e.g., perhaps JUnitConverter doesn't run tests in the
right order or somesuch.
Doug
TestNG in Avro [Was: Code reorg]
Posted by Konstantin Boudnik <co...@yahoo-inc.com>.
Hello all.
Per Nigel's generous offer, I've spent some time in the attempt to move
Avro's unit testing from JUnit to TestNG. Below is a short report.
I've tried a couple of different approached which are listed below:
1) Complete test conversion using provided JUnitConverter. For a
number of reasons it failed to work. One of the possible reasons my
insufficient TestNG skills. Another possible reason is relative
complexity of the tests, which is above JUnitConverter level of intellect.
2) The second approach was to run JUnit tests by TestNG without any
modifications. This approach has turned to be more productive, however a
manual execution of the tests was reporting 16 tests as failed. I assume
that some missed pieces of tests configuration were causing this. Kudos
my insufficient knowledge of Avro tests.
2.1) Running JUnit tests by TestNG using ant taskdef proved to be
relatively easy to setup and a total execution success. I was able to
run all 42 tests with 100% pass rate.
Why we might want to go with TestNG?
1) It provides richer control over test execution
2) It gives free parallel execution (as in many VM threads, not many
cluster nodes) of the tests at the level of classes, tests, or methods
3) It has an easy grouping of the tests, which potentially will
provide a way of combining tests into smoke, full, functional suites
with any kind of recompilations
4) Also it has an easy parametrization of test cases through
DataProviders mechanism
In order to start using TestNG right now instead of JUnit framework we'd
need a couple of things:
- to add testng.jar file to the classpath of the project (and to the
repository)
- create an XML listing of tests to be executed (simple selection of
**/*.class doesn't work in this case)
- make a couple of minor modifications of build.xml file
I'm willing to file JIRA on this and provide the patch, if this group
wants to move forward with this approach.
Ideally, I am willing to convert existing JUnit tests into "native"
TestNG ones so we can totally get rid of JUnit.
Now, I have a certain degree of confidence that somewhat similar
approach can be used for other Hadoop tests and provide us with all
listed and many other benefits on even greater scale.
I would appreciate any feedback.
Thanks in advance,
Konstantin (Cos) Boudnik
Nigel Daley wrote:
>>>
>>> Regarding Junit version and/or TestNG. Can we try TestNG in this
>>> project?
>> I'd be fine with that if someone wants to develop a patch and others
>> agree. Note that, for Avro, we need a test framework per
>> implementation, and TestNG/JUnit only help us with Java.
>
> Want to take on the challenge? Only a small number of tests currently
> in Avro. Maybe a good test for converting to Junit. Then we'll get
> feedback from Doug Cutting on whether he likes TestNG :-)
>
> n.
>
Re: Code reorg
Posted by Doug Cutting <cu...@apache.org>.
Nigel Daley wrote:
> I think we should try to keep the directory structure and ant targets as
> similar as possible across all Hadoop sub-projects. This helps leverage
> the same tools and test patch process more easily. So can we keep it
> similar to Core Hadoop?
We should not gratuitously differ, but Avro has a stronger multi-lingual
tilt than Hadoop. For example, Hadoop's src/test is Java-centric, while
Avro needs a separate test directory per implementation. The current
directory structure is a compromise: it's mostly like Hadoop Core, with
each implementation is in src/${impl}, but with test code in
src/test/${impl}, which is a bit asymmetric.
Currently Avro's lib/ directory is java-centric. Should we partition
this into lib/${impl}/, with separate directories per implementation?
> Regarding Junit version and/or TestNG. Can we
> try TestNG in this project?
I'd be fine with that if someone wants to develop a patch and others
agree. Note that, for Avro, we need a test framework per
implementation, and TestNG/JUnit only help us with Java.
Doug
Re: Code reorg
Posted by Nigel Daley <nd...@yahoo-inc.com>.
I think we should try to keep the directory structure and ant targets
as similar as possible across all Hadoop sub-projects. This helps
leverage the same tools and test patch process more easily. So can we
keep it similar to Core Hadoop? Regarding Junit version and/or
TestNG. Can we try TestNG in this project?
Cheers,
Nige
On Apr 11, 2009, at 4:58 AM, Thiruvalluvan M. G. wrote:
> Because these are early days of avro, it easier to get these changes
> effected now rather than later:
>
> 1. Directory structure
>
> There doesn't seem to be a de facto standard directory structure for
> these
> kinds of projects. But many projects keep the main code and test
> code at the
> same level. For example, protocol buffers keeps main and test code
> for Java
> under java/src. On the other hand, thrift keeps them as top level as
> compiler, lib and test and has directories under them for each
> language.
>
> Avro keeps test under src, which is somewhat asymmetric. Is there a
> good
> reason to keep that way? I'd prefer to follow the directory
> structure that
> is friendly to maven, which is similar to what protocol buffers uses.
>
> 2. Standardize Java tests
>
> The current code base uses JUnit3. It is preferable to move to
> either JUnit4
> TestNG. I don't have much experience in TestNG. I find JUnit4 much
> more
> convenient and expressive compared to JUnit3; this is particularly
> true for
> parameterized tests. Many think TestNG is better than JUnit4.
>
> If there's agreement that we should change these, I'll file JIRA for
> these
> decide on the way to go and then implement the decision.
>
> Thanks
>
> Thiru
>
>
>
>
Re: Code reorg
Posted by Doug Cutting <cu...@apache.org>.
Thiruvalluvan M. G. wrote:
> Avro keeps test under src, which is somewhat asymmetric. Is there a good
> reason to keep that way? I'd prefer to follow the directory structure that
> is friendly to maven, which is similar to what protocol buffers uses.
Can you make a concrete proposal? Are you suggesting we make it
something like src/java/{lib,test}/org/apache/avro/?
> The current code base uses JUnit3. It is preferable to move to either JUnit4
> TestNG. I don't have much experience in TestNG. I find JUnit4 much more
> convenient and expressive compared to JUnit3; this is particularly true for
> parameterized tests. Many think TestNG is better than JUnit4.
Avro includes JUnit 4.5, but existing tests are written in JUnit3's
style. However I see no problem with folks writing tests that require
JUnit4, or upgrading existing tests to that style.
> If there's agreement that we should change these, I'll file JIRA for these
> decide on the way to go and then implement the decision.
Yes, Jira issues would be a good place to discuss these.
Cheers,
Doug