You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@harmony.apache.org by Charles Lee <li...@gmail.com> on 2010/07/28 07:49:45 UTC
[hadoop]Status about hadoop unit testing.
Hi guys,
I am running hadoop unit testing and trying to fix some failures for a
while. Combining two great projects excites me a lot. Here is my current
status:
Env:
Harmony 6 (requires by hadoop) + linux 32 platform
Using: run-test-core
Whole test cases: 757
Failures: 8
Errors: 25
To get this result, some code need to be changed in hadoop-common [0], some
jiras should be commited [1] and some modules I borrowed from RI [2].
[0]
a. UserGroupInformation.java should be changed, because hadoop hard coding
the its login module, using com.sun.security things.
b. reduce the failure trying times from 45 to 2. Because junit testcase will
be timeout if we are trying 45 times. (ri also failed on this)
[1]
a. HARMONY-6529 </jira/browse/HARMONY-6529>. hadoop is required jdk1.6,
patch available.
b. HARMONY-6561 </jira/browse/HARMONY-6561> Fixed
c. HARMONY-6569 </jira/browse/HARMONY-6569> Fixed
d. HARMONY-6571 </jira/browse/HARMONY-6571> JNDI things
e. HARMONY-6575 </jira/browse/HARMONY-6575> It may be a drlvm, in my todo
list.
f. HARMONY-6580 </jira/browse/HARMONY-6580>,
HARMONY-6605</jira/browse/HARMONY-6605>
Fixed
g. HARMONY-6587 </jira/browse/HARMONY-6587> Fixed
h. HARMONY-6604 </jira/browse/HARMONY-6604> Behavior different between ri
and harmony
[2]
a. add tools.jar, which in the harmony jdk/lib, in the
bootclasspath.properties. Ant try to find the com.sun.tools.javac.Main to
determine whether JAVA_HOME is point to a jre or a jdk. This jar is not from
ri, but we may need to change our layout a little bit.
b. add ecj.jar to the $ANT_HOME/lib or add it in the
bootclasspath.properties.
c. javax.annotation.processing, javax.lang.model, javax.tools, these package
borrow from RI, which is missing in harmony java6.
d. sun.security.krb5, this package borrow from RI. Do we have couterpart in
harmony? I have not found one.
e. sun.misc.HexDumpEncoder, this borrow from RI. Missing in harmony java6.
f. sun.reflect.ReflectionFactory, this borrow from RI. Missing in harmony
java6. [3]
g. sun/misc/Unsafe.defineClass(Ljava/lang/String;[BIILjava/lang/ClassLoader;Ljava/security/ProtectionDomain;)Ljava/lang/Class;
This method is not defined in our Unsafe. [3]
h. javadoc and doclet package is borrowed from RI. Missing in harmony java6.
Is is one of our GSoC now?
[3]
f and g in [2] is raised from a popular project: mockito. Mockito is using
another project: objenesis. Now g is at the top of my todo list.
--
Yours sincerely,
Charles Lee
Re: [hadoop]Status about hadoop unit testing.
Posted by Alexei Fedotov <al...@gmail.com>.
1. IMHO, there is some logic in the substitution you've described.
Apache Harmony class library is able to work with J9 JVM. When it
comes to DRLVM integration, system class library files are substituted
wth DRLVM specific implementations.
2. For your reference, DRLVM specific Object and Thread classes can be
found here:
[1] http://www.google.com/codesearch/p?hl=en#-nnolL0D1xY/trunk/drlvm/vm/vmcore/src/kernel_classes/javasrc/java/lang/Thread.java
--
With best regards / с наилучшими пожеланиями,
Alexei Fedotov / Алексей Федотов,
http://www.telecom-express.ru/
http://harmony.apache.org/
http://dataved.ru/ http://klsh.ru/
On Thu, Jul 29, 2010 at 10:18 AM, Charles Lee <li...@gmail.com> wrote:
> Hi guys,
>
> I have some few questions about the Unsafe.java.
>
> There are Unsafes in repos: one is under modules/suncompact [1], another is
> under the drlvm/vm [2]. After build, [2] is in the kernel.jar, [1] is in the
> suncompact.jar. I suppose kernel.jar has the high priority than
> suncompact.jar. (suncompact.jar is commented out in the
> bootclasspath.properties, which means we just use [2]) What's the difference
> between these two files?
>
> Give a simple diff, we will find that [1] is more updated than [2]. More
> investigation on [1], i found [1] depends on some classes which is provided
> by the modules/luni-kernal. I supposed classes in the modules/luni-kernal
> are stub class, but I am wrong. Updates in [1] depends on Objects, Threads
> and VM class in modules/luni-kernal. And these three classes can not be
> found in drlvm. Due to these 3 classes, simply replace [2] with [1] will
> cause NoClassDefFoundError.
>
> Any idea about this?
>
> On Wed, Jul 28, 2010 at 4:08 PM, Charles Lee <li...@gmail.com> wrote:
>
>>
>>
>> On Wed, Jul 28, 2010 at 3:49 PM, Tim Ellison <t....@gmail.com>wrote:
>>
>>> A good summary, thanks Charles.
>>>
>>> More below...
>>>
>>> On 28/Jul/2010 06:49, Charles Lee wrote:
>>> > Hi guys,
>>> >
>>> > I am running hadoop unit testing and trying to fix some failures for a
>>> > while. Combining two great projects excites me a lot. Here is my current
>>> > status:
>>> >
>>> > Env:
>>> > Harmony 6 (requires by hadoop) + linux 32 platform
>>> >
>>> > Using: run-test-core
>>> > Whole test cases: 757
>>> > Failures: 8
>>> > Errors: 25
>>> >
>>> > To get this result, some code need to be changed in hadoop-common [0],
>>> some
>>> > jiras should be commited [1] and some modules I borrowed from RI [2].
>>> >
>>> > [0]
>>> > a. UserGroupInformation.java should be changed, because hadoop hard
>>> coding
>>> > the its login module, using com.sun.security things.
>>> > b. reduce the failure trying times from 45 to 2. Because junit testcase
>>> will
>>> > be timeout if we are trying 45 times. (ri also failed on this)
>>>
>>> Maybe raise Hadoop JIRAs?
>>>
>>> > [1]
>>> > a. HARMONY-6529 </jira/browse/HARMONY-6529>. hadoop is required jdk1.6,
>>> > patch available.
>>> > b. HARMONY-6561 </jira/browse/HARMONY-6561> Fixed
>>> > c. HARMONY-6569 </jira/browse/HARMONY-6569> Fixed
>>> > d. HARMONY-6571 </jira/browse/HARMONY-6571> JNDI things
>>> > e. HARMONY-6575 </jira/browse/HARMONY-6575> It may be a drlvm, in my
>>> todo
>>> > list.
>>> > f. HARMONY-6580 </jira/browse/HARMONY-6580>,
>>> > HARMONY-6605</jira/browse/HARMONY-6605>
>>> > Fixed
>>> > g. HARMONY-6587 </jira/browse/HARMONY-6587> Fixed
>>> > h. HARMONY-6604 </jira/browse/HARMONY-6604> Behavior different between
>>> ri
>>> > and harmony
>>>
>>> I can help with these.
>>>
>>
>> Great :-)
>>
>>
>>>
>>> > [2]
>>> > a. add tools.jar, which in the harmony jdk/lib, in the
>>> > bootclasspath.properties. Ant try to find the com.sun.tools.javac.Main
>>> to
>>> > determine whether JAVA_HOME is point to a jre or a jdk. This jar is not
>>> from
>>> > ri, but we may need to change our layout a little bit.
>>>
>>> I'm struggling a bit to follow you.
>>>
>>> Both the RI and harmony have got a <JAVA_HOME>/lib/tools.jar, and both
>>> contain a type called "com.sun.tools.javac.Main" (we added it in Harmony
>>> for compatibility).
>>>
>>> So what is the issue here?
>>>
>>>
>> Ant try to find the com.sun.tools.javac.Main, and suppose this class should
>> be on the classpath.
>> We do have a com.sun.tools.javac.Main in jdk/lib/tools.jar, but this
>> archive is not included in our classpath.
>>
>>
>>> > b. add ecj.jar to the $ANT_HOME/lib or add it in the
>>> > bootclasspath.properties.
>>>
>>> This is to get the Ant <javac> task working?
>>>
>>>
>> Yes.
>>
>>
>>> > c. javax.annotation.processing, javax.lang.model, javax.tools, these
>>> package
>>> > borrow from RI, which is missing in harmony java6.
>>>
>>> Ok - not sure if there is an ALv2 version around for these.
>>>
>>> > d. sun.security.krb5, this package borrow from RI. Do we have couterpart
>>> in
>>> > harmony? I have not found one.
>>> > e. sun.misc.HexDumpEncoder, this borrow from RI. Missing in harmony
>>> java6.
>>> > f. sun.reflect.ReflectionFactory, this borrow from RI. Missing in
>>> harmony
>>> > java6. [3]
>>> > g.
>>> sun/misc/Unsafe.defineClass(Ljava/lang/String;[BIILjava/lang/ClassLoader;Ljava/security/ProtectionDomain;)Ljava/lang/Class;
>>> > This method is not defined in our Unsafe. [3]
>>>
>>> No idea what d,e,f are doing, but seems unfortunate that Hadoop is using
>>> them, sounds like a conversation to have with the Hadoop project.
>>>
>>> We can guess what g would do, and include that in our Unsafe (but again
>>> would be interested to know why Hadoop is using this rather than the
>>> ClassLoader#defineClass API).
>>>
>>
>> Hadoop is not directly using these packages, but hadoop unit testing is
>> using mockito --> objenesis, and objenesis is using these packages.
>>
>>
>>>
>>> > h. javadoc and doclet package is borrowed from RI. Missing in harmony
>>> java6.
>>> > Is is one of our GSoC now?
>>>
>>> No, but I assume they are not needed for normal operations of Hadoop?
>>>
>>
>> Hadoop using these packages to preprocessor its annotations in files. Need
>> these packages to compile hadoop using harmony.
>>
>>
>>>
>>> > [3]
>>> > f and g in [2] is raised from a popular project: mockito. Mockito is
>>> using
>>> > another project: objenesis. Now g is at the top of my todo list.
>>>
>>> Ah, so these non-API calls are made by a Hadoop dependency?
>>>
>>
>> Yes.
>>
>>
>>>
>>> Regards,
>>> Tim
>>>
>>>
>>
>>
>> --
>> Yours sincerely,
>> Charles Lee
>>
>>
>
>
> --
> Yours sincerely,
> Charles Lee
>
Re: [hadoop]Status about hadoop unit testing.
Posted by Charles Lee <li...@gmail.com>.
Hi guys,
I have some few questions about the Unsafe.java.
There are Unsafes in repos: one is under modules/suncompact [1], another is
under the drlvm/vm [2]. After build, [2] is in the kernel.jar, [1] is in the
suncompact.jar. I suppose kernel.jar has the high priority than
suncompact.jar. (suncompact.jar is commented out in the
bootclasspath.properties, which means we just use [2]) What's the difference
between these two files?
Give a simple diff, we will find that [1] is more updated than [2]. More
investigation on [1], i found [1] depends on some classes which is provided
by the modules/luni-kernal. I supposed classes in the modules/luni-kernal
are stub class, but I am wrong. Updates in [1] depends on Objects, Threads
and VM class in modules/luni-kernal. And these three classes can not be
found in drlvm. Due to these 3 classes, simply replace [2] with [1] will
cause NoClassDefFoundError.
Any idea about this?
On Wed, Jul 28, 2010 at 4:08 PM, Charles Lee <li...@gmail.com> wrote:
>
>
> On Wed, Jul 28, 2010 at 3:49 PM, Tim Ellison <t....@gmail.com>wrote:
>
>> A good summary, thanks Charles.
>>
>> More below...
>>
>> On 28/Jul/2010 06:49, Charles Lee wrote:
>> > Hi guys,
>> >
>> > I am running hadoop unit testing and trying to fix some failures for a
>> > while. Combining two great projects excites me a lot. Here is my current
>> > status:
>> >
>> > Env:
>> > Harmony 6 (requires by hadoop) + linux 32 platform
>> >
>> > Using: run-test-core
>> > Whole test cases: 757
>> > Failures: 8
>> > Errors: 25
>> >
>> > To get this result, some code need to be changed in hadoop-common [0],
>> some
>> > jiras should be commited [1] and some modules I borrowed from RI [2].
>> >
>> > [0]
>> > a. UserGroupInformation.java should be changed, because hadoop hard
>> coding
>> > the its login module, using com.sun.security things.
>> > b. reduce the failure trying times from 45 to 2. Because junit testcase
>> will
>> > be timeout if we are trying 45 times. (ri also failed on this)
>>
>> Maybe raise Hadoop JIRAs?
>>
>> > [1]
>> > a. HARMONY-6529 </jira/browse/HARMONY-6529>. hadoop is required jdk1.6,
>> > patch available.
>> > b. HARMONY-6561 </jira/browse/HARMONY-6561> Fixed
>> > c. HARMONY-6569 </jira/browse/HARMONY-6569> Fixed
>> > d. HARMONY-6571 </jira/browse/HARMONY-6571> JNDI things
>> > e. HARMONY-6575 </jira/browse/HARMONY-6575> It may be a drlvm, in my
>> todo
>> > list.
>> > f. HARMONY-6580 </jira/browse/HARMONY-6580>,
>> > HARMONY-6605</jira/browse/HARMONY-6605>
>> > Fixed
>> > g. HARMONY-6587 </jira/browse/HARMONY-6587> Fixed
>> > h. HARMONY-6604 </jira/browse/HARMONY-6604> Behavior different between
>> ri
>> > and harmony
>>
>> I can help with these.
>>
>
> Great :-)
>
>
>>
>> > [2]
>> > a. add tools.jar, which in the harmony jdk/lib, in the
>> > bootclasspath.properties. Ant try to find the com.sun.tools.javac.Main
>> to
>> > determine whether JAVA_HOME is point to a jre or a jdk. This jar is not
>> from
>> > ri, but we may need to change our layout a little bit.
>>
>> I'm struggling a bit to follow you.
>>
>> Both the RI and harmony have got a <JAVA_HOME>/lib/tools.jar, and both
>> contain a type called "com.sun.tools.javac.Main" (we added it in Harmony
>> for compatibility).
>>
>> So what is the issue here?
>>
>>
> Ant try to find the com.sun.tools.javac.Main, and suppose this class should
> be on the classpath.
> We do have a com.sun.tools.javac.Main in jdk/lib/tools.jar, but this
> archive is not included in our classpath.
>
>
>> > b. add ecj.jar to the $ANT_HOME/lib or add it in the
>> > bootclasspath.properties.
>>
>> This is to get the Ant <javac> task working?
>>
>>
> Yes.
>
>
>> > c. javax.annotation.processing, javax.lang.model, javax.tools, these
>> package
>> > borrow from RI, which is missing in harmony java6.
>>
>> Ok - not sure if there is an ALv2 version around for these.
>>
>> > d. sun.security.krb5, this package borrow from RI. Do we have couterpart
>> in
>> > harmony? I have not found one.
>> > e. sun.misc.HexDumpEncoder, this borrow from RI. Missing in harmony
>> java6.
>> > f. sun.reflect.ReflectionFactory, this borrow from RI. Missing in
>> harmony
>> > java6. [3]
>> > g.
>> sun/misc/Unsafe.defineClass(Ljava/lang/String;[BIILjava/lang/ClassLoader;Ljava/security/ProtectionDomain;)Ljava/lang/Class;
>> > This method is not defined in our Unsafe. [3]
>>
>> No idea what d,e,f are doing, but seems unfortunate that Hadoop is using
>> them, sounds like a conversation to have with the Hadoop project.
>>
>> We can guess what g would do, and include that in our Unsafe (but again
>> would be interested to know why Hadoop is using this rather than the
>> ClassLoader#defineClass API).
>>
>
> Hadoop is not directly using these packages, but hadoop unit testing is
> using mockito --> objenesis, and objenesis is using these packages.
>
>
>>
>> > h. javadoc and doclet package is borrowed from RI. Missing in harmony
>> java6.
>> > Is is one of our GSoC now?
>>
>> No, but I assume they are not needed for normal operations of Hadoop?
>>
>
> Hadoop using these packages to preprocessor its annotations in files. Need
> these packages to compile hadoop using harmony.
>
>
>>
>> > [3]
>> > f and g in [2] is raised from a popular project: mockito. Mockito is
>> using
>> > another project: objenesis. Now g is at the top of my todo list.
>>
>> Ah, so these non-API calls are made by a Hadoop dependency?
>>
>
> Yes.
>
>
>>
>> Regards,
>> Tim
>>
>>
>
>
> --
> Yours sincerely,
> Charles Lee
>
>
--
Yours sincerely,
Charles Lee
Re: [hadoop]Status about hadoop unit testing.
Posted by Charles Lee <li...@gmail.com>.
On Wed, Jul 28, 2010 at 3:49 PM, Tim Ellison <t....@gmail.com> wrote:
> A good summary, thanks Charles.
>
> More below...
>
> On 28/Jul/2010 06:49, Charles Lee wrote:
> > Hi guys,
> >
> > I am running hadoop unit testing and trying to fix some failures for a
> > while. Combining two great projects excites me a lot. Here is my current
> > status:
> >
> > Env:
> > Harmony 6 (requires by hadoop) + linux 32 platform
> >
> > Using: run-test-core
> > Whole test cases: 757
> > Failures: 8
> > Errors: 25
> >
> > To get this result, some code need to be changed in hadoop-common [0],
> some
> > jiras should be commited [1] and some modules I borrowed from RI [2].
> >
> > [0]
> > a. UserGroupInformation.java should be changed, because hadoop hard
> coding
> > the its login module, using com.sun.security things.
> > b. reduce the failure trying times from 45 to 2. Because junit testcase
> will
> > be timeout if we are trying 45 times. (ri also failed on this)
>
> Maybe raise Hadoop JIRAs?
>
> > [1]
> > a. HARMONY-6529 </jira/browse/HARMONY-6529>. hadoop is required jdk1.6,
> > patch available.
> > b. HARMONY-6561 </jira/browse/HARMONY-6561> Fixed
> > c. HARMONY-6569 </jira/browse/HARMONY-6569> Fixed
> > d. HARMONY-6571 </jira/browse/HARMONY-6571> JNDI things
> > e. HARMONY-6575 </jira/browse/HARMONY-6575> It may be a drlvm, in my todo
> > list.
> > f. HARMONY-6580 </jira/browse/HARMONY-6580>,
> > HARMONY-6605</jira/browse/HARMONY-6605>
> > Fixed
> > g. HARMONY-6587 </jira/browse/HARMONY-6587> Fixed
> > h. HARMONY-6604 </jira/browse/HARMONY-6604> Behavior different between ri
> > and harmony
>
> I can help with these.
>
Great :-)
>
> > [2]
> > a. add tools.jar, which in the harmony jdk/lib, in the
> > bootclasspath.properties. Ant try to find the com.sun.tools.javac.Main to
> > determine whether JAVA_HOME is point to a jre or a jdk. This jar is not
> from
> > ri, but we may need to change our layout a little bit.
>
> I'm struggling a bit to follow you.
>
> Both the RI and harmony have got a <JAVA_HOME>/lib/tools.jar, and both
> contain a type called "com.sun.tools.javac.Main" (we added it in Harmony
> for compatibility).
>
> So what is the issue here?
>
>
Ant try to find the com.sun.tools.javac.Main, and suppose this class should
be on the classpath.
We do have a com.sun.tools.javac.Main in jdk/lib/tools.jar, but this archive
is not included in our classpath.
> > b. add ecj.jar to the $ANT_HOME/lib or add it in the
> > bootclasspath.properties.
>
> This is to get the Ant <javac> task working?
>
>
Yes.
> > c. javax.annotation.processing, javax.lang.model, javax.tools, these
> package
> > borrow from RI, which is missing in harmony java6.
>
> Ok - not sure if there is an ALv2 version around for these.
>
> > d. sun.security.krb5, this package borrow from RI. Do we have couterpart
> in
> > harmony? I have not found one.
> > e. sun.misc.HexDumpEncoder, this borrow from RI. Missing in harmony
> java6.
> > f. sun.reflect.ReflectionFactory, this borrow from RI. Missing in
> harmony
> > java6. [3]
> > g.
> sun/misc/Unsafe.defineClass(Ljava/lang/String;[BIILjava/lang/ClassLoader;Ljava/security/ProtectionDomain;)Ljava/lang/Class;
> > This method is not defined in our Unsafe. [3]
>
> No idea what d,e,f are doing, but seems unfortunate that Hadoop is using
> them, sounds like a conversation to have with the Hadoop project.
>
> We can guess what g would do, and include that in our Unsafe (but again
> would be interested to know why Hadoop is using this rather than the
> ClassLoader#defineClass API).
>
Hadoop is not directly using these packages, but hadoop unit testing is
using mockito --> objenesis, and objenesis is using these packages.
>
> > h. javadoc and doclet package is borrowed from RI. Missing in harmony
> java6.
> > Is is one of our GSoC now?
>
> No, but I assume they are not needed for normal operations of Hadoop?
>
Hadoop using these packages to preprocessor its annotations in files. Need
these packages to compile hadoop using harmony.
>
> > [3]
> > f and g in [2] is raised from a popular project: mockito. Mockito is
> using
> > another project: objenesis. Now g is at the top of my todo list.
>
> Ah, so these non-API calls are made by a Hadoop dependency?
>
Yes.
>
> Regards,
> Tim
>
>
--
Yours sincerely,
Charles Lee
Re: [hadoop]Status about hadoop unit testing.
Posted by Tim Ellison <t....@gmail.com>.
A good summary, thanks Charles.
More below...
On 28/Jul/2010 06:49, Charles Lee wrote:
> Hi guys,
>
> I am running hadoop unit testing and trying to fix some failures for a
> while. Combining two great projects excites me a lot. Here is my current
> status:
>
> Env:
> Harmony 6 (requires by hadoop) + linux 32 platform
>
> Using: run-test-core
> Whole test cases: 757
> Failures: 8
> Errors: 25
>
> To get this result, some code need to be changed in hadoop-common [0], some
> jiras should be commited [1] and some modules I borrowed from RI [2].
>
> [0]
> a. UserGroupInformation.java should be changed, because hadoop hard coding
> the its login module, using com.sun.security things.
> b. reduce the failure trying times from 45 to 2. Because junit testcase will
> be timeout if we are trying 45 times. (ri also failed on this)
Maybe raise Hadoop JIRAs?
> [1]
> a. HARMONY-6529 </jira/browse/HARMONY-6529>. hadoop is required jdk1.6,
> patch available.
> b. HARMONY-6561 </jira/browse/HARMONY-6561> Fixed
> c. HARMONY-6569 </jira/browse/HARMONY-6569> Fixed
> d. HARMONY-6571 </jira/browse/HARMONY-6571> JNDI things
> e. HARMONY-6575 </jira/browse/HARMONY-6575> It may be a drlvm, in my todo
> list.
> f. HARMONY-6580 </jira/browse/HARMONY-6580>,
> HARMONY-6605</jira/browse/HARMONY-6605>
> Fixed
> g. HARMONY-6587 </jira/browse/HARMONY-6587> Fixed
> h. HARMONY-6604 </jira/browse/HARMONY-6604> Behavior different between ri
> and harmony
I can help with these.
> [2]
> a. add tools.jar, which in the harmony jdk/lib, in the
> bootclasspath.properties. Ant try to find the com.sun.tools.javac.Main to
> determine whether JAVA_HOME is point to a jre or a jdk. This jar is not from
> ri, but we may need to change our layout a little bit.
I'm struggling a bit to follow you.
Both the RI and harmony have got a <JAVA_HOME>/lib/tools.jar, and both
contain a type called "com.sun.tools.javac.Main" (we added it in Harmony
for compatibility).
So what is the issue here?
> b. add ecj.jar to the $ANT_HOME/lib or add it in the
> bootclasspath.properties.
This is to get the Ant <javac> task working?
> c. javax.annotation.processing, javax.lang.model, javax.tools, these package
> borrow from RI, which is missing in harmony java6.
Ok - not sure if there is an ALv2 version around for these.
> d. sun.security.krb5, this package borrow from RI. Do we have couterpart in
> harmony? I have not found one.
> e. sun.misc.HexDumpEncoder, this borrow from RI. Missing in harmony java6.
> f. sun.reflect.ReflectionFactory, this borrow from RI. Missing in harmony
> java6. [3]
> g. sun/misc/Unsafe.defineClass(Ljava/lang/String;[BIILjava/lang/ClassLoader;Ljava/security/ProtectionDomain;)Ljava/lang/Class;
> This method is not defined in our Unsafe. [3]
No idea what d,e,f are doing, but seems unfortunate that Hadoop is using
them, sounds like a conversation to have with the Hadoop project.
We can guess what g would do, and include that in our Unsafe (but again
would be interested to know why Hadoop is using this rather than the
ClassLoader#defineClass API).
> h. javadoc and doclet package is borrowed from RI. Missing in harmony java6.
> Is is one of our GSoC now?
No, but I assume they are not needed for normal operations of Hadoop?
> [3]
> f and g in [2] is raised from a popular project: mockito. Mockito is using
> another project: objenesis. Now g is at the top of my todo list.
Ah, so these non-API calls are made by a Hadoop dependency?
Regards,
Tim
Re: [hadoop]Status about hadoop unit testing.
Posted by Charles Lee <li...@gmail.com>.
On Wed, Jul 28, 2010 at 2:55 PM, Alexei Fedotov <al...@gmail.com>wrote:
> Charles,
> Is it possible to rewrite Hadoop tests to avoid using sun.* packages?
Yes. It is possible. Most of these packages is needed by
objenesis<http://code.google.com/p/objenesis/>.
Most mockito in the test case can be rewritten to a mock class. It is
possible. But I am thinking instead of not supporting objenesis, we may add
harmony in the support list of mockito.
>
> On Wed, Jul 28, 2010 at 9:49 AM, Charles Lee <li...@gmail.com>
> wrote:
> > Hi guys,
> >
> > I am running hadoop unit testing and trying to fix some failures for a
> > while. Combining two great projects excites me a lot. Here is my current
> > status:
> >
> > Env:
> > Harmony 6 (requires by hadoop) + linux 32 platform
> >
> > Using: run-test-core
> > Whole test cases: 757
> > Failures: 8
> > Errors: 25
> >
> > To get this result, some code need to be changed in hadoop-common [0],
> some
> > jiras should be commited [1] and some modules I borrowed from RI [2].
> >
> > [0]
> > a. UserGroupInformation.java should be changed, because hadoop hard
> coding
> > the its login module, using com.sun.security things.
> > b. reduce the failure trying times from 45 to 2. Because junit testcase
> will
> > be timeout if we are trying 45 times. (ri also failed on this)
> >
> > [1]
> > a. HARMONY-6529 </jira/browse/HARMONY-6529>. hadoop is required jdk1.6,
> > patch available.
> > b. HARMONY-6561 </jira/browse/HARMONY-6561> Fixed
> > c. HARMONY-6569 </jira/browse/HARMONY-6569> Fixed
> > d. HARMONY-6571 </jira/browse/HARMONY-6571> JNDI things
> > e. HARMONY-6575 </jira/browse/HARMONY-6575> It may be a drlvm, in my todo
> > list.
> > f. HARMONY-6580 </jira/browse/HARMONY-6580>,
> > HARMONY-6605</jira/browse/HARMONY-6605>
> > Fixed
> > g. HARMONY-6587 </jira/browse/HARMONY-6587> Fixed
> > h. HARMONY-6604 </jira/browse/HARMONY-6604> Behavior different between ri
> > and harmony
> >
> > [2]
> > a. add tools.jar, which in the harmony jdk/lib, in the
> > bootclasspath.properties. Ant try to find the com.sun.tools.javac.Main to
> > determine whether JAVA_HOME is point to a jre or a jdk. This jar is not
> from
> > ri, but we may need to change our layout a little bit.
> > b. add ecj.jar to the $ANT_HOME/lib or add it in the
> > bootclasspath.properties.
> > c. javax.annotation.processing, javax.lang.model, javax.tools, these
> package
> > borrow from RI, which is missing in harmony java6.
> > d. sun.security.krb5, this package borrow from RI. Do we have couterpart
> in
> > harmony? I have not found one.
> > e. sun.misc.HexDumpEncoder, this borrow from RI. Missing in harmony
> java6.
> > f. sun.reflect.ReflectionFactory, this borrow from RI. Missing in
> harmony
> > java6. [3]
> > g.
> sun/misc/Unsafe.defineClass(Ljava/lang/String;[BIILjava/lang/ClassLoader;Ljava/security/ProtectionDomain;)Ljava/lang/Class;
> > This method is not defined in our Unsafe. [3]
> > h. javadoc and doclet package is borrowed from RI. Missing in harmony
> java6.
> > Is is one of our GSoC now?
> >
> > [3]
> > f and g in [2] is raised from a popular project: mockito. Mockito is
> using
> > another project: objenesis. Now g is at the top of my todo list.
> >
> > --
> > Yours sincerely,
> > Charles Lee
> >
>
--
Yours sincerely,
Charles Lee
Re: [hadoop]Status about hadoop unit testing.
Posted by Alexei Fedotov <al...@gmail.com>.
Charles,
Is it possible to rewrite Hadoop tests to avoid using sun.* packages?
On Wed, Jul 28, 2010 at 9:49 AM, Charles Lee <li...@gmail.com> wrote:
> Hi guys,
>
> I am running hadoop unit testing and trying to fix some failures for a
> while. Combining two great projects excites me a lot. Here is my current
> status:
>
> Env:
> Harmony 6 (requires by hadoop) + linux 32 platform
>
> Using: run-test-core
> Whole test cases: 757
> Failures: 8
> Errors: 25
>
> To get this result, some code need to be changed in hadoop-common [0], some
> jiras should be commited [1] and some modules I borrowed from RI [2].
>
> [0]
> a. UserGroupInformation.java should be changed, because hadoop hard coding
> the its login module, using com.sun.security things.
> b. reduce the failure trying times from 45 to 2. Because junit testcase will
> be timeout if we are trying 45 times. (ri also failed on this)
>
> [1]
> a. HARMONY-6529 </jira/browse/HARMONY-6529>. hadoop is required jdk1.6,
> patch available.
> b. HARMONY-6561 </jira/browse/HARMONY-6561> Fixed
> c. HARMONY-6569 </jira/browse/HARMONY-6569> Fixed
> d. HARMONY-6571 </jira/browse/HARMONY-6571> JNDI things
> e. HARMONY-6575 </jira/browse/HARMONY-6575> It may be a drlvm, in my todo
> list.
> f. HARMONY-6580 </jira/browse/HARMONY-6580>,
> HARMONY-6605</jira/browse/HARMONY-6605>
> Fixed
> g. HARMONY-6587 </jira/browse/HARMONY-6587> Fixed
> h. HARMONY-6604 </jira/browse/HARMONY-6604> Behavior different between ri
> and harmony
>
> [2]
> a. add tools.jar, which in the harmony jdk/lib, in the
> bootclasspath.properties. Ant try to find the com.sun.tools.javac.Main to
> determine whether JAVA_HOME is point to a jre or a jdk. This jar is not from
> ri, but we may need to change our layout a little bit.
> b. add ecj.jar to the $ANT_HOME/lib or add it in the
> bootclasspath.properties.
> c. javax.annotation.processing, javax.lang.model, javax.tools, these package
> borrow from RI, which is missing in harmony java6.
> d. sun.security.krb5, this package borrow from RI. Do we have couterpart in
> harmony? I have not found one.
> e. sun.misc.HexDumpEncoder, this borrow from RI. Missing in harmony java6.
> f. sun.reflect.ReflectionFactory, this borrow from RI. Missing in harmony
> java6. [3]
> g. sun/misc/Unsafe.defineClass(Ljava/lang/String;[BIILjava/lang/ClassLoader;Ljava/security/ProtectionDomain;)Ljava/lang/Class;
> This method is not defined in our Unsafe. [3]
> h. javadoc and doclet package is borrowed from RI. Missing in harmony java6.
> Is is one of our GSoC now?
>
> [3]
> f and g in [2] is raised from a popular project: mockito. Mockito is using
> another project: objenesis. Now g is at the top of my todo list.
>
> --
> Yours sincerely,
> Charles Lee
>