You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-dev@hadoop.apache.org by Ravi Prakash <ra...@gmail.com> on 2012/05/07 23:45:10 UTC
What's the status of AspectJ files in branch-1?
Hi folks,
I'm patching changes to StatisticsCollector in
https://issues.apache.org/jira/browse/MAPREDUCE-4227 .
A simple grep shows StatisticsCollector is also referenced in
src/test/system/aop/org/apache/hadoop/mapred/StatisticsCollectorAspect.aj
src/test/system/aop/org/apache/hadoop/mapred/JobTrackerAspect.aj
How can I check whether my changes cause any issues or not in the AspectJ?
(tests?)
Thanks
Ravi.
Re: What's the status of AspectJ files in branch-1?
Posted by Konstantin Boudnik <co...@apache.org>.
Looks like you are missing hadoop configs somehow, because the
error is an indication that Herriot can't populate remote nodes with
configuration files.
Cos
On Tue, May 08, 2012 at 09:22AM, Ravi Prakash wrote:
> Thanks Cos!
>
> I was able to run ant jar-system jar-test-system successfully with my
> change. However, when I tried to run
>
> ant test-system -Dhadoop.conf.dir.deployed=${HADOOP_CONF_DIR}
>
> as mentioned in
> https://wiki.apache.org/hadoop/HowToUseSystemTestFrameworkall the
> streaming and gridmix tests had this error:
> java.lang.IllegalArgumentException: No Configuration passed for hadoop
> home and hadoop conf directories
> at
> org.apache.hadoop.test.system.process.HadoopDaemonRemoteCluster.populateDirectories(HadoopDaemonRemoteCluster.java:206)
> at
> org.apache.hadoop.test.system.process.HadoopDaemonRemoteCluster.init(HadoopDaemonRemoteCluster.java:170)
> at
> org.apache.hadoop.mapreduce.test.system.MRCluster.createCluster(MRCluster.java:107)
> at
> org.apache.hadoop.mapred.gridmix.GridmixSystemTestCase.before(GridmixSystemTestCase.java:69)
>
> I tried setting -Dtest.system.hdrc.hadoopconfdir=<confDir>
> -Dtest.system.hdrc.hadoophome=<hadoophome>
> -Dhadoop.conf.dir.deployed=<confdir> but the tests still failed.
>
> I understand these are gridmix and streaming test failures, and may not be
> directly related to the FI framework. But the dependency graph of
> test-system is such that these tests are being run before FI tests and so
> its failing before reaching FI tests. Are you able to run these
> successfully, and if yes, how?
>
> Thanks
> Ravi
>
>
> On Mon, May 7, 2012 at 5:47 PM, Konstantin Boudnik <co...@apache.org> wrote:
>
> > Hi Ravi.
> >
> > You need to run Herriot build to make sure that everything is ok after your
> > changes. The way to to it is as follows:
> >
> > % ant jar-system jar-test-system
> >
> > this will perform the compilation of Hadoop binaries with weaved Herriot
> > APIs.
> >
> > More information about Herriot can be found here
> > https://wiki.apache.org/hadoop/HowToUseSystemTestFramework
> >
> > Cos
> >
> > On Mon, May 07, 2012 at 04:45PM, Ravi Prakash wrote:
> > > Hi folks,
> > >
> > > I'm patching changes to StatisticsCollector in
> > > https://issues.apache.org/jira/browse/MAPREDUCE-4227 .
> > >
> > > A simple grep shows StatisticsCollector is also referenced in
> > > src/test/system/aop/org/apache/hadoop/mapred/StatisticsCollectorAspect.aj
> > > src/test/system/aop/org/apache/hadoop/mapred/JobTrackerAspect.aj
> > >
> > > How can I check whether my changes cause any issues or not in the
> > AspectJ?
> > > (tests?)
> > >
> > > Thanks
> > > Ravi.
> >
Re: What's the status of AspectJ files in branch-1?
Posted by Ravi Prakash <ra...@gmail.com>.
Thanks Cos!
I was able to run ant jar-system jar-test-system successfully with my
change. However, when I tried to run
ant test-system -Dhadoop.conf.dir.deployed=${HADOOP_CONF_DIR}
as mentioned in
https://wiki.apache.org/hadoop/HowToUseSystemTestFrameworkall the
streaming and gridmix tests had this error:
java.lang.IllegalArgumentException: No Configuration passed for hadoop
home and hadoop conf directories
at
org.apache.hadoop.test.system.process.HadoopDaemonRemoteCluster.populateDirectories(HadoopDaemonRemoteCluster.java:206)
at
org.apache.hadoop.test.system.process.HadoopDaemonRemoteCluster.init(HadoopDaemonRemoteCluster.java:170)
at
org.apache.hadoop.mapreduce.test.system.MRCluster.createCluster(MRCluster.java:107)
at
org.apache.hadoop.mapred.gridmix.GridmixSystemTestCase.before(GridmixSystemTestCase.java:69)
I tried setting -Dtest.system.hdrc.hadoopconfdir=<confDir>
-Dtest.system.hdrc.hadoophome=<hadoophome>
-Dhadoop.conf.dir.deployed=<confdir> but the tests still failed.
I understand these are gridmix and streaming test failures, and may not be
directly related to the FI framework. But the dependency graph of
test-system is such that these tests are being run before FI tests and so
its failing before reaching FI tests. Are you able to run these
successfully, and if yes, how?
Thanks
Ravi
On Mon, May 7, 2012 at 5:47 PM, Konstantin Boudnik <co...@apache.org> wrote:
> Hi Ravi.
>
> You need to run Herriot build to make sure that everything is ok after your
> changes. The way to to it is as follows:
>
> % ant jar-system jar-test-system
>
> this will perform the compilation of Hadoop binaries with weaved Herriot
> APIs.
>
> More information about Herriot can be found here
> https://wiki.apache.org/hadoop/HowToUseSystemTestFramework
>
> Cos
>
> On Mon, May 07, 2012 at 04:45PM, Ravi Prakash wrote:
> > Hi folks,
> >
> > I'm patching changes to StatisticsCollector in
> > https://issues.apache.org/jira/browse/MAPREDUCE-4227 .
> >
> > A simple grep shows StatisticsCollector is also referenced in
> > src/test/system/aop/org/apache/hadoop/mapred/StatisticsCollectorAspect.aj
> > src/test/system/aop/org/apache/hadoop/mapred/JobTrackerAspect.aj
> >
> > How can I check whether my changes cause any issues or not in the
> AspectJ?
> > (tests?)
> >
> > Thanks
> > Ravi.
>
Re: What's the status of AspectJ files in branch-1?
Posted by Konstantin Boudnik <co...@apache.org>.
Hi Ravi.
You need to run Herriot build to make sure that everything is ok after your
changes. The way to to it is as follows:
% ant jar-system jar-test-system
this will perform the compilation of Hadoop binaries with weaved Herriot APIs.
More information about Herriot can be found here
https://wiki.apache.org/hadoop/HowToUseSystemTestFramework
Cos
On Mon, May 07, 2012 at 04:45PM, Ravi Prakash wrote:
> Hi folks,
>
> I'm patching changes to StatisticsCollector in
> https://issues.apache.org/jira/browse/MAPREDUCE-4227 .
>
> A simple grep shows StatisticsCollector is also referenced in
> src/test/system/aop/org/apache/hadoop/mapred/StatisticsCollectorAspect.aj
> src/test/system/aop/org/apache/hadoop/mapred/JobTrackerAspect.aj
>
> How can I check whether my changes cause any issues or not in the AspectJ?
> (tests?)
>
> Thanks
> Ravi.