You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mahout.apache.org by "Tom White (Updated) (JIRA)" <ji...@apache.org> on 2011/12/29 07:32:30 UTC
[jira] [Updated] (MAHOUT-822) Mahout needs to be made compatible
with Hadoop .23 releases
[ https://issues.apache.org/jira/browse/MAHOUT-822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tom White updated MAHOUT-822:
-----------------------------
Attachment: MAHOUT-822.patch
Here's a new version of the patch that applies against trunk. As it stands it passes tests against 0.20-based Hadoop, and there a few failures against 0.23.0, but Mahout is essentially usable against that version. So committing this would not cause any regressions and would help the Bigtop folks.
I also opened MAPREDUCE-3607 to get the new API version of MultipleOutputs into Hadoop 1.x (the evolution of the 0.20 line), which will hopefully help get BtJob running consistently on all Hadoop versions.
> Mahout needs to be made compatible with Hadoop .23 releases
> -----------------------------------------------------------
>
> Key: MAHOUT-822
> URL: https://issues.apache.org/jira/browse/MAHOUT-822
> Project: Mahout
> Issue Type: Improvement
> Components: build
> Affects Versions: 0.6
> Reporter: Roman Shaposhnik
> Labels: bigtop
> Attachments: MAHOUT-822-build.patch.txt, MAHOUT-822.patch, MAHOUT-822.patch, MAHOUT-822.patch, MAHOUT-822.patch, MAHOUT-822.patch
>
>
> As part of the Hadoop stack integration project (Apache Bigtop) we are now trying to compile Mahout's upcoming 0.6 release against Hadoop 0.22 and 0.23.
> I'm attaching the patch to Mahout's Maven build system that made it possible. I would also like to request help in solving the real issues that poped
> up when we tried to compile Mahout: http://bigtop01.cloudera.org:8080/job/Bigtop-hadoop22/COMPONENT=mahout,label=centos5/6/console
> {noformat}
> [ERROR] /mnt/jenkins/workspace/workspace/Bigtop-hadoop22/COMPONENT/mahout/label/centos5/build/mahout/rpm/BUILD/apache-mahout-c298f70/core/src/test/java/org/apache/mahout/df/mapreduce/partial/Step0JobTest.java:[182,33] org.apache.hadoop.mapreduce.TaskAttemptContext is abstract; cannot be instantiated
> [ERROR] /mnt/jenkins/workspace/workspace/Bigtop-hadoop22/COMPONENT/mahout/label/centos5/build/mahout/rpm/BUILD/apache-mahout-c298f70/core/src/test/java/org/apache/mahout/df/mapreduce/partial/Step0JobTest.java:[218,9] org.apache.mahout.df.mapreduce.partial.Step0JobTest.Step0Context is not abstract and does not override abstract method getInputSplit() in org.apache.hadoop.mapreduce.MapContext
> [ERROR] /mnt/jenkins/workspace/workspace/Bigtop-hadoop22/COMPONENT/mahout/label/centos5/build/mahout/rpm/BUILD/apache-mahout-c298f70/core/src/test/java/org/apache/mahout/df/mapreduce/partial/Step0JobTest.java:[229,12] cannot find symbol
> [ERROR] symbol : constructor Context(org.apache.hadoop.conf.Configuration,org.apache.hadoop.mapreduce.TaskAttemptID,<nulltype>,<nulltype>,<nulltype>,<nulltype>,<nulltype>)
> [ERROR] location: class org.apache.hadoop.mapreduce.Mapper.Context
> [ERROR] /mnt/jenkins/workspace/workspace/Bigtop-hadoop22/COMPONENT/mahout/label/centos5/build/mahout/rpm/BUILD/apache-mahout-c298f70/core/src/test/java/org/apache/mahout/common/DummyRecordWriter.java:[68,18] org.apache.hadoop.mapreduce.Mapper.Context is abstract; cannot be instantiated
> [ERROR] /mnt/jenkins/workspace/workspace/Bigtop-hadoop22/COMPONENT/mahout/label/centos5/build/mahout/rpm/BUILD/apache-mahout-c298f70/core/src/test/java/org/apache/mahout/common/DummyRecordWriter.java:[77,19] org.apache.hadoop.mapreduce.Reducer.Context is abstract; cannot be instantiated
> [ERROR] /mnt/jenkins/workspace/workspace/Bigtop-hadoop22/COMPONENT/mahout/label/centos5/build/mahout/rpm/BUILD/apache-mahout-c298f70/core/src/test/java/org/apache/mahout/df/mapreduce/partial/PartialSequentialBuilder.java:[110,30] org.apache.hadoop.mapreduce.TaskAttemptContext is abstract; cannot be instantiated
> [ERROR] /mnt/jenkins/workspace/workspace/Bigtop-hadoop22/COMPONENT/mahout/label/centos5/build/mahout/rpm/BUILD/apache-mahout-c298f70/core/src/test/java/org/apache/mahout/df/mapreduce/partial/PartialSequentialBuilder.java:[206,28] org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
> [ERROR] /mnt/jenkins/workspace/workspace/Bigtop-hadoop22/COMPONENT/mahout/label/centos5/build/mahout/rpm/BUILD/apache-mahout-c298f70/core/src/test/java/org/apache/mahout/df/mapreduce/partial/PartialSequentialBuilder.java:[227,30] org.apache.hadoop.mapreduce.TaskAttemptContext is abstract; cannot be instantiated
> [ERROR] /mnt/jenkins/workspace/workspace/Bigtop-hadoop22/COMPONENT/mahout/label/centos5/build/mahout/rpm/BUILD/apache-mahout-c298f70/core/src/test/java/org/apache/mahout/df/mapreduce/partial/MockContext.java:[30,6] org.apache.mahout.df.mapreduce.partial.MockContext is not abstract and does not override abstract method getInputSplit() in org.apache.hadoop.mapreduce.MapContext
> [ERROR] /mnt/jenkins/workspace/workspace/Bigtop-hadoop22/COMPONENT/mahout/label/centos5/build/mahout/rpm/BUILD/apache-mahout-c298f70/core/src/test/java/org/apache/mahout/df/mapreduce/partial/MockContext.java:[38,10] cannot find symbol
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [jira] [Updated] (MAHOUT-822) Mahout needs to be made compatible
with Hadoop .23 releases
Posted by Ted Dunning <te...@gmail.com>.
Thanks Tom
On Wed, Dec 28, 2011 at 10:32 PM, Tom White (Updated) (JIRA) <
jira@apache.org> wrote:
>
> [
> https://issues.apache.org/jira/browse/MAHOUT-822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel]
>
> Tom White updated MAHOUT-822:
> -----------------------------
>
> Attachment: MAHOUT-822.patch
>
> Here's a new version of the patch that applies against trunk. As it stands
> it passes tests against 0.20-based Hadoop, and there a few failures against
> 0.23.0, but Mahout is essentially usable against that version. So
> committing this would not cause any regressions and would help the Bigtop
> folks.
>
> I also opened MAPREDUCE-3607 to get the new API version of MultipleOutputs
> into Hadoop 1.x (the evolution of the 0.20 line), which will hopefully help
> get BtJob running consistently on all Hadoop versions.
>
>
> > Mahout needs to be made compatible with Hadoop .23 releases
> > -----------------------------------------------------------
> >
> > Key: MAHOUT-822
> > URL: https://issues.apache.org/jira/browse/MAHOUT-822
> > Project: Mahout
> > Issue Type: Improvement
> > Components: build
> > Affects Versions: 0.6
> > Reporter: Roman Shaposhnik
> > Labels: bigtop
> > Attachments: MAHOUT-822-build.patch.txt, MAHOUT-822.patch,
> MAHOUT-822.patch, MAHOUT-822.patch, MAHOUT-822.patch, MAHOUT-822.patch
> >
> >
> > As part of the Hadoop stack integration project (Apache Bigtop) we are
> now trying to compile Mahout's upcoming 0.6 release against Hadoop 0.22 and
> 0.23.
> > I'm attaching the patch to Mahout's Maven build system that made it
> possible. I would also like to request help in solving the real issues that
> poped
> > up when we tried to compile Mahout:
> http://bigtop01.cloudera.org:8080/job/Bigtop-hadoop22/COMPONENT=mahout,label=centos5/6/console
> > {noformat}
> > [ERROR]
> /mnt/jenkins/workspace/workspace/Bigtop-hadoop22/COMPONENT/mahout/label/centos5/build/mahout/rpm/BUILD/apache-mahout-c298f70/core/src/test/java/org/apache/mahout/df/mapreduce/partial/Step0JobTest.java:[182,33]
> org.apache.hadoop.mapreduce.TaskAttemptContext is abstract; cannot be
> instantiated
> > [ERROR]
> /mnt/jenkins/workspace/workspace/Bigtop-hadoop22/COMPONENT/mahout/label/centos5/build/mahout/rpm/BUILD/apache-mahout-c298f70/core/src/test/java/org/apache/mahout/df/mapreduce/partial/Step0JobTest.java:[218,9]
> org.apache.mahout.df.mapreduce.partial.Step0JobTest.Step0Context is not
> abstract and does not override abstract method getInputSplit() in
> org.apache.hadoop.mapreduce.MapContext
> > [ERROR]
> /mnt/jenkins/workspace/workspace/Bigtop-hadoop22/COMPONENT/mahout/label/centos5/build/mahout/rpm/BUILD/apache-mahout-c298f70/core/src/test/java/org/apache/mahout/df/mapreduce/partial/Step0JobTest.java:[229,12]
> cannot find symbol
> > [ERROR] symbol : constructor
> Context(org.apache.hadoop.conf.Configuration,org.apache.hadoop.mapreduce.TaskAttemptID,<nulltype>,<nulltype>,<nulltype>,<nulltype>,<nulltype>)
> > [ERROR] location: class org.apache.hadoop.mapreduce.Mapper.Context
> > [ERROR]
> /mnt/jenkins/workspace/workspace/Bigtop-hadoop22/COMPONENT/mahout/label/centos5/build/mahout/rpm/BUILD/apache-mahout-c298f70/core/src/test/java/org/apache/mahout/common/DummyRecordWriter.java:[68,18]
> org.apache.hadoop.mapreduce.Mapper.Context is abstract; cannot be
> instantiated
> > [ERROR]
> /mnt/jenkins/workspace/workspace/Bigtop-hadoop22/COMPONENT/mahout/label/centos5/build/mahout/rpm/BUILD/apache-mahout-c298f70/core/src/test/java/org/apache/mahout/common/DummyRecordWriter.java:[77,19]
> org.apache.hadoop.mapreduce.Reducer.Context is abstract; cannot be
> instantiated
> > [ERROR]
> /mnt/jenkins/workspace/workspace/Bigtop-hadoop22/COMPONENT/mahout/label/centos5/build/mahout/rpm/BUILD/apache-mahout-c298f70/core/src/test/java/org/apache/mahout/df/mapreduce/partial/PartialSequentialBuilder.java:[110,30]
> org.apache.hadoop.mapreduce.TaskAttemptContext is abstract; cannot be
> instantiated
> > [ERROR]
> /mnt/jenkins/workspace/workspace/Bigtop-hadoop22/COMPONENT/mahout/label/centos5/build/mahout/rpm/BUILD/apache-mahout-c298f70/core/src/test/java/org/apache/mahout/df/mapreduce/partial/PartialSequentialBuilder.java:[206,28]
> org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
> > [ERROR]
> /mnt/jenkins/workspace/workspace/Bigtop-hadoop22/COMPONENT/mahout/label/centos5/build/mahout/rpm/BUILD/apache-mahout-c298f70/core/src/test/java/org/apache/mahout/df/mapreduce/partial/PartialSequentialBuilder.java:[227,30]
> org.apache.hadoop.mapreduce.TaskAttemptContext is abstract; cannot be
> instantiated
> > [ERROR]
> /mnt/jenkins/workspace/workspace/Bigtop-hadoop22/COMPONENT/mahout/label/centos5/build/mahout/rpm/BUILD/apache-mahout-c298f70/core/src/test/java/org/apache/mahout/df/mapreduce/partial/MockContext.java:[30,6]
> org.apache.mahout.df.mapreduce.partial.MockContext is not abstract and does
> not override abstract method getInputSplit() in
> org.apache.hadoop.mapreduce.MapContext
> > [ERROR]
> /mnt/jenkins/workspace/workspace/Bigtop-hadoop22/COMPONENT/mahout/label/centos5/build/mahout/rpm/BUILD/apache-mahout-c298f70/core/src/test/java/org/apache/mahout/df/mapreduce/partial/MockContext.java:[38,10]
> cannot find symbol
> > {noformat}
>
> --
> This message is automatically generated by JIRA.
> If you think it was sent incorrectly, please contact your JIRA
> administrators:
> https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
> For more information on JIRA, see: http://www.atlassian.com/software/jira
>
>
>