You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mahout.apache.org by Jake Mannix <ja...@gmail.com> on 2010/02/12 18:51:13 UTC
Re: [jira] Resolved: (MAHOUT-185) Add mahout shell script for easy
launching of various algorithms
Sean was pushing out tickets to 0.4 he felt had timed out, but now that
you've checked that one it, it clearly is done for 0.3.
-jake
On Fri, Feb 12, 2010 at 9:48 AM, Grant Ingersoll <gs...@apache.org>wrote:
> Wasn't this 0.3 or did I miss a release?
>
> Begin forwarded message:
>
> > From: "Sean Owen (JIRA)" <ji...@apache.org>
> > Date: February 12, 2010 8:58:27 AM EST
> > To: mahout-dev@lucene.apache.org
> > Subject: [jira] Resolved: (MAHOUT-185) Add mahout shell script for easy
> launching of various algorithms
> > Reply-To: mahout-dev@lucene.apache.org
> >
> >
> > [
> https://issues.apache.org/jira/browse/MAHOUT-185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel]
> >
> > Sean Owen resolved MAHOUT-185.
> > ------------------------------
> >
> > Resolution: Fixed
> > Fix Version/s: (was: 0.3)
> > 0.4
> > Assignee: Robin Anil (was: Grant Ingersoll)
> >
> >> Add mahout shell script for easy launching of various algorithms
> >> ----------------------------------------------------------------
> >>
> >> Key: MAHOUT-185
> >> URL: https://issues.apache.org/jira/browse/MAHOUT-185
> >> Project: Mahout
> >> Issue Type: New Feature
> >> Environment: linux, bash
> >> Reporter: Robin Anil
> >> Assignee: Robin Anil
> >> Fix For: 0.4
> >>
> >> Attachments: MAHOUT-185.patch
> >>
> >>
> >> Currently, Each algorithm has a different point of entry. At its too
> complicated to understand and launch each one. A mahout shell script needs
> to be made in the bin directory which does something like the following
> >> mahout classify -algorithm bayes [OPTIONS]
> >> mahout cluster -algorithm canopy [OPTIONS]
> >> mahout fpm -algorithm pfpgrowth [OPTIONS]
> >> mahout taste -algorithm slopeone [OPTIONS]
> >> mahout misc -algorithm createVectorsFromText [OPTIONS]
> >> mahout examples WikipediaExample
> >
> > --
> > This message is automatically generated by JIRA.
> > -
> > You can reply to this email to add a comment to the issue online.
> >
>
> --------------------------
> Grant Ingersoll
> http://www.lucidimagination.com/
>
> Search the Lucene ecosystem using Solr/Lucene:
> http://www.lucidimagination.com/search
>
>
Re: [jira] Resolved: (MAHOUT-185) Add mahout shell script for easy
launching of various algorithms
Posted by Robin Anil <ro...@gmail.com>.
Yeah for now the former. Am I right deneche?
On Sat, Feb 13, 2010 at 12:11 AM, Drew Farris <dr...@gmail.com> wrote:
> Ok, this is better: http://pastebin.com/d3783fa0f
>
> The only overlap is:
>
> BuildForest org.apache.mahout.df.mapred.BuildForest
> BuildForest org.apache.mahout.df.mapreduce.BuildForest
>
> We only want to choose one of those anyway, right?
>
> On Fri, Feb 12, 2010 at 1:36 PM, Drew Farris <dr...@gmail.com>
> wrote:
> > bah, no there are some dupes in there.
> >
> > On Fri, Feb 12, 2010 at 1:35 PM, Drew Farris <dr...@gmail.com>
> wrote:
> >> how about this:
> >> http://pastebin.com/m55f8dced
> >>
> >> On Fri, Feb 12, 2010 at 1:26 PM, Robin Anil <ro...@gmail.com>
> wrote:
> >>> We can always rename them. Its an easier way out. The other way out is
> to
> >>> keep a mapping list.
> >>>
> >>> Plenty of options on the table. Just need to choose one.
> >>>
> >
>
Re: [jira] Resolved: (MAHOUT-185) Add mahout shell script for easy
launching of various algorithms
Posted by Drew Farris <dr...@gmail.com>.
Ok, this is better: http://pastebin.com/d3783fa0f
The only overlap is:
BuildForest org.apache.mahout.df.mapred.BuildForest
BuildForest org.apache.mahout.df.mapreduce.BuildForest
We only want to choose one of those anyway, right?
On Fri, Feb 12, 2010 at 1:36 PM, Drew Farris <dr...@gmail.com> wrote:
> bah, no there are some dupes in there.
>
> On Fri, Feb 12, 2010 at 1:35 PM, Drew Farris <dr...@gmail.com> wrote:
>> how about this:
>> http://pastebin.com/m55f8dced
>>
>> On Fri, Feb 12, 2010 at 1:26 PM, Robin Anil <ro...@gmail.com> wrote:
>>> We can always rename them. Its an easier way out. The other way out is to
>>> keep a mapping list.
>>>
>>> Plenty of options on the table. Just need to choose one.
>>>
>
Re: [jira] Resolved: (MAHOUT-185) Add mahout shell script for easy
launching of various algorithms
Posted by Drew Farris <dr...@gmail.com>.
bah, no there are some dupes in there.
On Fri, Feb 12, 2010 at 1:35 PM, Drew Farris <dr...@gmail.com> wrote:
> how about this:
> http://pastebin.com/m55f8dced
>
> On Fri, Feb 12, 2010 at 1:26 PM, Robin Anil <ro...@gmail.com> wrote:
>> We can always rename them. Its an easier way out. The other way out is to
>> keep a mapping list.
>>
>> Plenty of options on the table. Just need to choose one.
>>
Re: [jira] Resolved: (MAHOUT-185) Add mahout shell script for easy
launching of various algorithms
Posted by Drew Farris <dr...@gmail.com>.
how about this:
http://pastebin.com/m55f8dced
On Fri, Feb 12, 2010 at 1:26 PM, Robin Anil <ro...@gmail.com> wrote:
> We can always rename them. Its an easier way out. The other way out is to
> keep a mapping list.
>
> Plenty of options on the table. Just need to choose one.
>
> On Fri, Feb 12, 2010 at 11:53 PM, Jake Mannix <ja...@gmail.com> wrote:
>
>> Don't we have a few examples of "Driver" as the class name?
>>
>> On Fri, Feb 12, 2010 at 10:21 AM, Robin Anil <ro...@gmail.com> wrote:
>>
>> > To Simply things how about just use the ClassName. Everything seems
>> unique?
>> >
>> > mahout TestClassifier
>> > mahout KmeansDriver
>> > mahout ClusterDumper.
>> >
>> > its easier to explain to people if the EntryPoints a verbose enough. Like
>> > SparseVectorsFromSequenceFiles
>> >
>> > and have a script to generate mappings and detect collisions.
>> >
>> > Robin
>> >
>>
>
Re: [jira] Resolved: (MAHOUT-185) Add mahout shell script for easy
launching of various algorithms
Posted by Robin Anil <ro...@gmail.com>.
We can always rename them. Its an easier way out. The other way out is to
keep a mapping list.
Plenty of options on the table. Just need to choose one.
On Fri, Feb 12, 2010 at 11:53 PM, Jake Mannix <ja...@gmail.com> wrote:
> Don't we have a few examples of "Driver" as the class name?
>
> On Fri, Feb 12, 2010 at 10:21 AM, Robin Anil <ro...@gmail.com> wrote:
>
> > To Simply things how about just use the ClassName. Everything seems
> unique?
> >
> > mahout TestClassifier
> > mahout KmeansDriver
> > mahout ClusterDumper.
> >
> > its easier to explain to people if the EntryPoints a verbose enough. Like
> > SparseVectorsFromSequenceFiles
> >
> > and have a script to generate mappings and detect collisions.
> >
> > Robin
> >
>
Re: [jira] Resolved: (MAHOUT-185) Add mahout shell script for easy
launching of various algorithms
Posted by Drew Farris <dr...@gmail.com>.
And 'Job' in the synthetic control stuff.
On Fri, Feb 12, 2010 at 1:23 PM, Jake Mannix <ja...@gmail.com> wrote:
> Don't we have a few examples of "Driver" as the class name?
>
> On Fri, Feb 12, 2010 at 10:21 AM, Robin Anil <ro...@gmail.com> wrote:
>
>> To Simply things how about just use the ClassName. Everything seems unique?
>>
>> mahout TestClassifier
>> mahout KmeansDriver
>> mahout ClusterDumper.
>>
>> its easier to explain to people if the EntryPoints a verbose enough. Like
>> SparseVectorsFromSequenceFiles
>>
>> and have a script to generate mappings and detect collisions.
>>
>> Robin
>>
>
Re: [jira] Resolved: (MAHOUT-185) Add mahout shell script for easy
launching of various algorithms
Posted by Jake Mannix <ja...@gmail.com>.
Don't we have a few examples of "Driver" as the class name?
On Fri, Feb 12, 2010 at 10:21 AM, Robin Anil <ro...@gmail.com> wrote:
> To Simply things how about just use the ClassName. Everything seems unique?
>
> mahout TestClassifier
> mahout KmeansDriver
> mahout ClusterDumper.
>
> its easier to explain to people if the EntryPoints a verbose enough. Like
> SparseVectorsFromSequenceFiles
>
> and have a script to generate mappings and detect collisions.
>
> Robin
>
Re: [jira] Resolved: (MAHOUT-185) Add mahout shell script for easy
launching of various algorithms
Posted by Robin Anil <ro...@gmail.com>.
To Simply things how about just use the ClassName. Everything seems unique?
mahout TestClassifier
mahout KmeansDriver
mahout ClusterDumper.
its easier to explain to people if the EntryPoints a verbose enough. Like
SparseVectorsFromSequenceFiles
and have a script to generate mappings and detect collisions.
Robin
Re: [jira] Resolved: (MAHOUT-185) Add mahout shell script for easy
launching of various algorithms
Posted by Drew Farris <dr...@gmail.com>.
For what it is worth, if we don't get aliases for all the classes in
there, you can still to stuff like
mahout org.apache.mahout.fpm.pfpgrowth.DeliciousTagsExample ...
..and the script will setup the classpath properly and execute the
class -- just sayin' :)
I can take some of these, but what sort of naming pattern should we
use? Something like the following?:
dfBriemanExample
dfBuildForest
fpmDeliciousTagsExample
scKmeans (for syntheticcontrol.kmeans.Job)
kmeans
Do we need all of the examples?
What's the difference between the syntheticcontrol and other
clustering stuff, e.g:
org.apache.mahout.clustering.syntheticcontrol.kmeans.Job
org.apache.mahout.clustering.kmeans.KMeansDriver
While we're at it, If someone's editing and committing, there's a
spelling error in the comments:
HADOOP_CONFIF_DIR which should be HADOOP_CONF_DIR
On Fri, Feb 12, 2010 at 1:00 PM, Robin Anil <ro...@gmail.com> wrote:
> A Sorted Version here http://pastebin.com/m25252110. Just add the name next
> to the class
>
> Robin
>
Re: [jira] Resolved: (MAHOUT-185) Add mahout shell script for easy
launching of various algorithms
Posted by Sean Owen <sr...@gmail.com>.
Oops I marked it Fixed for 0.4. It should be Fixed for 0.3 -- or in
progress for 0.4 if you want to keep it open. Change accordingly.
On Fri, Feb 12, 2010 at 6:03 PM, Jake Mannix <ja...@gmail.com> wrote:
> I have code for a more maintainable way to do this, at least on the level of
> not
> having to change code / recompile to add more, using some property files,
> but
> it's not done yet / tested fully yet. We can open a new ticket for
> continuing,
> as it's not really a shell script.
>
Re: [jira] Resolved: (MAHOUT-185) Add mahout shell script for easy
launching of various algorithms
Posted by Jake Mannix <ja...@gmail.com>.
I have code for a more maintainable way to do this, at least on the level of
not
having to change code / recompile to add more, using some property files,
but
it's not done yet / tested fully yet. We can open a new ticket for
continuing,
as it's not really a shell script.
-jake
On Fri, Feb 12, 2010 at 10:00 AM, Robin Anil <ro...@gmail.com> wrote:
> A Sorted Version here http://pastebin.com/m25252110. Just add the name
> next
> to the class
>
> Robin
>
Re: [jira] Resolved: (MAHOUT-185) Add mahout shell script for easy
launching of various algorithms
Posted by Robin Anil <ro...@gmail.com>.
A Sorted Version here http://pastebin.com/m25252110. Just add the name next
to the class
Robin
Re: [jira] Resolved: (MAHOUT-185) Add mahout shell script for easy launching of various algorithms
Posted by Grant Ingersoll <gs...@apache.org>.
On Feb 12, 2010, at 2:38 PM, Grant Ingersoll wrote:
> FWIW, the script has a way that you can pass in the classname of the program to run.
D'oh, should always finish reading the thread before responding ;-)
-Grant
Re: [jira] Resolved: (MAHOUT-185) Add mahout shell script for easy launching of various algorithms
Posted by Grant Ingersoll <gs...@apache.org>.
FWIW, the script has a way that you can pass in the classname of the program to run.
-Grant
On Feb 12, 2010, at 12:57 PM, Robin Anil wrote:
> It not complete to really say its done. How about someone picking out all
> main classes and putting it there.
> I just did this
>
> find . |grep -v svn |grep "java$" | xargs grep -i "main(String" | cut -d " "
> -f1|cut -d':' -f1 |cut -d "." -f2 |cut -d"/" -f 2-| sed 's/\//./g' |cut
> -d"." -f 5-
>
> Can someone help naming them
>
> org.apache.mahout.cf.taste.hadoop.item.RecommenderJob
> org.apache.mahout.cf.taste.hadoop.pseudo.RecommenderJob
> org.apache.mahout.cf.taste.hadoop.slopeone.SlopeOneAverageDiffsJob
> org.apache.mahout.classifier.bayes.TestClassifier
> org.apache.mahout.classifier.bayes.TrainClassifier
> org.apache.mahout.classifier.BayesFileFormatter
> org.apache.mahout.classifier.Classify
> org.apache.mahout.clustering.canopy.CanopyClusteringJob
> org.apache.mahout.clustering.canopy.CanopyDriver
> org.apache.mahout.clustering.canopy.ClusterDriver
> org.apache.mahout.clustering.dirichlet.DirichletDriver
> org.apache.mahout.clustering.dirichlet.DirichletJob
> org.apache.mahout.clustering.fuzzykmeans.FuzzyKMeansDriver
> org.apache.mahout.clustering.fuzzykmeans.FuzzyKMeansJob
> org.apache.mahout.clustering.kmeans.KMeansDriver
> org.apache.mahout.clustering.lda.LDADriver
> org.apache.mahout.clustering.lda.LDAPrintTopics
> org.apache.mahout.clustering.meanshift.MeanShiftCanopyDriver
> org.apache.mahout.clustering.meanshift.MeanShiftCanopyJob
> org.apache.mahout.df.tools.Describe
> org.apache.mahout.df.tools.Frequencies
> org.apache.mahout.df.tools.UDistrib
> org.apache.mahout.fpm.pfpgrowth.FPGrowthDriver
> org.apache.mahout.cf.taste.impl.LoadTest
> org.apache.mahout.clustering.kmeans.SimpleKMeansClustering
> org.apache.mahout.cf.taste.example.bookcrossing.BookCrossingRecommenderEvaluatorRunner
> org.apache.mahout.cf.taste.example.grouplens.GroupLensRecommenderEvaluatorRunner
> org.apache.mahout.cf.taste.example.jester.JesterRecommenderEvaluatorRunner
> org.apache.mahout.cf.taste.example.netflix.NetflixRecommenderEvaluatorRunner
> org.apache.mahout.cf.taste.example.netflix.TransposeToByUser
> org.apache.mahout.classifier.bayes.PrepareARFFBinaryFeatures
> org.apache.mahout.classifier.bayes.PrepareTwentyNewsgroups
> org.apache.mahout.classifier.bayes.WikipediaDatasetCreatorDriver
> org.apache.mahout.classifier.bayes.WikipediaXmlSplitter
> org.apache.mahout.clustering.canopy.DisplayCanopy
> org.apache.mahout.clustering.dirichlet.Display2dASNDirichlet
> org.apache.mahout.clustering.dirichlet.DisplayASNDirichlet
> org.apache.mahout.clustering.dirichlet.DisplayASNOutputState
> org.apache.mahout.clustering.dirichlet.DisplayDirichlet
> org.apache.mahout.clustering.dirichlet.DisplayNDirichlet
> org.apache.mahout.clustering.dirichlet.DisplayOutputState
> org.apache.mahout.clustering.dirichlet.DisplaySNDirichlet
> org.apache.mahout.clustering.fuzzykmeans.DisplayFuzzyKMeans
> org.apache.mahout.clustering.kmeans.DisplayKMeans
> org.apache.mahout.clustering.meanshift.DisplayMeanShift
> org.apache.mahout.clustering.syntheticcontrol.canopy.InputDriver
> org.apache.mahout.clustering.syntheticcontrol.canopy.Job
> org.apache.mahout.clustering.syntheticcontrol.dirichlet.Job
> org.apache.mahout.clustering.syntheticcontrol.kmeans.Job
> org.apache.mahout.clustering.syntheticcontrol.meanshift.InputDriver
> org.apache.mahout.clustering.syntheticcontrol.meanshift.Job
> org.apache.mahout.clustering.syntheticcontrol.meanshift.OutputDriver
> org.apache.mahout.df.BreimanExample
> org.apache.mahout.df.mapred.BuildForest
> org.apache.mahout.df.mapreduce.BuildForest
> org.apache.mahout.fpm.pfpgrowth.DeliciousTagsExample
> org.apache.mahout.ga.watchmaker.cd.CDGA
> org.apache.mahout.ga.watchmaker.cd.tool.CDInfosTool
> org.apache.mahout.ga.watchmaker.travellingsalesman.TravellingSalesman
> org.apache.mahout.text.SequenceFilesFromDirectory
> org.apache.mahout.text.WikipediaToSequenceFile
> org.apache.mahout.math.decomposer.hebbian.HebbianSolver
> org.apache.mahout.math.jet.stat.quantile.QuantileCalc
> org.apache.mahout.text.SparseVectorsFromSequenceFiles
> org.apache.mahout.utils.clustering.ClusterDumper
> org.apache.mahout.utils.nlp.collocations.llr.CollocDriver
> org.apache.mahout.utils.SequenceFileDumper
> org.apache.mahout.utils.vectors.arff.Driver
> org.apache.mahout.utils.vectors.lucene.ClusterLabels
> org.apache.mahout.utils.vectors.lucene.Driver
> org.apache.mahout.utils.vectors.VectorDumper
--------------------------
Grant Ingersoll
http://www.lucidimagination.com/
Search the Lucene ecosystem using Solr/Lucene: http://www.lucidimagination.com/search
Re: [jira] Resolved: (MAHOUT-185) Add mahout shell script for easy
launching of various algorithms
Posted by Robin Anil <ro...@gmail.com>.
It not complete to really say its done. How about someone picking out all
main classes and putting it there.
I just did this
find . |grep -v svn |grep "java$" | xargs grep -i "main(String" | cut -d " "
-f1|cut -d':' -f1 |cut -d "." -f2 |cut -d"/" -f 2-| sed 's/\//./g' |cut
-d"." -f 5-
Can someone help naming them
org.apache.mahout.cf.taste.hadoop.item.RecommenderJob
org.apache.mahout.cf.taste.hadoop.pseudo.RecommenderJob
org.apache.mahout.cf.taste.hadoop.slopeone.SlopeOneAverageDiffsJob
org.apache.mahout.classifier.bayes.TestClassifier
org.apache.mahout.classifier.bayes.TrainClassifier
org.apache.mahout.classifier.BayesFileFormatter
org.apache.mahout.classifier.Classify
org.apache.mahout.clustering.canopy.CanopyClusteringJob
org.apache.mahout.clustering.canopy.CanopyDriver
org.apache.mahout.clustering.canopy.ClusterDriver
org.apache.mahout.clustering.dirichlet.DirichletDriver
org.apache.mahout.clustering.dirichlet.DirichletJob
org.apache.mahout.clustering.fuzzykmeans.FuzzyKMeansDriver
org.apache.mahout.clustering.fuzzykmeans.FuzzyKMeansJob
org.apache.mahout.clustering.kmeans.KMeansDriver
org.apache.mahout.clustering.lda.LDADriver
org.apache.mahout.clustering.lda.LDAPrintTopics
org.apache.mahout.clustering.meanshift.MeanShiftCanopyDriver
org.apache.mahout.clustering.meanshift.MeanShiftCanopyJob
org.apache.mahout.df.tools.Describe
org.apache.mahout.df.tools.Frequencies
org.apache.mahout.df.tools.UDistrib
org.apache.mahout.fpm.pfpgrowth.FPGrowthDriver
org.apache.mahout.cf.taste.impl.LoadTest
org.apache.mahout.clustering.kmeans.SimpleKMeansClustering
org.apache.mahout.cf.taste.example.bookcrossing.BookCrossingRecommenderEvaluatorRunner
org.apache.mahout.cf.taste.example.grouplens.GroupLensRecommenderEvaluatorRunner
org.apache.mahout.cf.taste.example.jester.JesterRecommenderEvaluatorRunner
org.apache.mahout.cf.taste.example.netflix.NetflixRecommenderEvaluatorRunner
org.apache.mahout.cf.taste.example.netflix.TransposeToByUser
org.apache.mahout.classifier.bayes.PrepareARFFBinaryFeatures
org.apache.mahout.classifier.bayes.PrepareTwentyNewsgroups
org.apache.mahout.classifier.bayes.WikipediaDatasetCreatorDriver
org.apache.mahout.classifier.bayes.WikipediaXmlSplitter
org.apache.mahout.clustering.canopy.DisplayCanopy
org.apache.mahout.clustering.dirichlet.Display2dASNDirichlet
org.apache.mahout.clustering.dirichlet.DisplayASNDirichlet
org.apache.mahout.clustering.dirichlet.DisplayASNOutputState
org.apache.mahout.clustering.dirichlet.DisplayDirichlet
org.apache.mahout.clustering.dirichlet.DisplayNDirichlet
org.apache.mahout.clustering.dirichlet.DisplayOutputState
org.apache.mahout.clustering.dirichlet.DisplaySNDirichlet
org.apache.mahout.clustering.fuzzykmeans.DisplayFuzzyKMeans
org.apache.mahout.clustering.kmeans.DisplayKMeans
org.apache.mahout.clustering.meanshift.DisplayMeanShift
org.apache.mahout.clustering.syntheticcontrol.canopy.InputDriver
org.apache.mahout.clustering.syntheticcontrol.canopy.Job
org.apache.mahout.clustering.syntheticcontrol.dirichlet.Job
org.apache.mahout.clustering.syntheticcontrol.kmeans.Job
org.apache.mahout.clustering.syntheticcontrol.meanshift.InputDriver
org.apache.mahout.clustering.syntheticcontrol.meanshift.Job
org.apache.mahout.clustering.syntheticcontrol.meanshift.OutputDriver
org.apache.mahout.df.BreimanExample
org.apache.mahout.df.mapred.BuildForest
org.apache.mahout.df.mapreduce.BuildForest
org.apache.mahout.fpm.pfpgrowth.DeliciousTagsExample
org.apache.mahout.ga.watchmaker.cd.CDGA
org.apache.mahout.ga.watchmaker.cd.tool.CDInfosTool
org.apache.mahout.ga.watchmaker.travellingsalesman.TravellingSalesman
org.apache.mahout.text.SequenceFilesFromDirectory
org.apache.mahout.text.WikipediaToSequenceFile
org.apache.mahout.math.decomposer.hebbian.HebbianSolver
org.apache.mahout.math.jet.stat.quantile.QuantileCalc
org.apache.mahout.text.SparseVectorsFromSequenceFiles
org.apache.mahout.utils.clustering.ClusterDumper
org.apache.mahout.utils.nlp.collocations.llr.CollocDriver
org.apache.mahout.utils.SequenceFileDumper
org.apache.mahout.utils.vectors.arff.Driver
org.apache.mahout.utils.vectors.lucene.ClusterLabels
org.apache.mahout.utils.vectors.lucene.Driver
org.apache.mahout.utils.vectors.VectorDumper