You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@mahout.apache.org by cdwzwd <do...@gmail.com> on 2010/08/30 11:52:51 UTC

Re: Proper way to dump kmeans clusters?

Hi
I hava the same problem with you.I want to know what is the meaning of "copy
the files from hdts to local
filesystem." I got the whole output of my kemans algorithm.And then I run
the clusterDump command using all the path in my LocalFileSystem. But there
is only an Exception.

running on hadoop, using HADOOP_HOME=/root/software/hadoop-0.20.2 and
HADOOP_CONF_DIR=/root/software/hadoop-0.20.2/conf
10/08/27 10:21:46 ERROR clustering.ClusterDumper: Exception
org.apache.commons.cli2.OptionException: Unexpected sequencefile while
processing Options
    at org.apache.commons.cli2.commandline.Parser.parse(Parser.java:99)
    at
org.apache.mahout.utils.clustering.ClusterDumper.main(ClusterDumper.java:248)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at
org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
    at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
    at org.apache.mahout.driver.MahoutDriver.main(MahoutDriver.java:172)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
-- 
View this message in context: http://lucene.472066.n3.nabble.com/Proper-way-to-dump-kmeans-clusters-tp638851p1387851.html
Sent from the Mahout Developer List mailing list archive at Nabble.com.

Re: Proper way to dump kmeans clusters?

Posted by Joe Kumar <jo...@gmail.com>.

You can also have a look at "Retrieving the Output" section in
https://cwiki.apache.org/confluence/display/MAHOUT/ClusteringYourData.
It has references on the options that needs to be set when using cluster
dumper.

As Jeff has mentioned, you would need to get the o/p files from hdfs to your
local machine before you run clusterdump.
You can reference "Read / Analyze Output" section in
https://cwiki.apache.org/confluence/display/MAHOUT/Clustering+of+synthetic+control+datafor
the actual command.


On Mon, Aug 30, 2010 at 12:33 PM, Jeff Eastman
<jd...@windwardsolutions.com>wrote:

>  On 8/30/10 2:52 AM, cdwzwd wrote:
>
>> Hi
>> I hava the same problem with you.I want to know what is the meaning of
>> "copy
>> the files from hdts to local
>> filesystem."
>>
> This means you need to get the files out of Hadoop's hdfs and into local
> files on your machine. Something like $HADOOP_HOME/bin/hadoop fs -get <hdfs
> file descriptor> <local file descriptor>.
>
>  I got the whole output of my kemans algorithm.And then I run
>> the clusterDump command using all the path in my LocalFileSystem. But
>> there
>> is only an Exception.
>>
> You will need to provide more information to get more help, such as the
> command line you are invoking, where your files are, etc. The clusterdumper
> will run either against the local file system (in stand alone mode) or
> against the Hadoop file system (if Hadoop is running).
>
>  running on hadoop, using HADOOP_HOME=/root/software/hadoop-0.20.2 and
>> HADOOP_CONF_DIR=/root/software/hadoop-0.20.2/conf
>> 10/08/27 10:21:46 ERROR clustering.ClusterDumper: Exception
>> org.apache.commons.cli2.OptionException: Unexpected sequencefile while
>> processing Options
>>     at org.apache.commons.cli2.commandline.Parser.parse(Parser.java:99)
>>     at
>>
>> org.apache.mahout.utils.clustering.ClusterDumper.main(ClusterDumper.java:248)
>>     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>     at
>>
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>     at
>>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>     at java.lang.reflect.Method.invoke(Method.java:597)
>>     at
>>
>> org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
>>     at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
>>     at org.apache.mahout.driver.MahoutDriver.main(MahoutDriver.java:172)
>>     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>     at
>>
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>     at
>>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>     at java.lang.reflect.Method.invoke(Method.java:597)
>>     at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>>
>
>

Re: Proper way to dump kmeans clusters?

Posted by Jeff Eastman <jd...@windwardsolutions.com>.

  On 8/30/10 2:52 AM, cdwzwd wrote:
> Hi
> I hava the same problem with you.I want to know what is the meaning of "copy
> the files from hdts to local
> filesystem."
This means you need to get the files out of Hadoop's hdfs and into local 
files on your machine. Something like $HADOOP_HOME/bin/hadoop fs -get 
<hdfs file descriptor> <local file descriptor>.
> I got the whole output of my kemans algorithm.And then I run
> the clusterDump command using all the path in my LocalFileSystem. But there
> is only an Exception.
You will need to provide more information to get more help, such as the 
command line you are invoking, where your files are, etc. The 
clusterdumper will run either against the local file system (in stand 
alone mode) or against the Hadoop file system (if Hadoop is running).
> running on hadoop, using HADOOP_HOME=/root/software/hadoop-0.20.2 and
> HADOOP_CONF_DIR=/root/software/hadoop-0.20.2/conf
> 10/08/27 10:21:46 ERROR clustering.ClusterDumper: Exception
> org.apache.commons.cli2.OptionException: Unexpected sequencefile while
> processing Options
>      at org.apache.commons.cli2.commandline.Parser.parse(Parser.java:99)
>      at
> org.apache.mahout.utils.clustering.ClusterDumper.main(ClusterDumper.java:248)
>      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>      at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>      at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at
> org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
>      at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
>      at org.apache.mahout.driver.MahoutDriver.main(MahoutDriver.java:172)
>      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>      at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>      at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.util.RunJar.main(RunJar.java:156)