Re: mahout command seq2sparse

[Tharindu Rusira <th...@gmail.com>]

On Thu, Jan 30, 2014 at 10:20 PM, qiaoresearcher
<qi...@gmail.com>wrote:

> Andrew Musselman and Suneel Marthi,
>
> Thank you two and I really appreciate it! Wish you two very successful in
> your company!
>
> I was wondering is there any documentation or tutorial which can go through
> how the Hadoop is written, not like the definition guide one.
> For example, from the very beginning when we run start-all.sh to setup
> Hadoop till the Hadoop is ready to run MapReduce jobs.
>
Hi ,
I think Hadoop official website provides everything you need here. 

Assuming you are using Hadoop 1.2.1 stable release(since Mahout does not
support Hadoop 2.x yet), [1] would guide you to configure your Hadoop
installation.

[1] http://hadoop.apache.org/docs/r1.2.1/index.html


> is it possible to debug the every step from the very beginning to the end?
>
> When you have a Java program, you can know exactly every step by debug it
> in Java IDE. Is it possible to debug every step in Hadoop from
> start-all.sh, or even format namenode, till the end of some Mapreduce job?
>
What you are asking is not so clear to me, 

however you can format namenode using $HADOOP_HOME/bin/hadoop namenode
-format
Please make sure that you stop your Hadoop servers before formatting
namenode. ($HADOOP_HOME/bin/.stop-all.sh)

If you want to find more details related to Hadoop implementation, Hadoop
mailing list would be the perfect place to get guidance.

>
> Any suggestion or guidance will be deeply appreciated.
>
> Hope this helps,
Regards


>
>
>
> On Wed, Jan 29, 2014 at 6:16 PM, Andrew Musselman <
> andrew.musselman@gmail.com> wrote:
>
> > Those aliases are defined in src/conf/driver.classes.default.props.
> >
> > That language is shell-scripting, e.g. bash.
> >
> >
> > On Wed, Jan 29, 2014 at 2:15 PM, qiaoresearcher <
> qiaoresearcher@gmail.com
> > >wrote:
> >
> > > when run the command like:
> > >
> > > mahout seq2sparse -i inputfile -o outputfile
> > >
> > > where is the command seq2sparse defined? how does the system know to
> > > actually run the SparseFileFromSequenceFile class?
> > >
> > >  what is the language used in the command Mahout such as the language
> > given
> > > below:
> > >
> > > ========================================
> > >
> > > THIS="$0"
> > > while [ -h "$THIS" ]; do
> > >   ls=`ls -ld "$THIS"`
> > >   link=`expr "$ls" : '.*-> \(.*\)$'`
> > >   if expr "$link" : '.*/.*' > /dev/null; then
> > >     THIS="$link"
> > >   else
> > >     THIS=`dirname "$THIS"`/"$link"
> > >   fi
> > > done
> > >
> > > IS_CORE=0
> > > if [ "$1" == "-core" ] ; then
> > >   IS_CORE=1
> > >   shift
> > > fi
> > >
> > > if [ "$MAHOUT_CORE" != "" ]; then
> > >   IS_CORE=1
> > > fi
> > >
> > > # some directories
> > > THIS_DIR=`dirname "$THIS"`
> > > MAHOUT_HOME=`cd "$THIS_DIR/.." ; pwd`
> > >
> > > # some Java parameters
> > > if [ "$MAHOUT_JAVA_HOME" != "" ]; then
> > >   #echo "run java in $MAHOUT_JAVA_HOME"
> > >   JAVA_HOME=$MAHOUT_JAVA_HOME
> > > fi
> > >
> >
>



-- 
M.P. Tharindu Rusira Kumara

Department of Computer Science and Engineering,
University of Moratuwa,
Sri Lanka.
+94757033733
www.tharindu-rusira.blogspot.com