You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@mahout.apache.org by Dale McDiarmid <da...@ravn.co.uk> on 2011/12/16 19:04:56 UTC

Generating vectors from custom source

Hi,
Im new to the community, mahout and machine learning.  Upto now i've been
experimenting the Gensim libraries to perform LDA analysis on a large
document corpus.  I do, however, have an hadoop cluster to hand and would
like to explore the capabilities of Mahout. I
 have my files in a proprietary format - structured text effectively where a
text file contains multiple document entries.  I understand i need to
produce sparse vectors for the LDA analysis - im happy to write a document
parser and have setup my IntelliJ/maven env with the necessary dependencies. 
Could someone please share some code from a similar requirement to get me
started - an example reading a csv file for example.  All help appreciated.

Thanks
Dale

--
View this message in context: http://lucene.472066.n3.nabble.com/Generating-vectors-from-custom-source-tp3592300p3592300.html
Sent from the Mahout User List mailing list archive at Nabble.com.

Re: Generating vectors from custom source

Posted by tanzek <ta...@gmail.com>.
Beside above,  I wanna to know when I need a FeatureVector and why.
Can you give me an example with source code?

2011/12/22 Isabel Drost <is...@apache.org>

> On 16.12.2011 Dale McDiarmid wrote:
> > Could someone please share some code from a similar
> > requirement to get me started - an example reading a csv file for
> example.
>
> There is also some documentation on the wiki:
>
> https://cwiki.apache.org/MAHOUT/creating-vectors-from-text.html
> https://cwiki.apache.org/MAHOUT/lda-commandline.html
>
> Isabel
>

Re: Generating vectors from custom source

Posted by Isabel Drost <is...@apache.org>.
On 16.12.2011 Dale McDiarmid wrote:
> Could someone please share some code from a similar
> requirement to get me started - an example reading a csv file for example.

There is also some documentation on the wiki:

https://cwiki.apache.org/MAHOUT/creating-vectors-from-text.html
https://cwiki.apache.org/MAHOUT/lda-commandline.html

Isabel

Re: Generating vectors from custom source

Posted by Dale McDiarmid <da...@ravn.co.uk>.
Looking through the utils package it seems i need to be
SequenceFilesFromDirectoryFilter is what i need to look at.  Any examples
appreciated but it looks pretty straight fwd.

--
View this message in context: http://lucene.472066.n3.nabble.com/Generating-vectors-from-custom-source-tp3592300p3592410.html
Sent from the Mahout User List mailing list archive at Nabble.com.

Re: Generating vectors from custom source

Posted by Timothy Potter <th...@gmail.com>.
Hi Dale,

Have a look at:

integration/src/main/java/org/apache/mahout/text/MailArchivesClusteringAnalyzer.java
integration/src/main/java/org/apache/mahout/text/SequenceFilesFromMailArchives.java

Cheers,
Tim

On Fri, Dec 16, 2011 at 12:04 PM, Dale McDiarmid <da...@ravn.co.uk> wrote:

> Hi,
> Im new to the community, mahout and machine learning.  Upto now i've been
> experimenting the Gensim libraries to perform LDA analysis on a large
> document corpus.  I do, however, have an hadoop cluster to hand and would
> like to explore the capabilities of Mahout. I
>  have my files in a proprietary format - structured text effectively where
> a
> text file contains multiple document entries.  I understand i need to
> produce sparse vectors for the LDA analysis - im happy to write a document
> parser and have setup my IntelliJ/maven env with the necessary
> dependencies.
> Could someone please share some code from a similar requirement to get me
> started - an example reading a csv file for example.  All help appreciated.
>
> Thanks
> Dale
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Generating-vectors-from-custom-source-tp3592300p3592300.html
> Sent from the Mahout User List mailing list archive at Nabble.com.
>