You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@mahout.apache.org by Dale McDiarmid <da...@ravn.co.uk> on 2011/12/16 19:04:56 UTC
Generating vectors from custom source
Hi,
Im new to the community, mahout and machine learning. Upto now i've been
experimenting the Gensim libraries to perform LDA analysis on a large
document corpus. I do, however, have an hadoop cluster to hand and would
like to explore the capabilities of Mahout. I
have my files in a proprietary format - structured text effectively where a
text file contains multiple document entries. I understand i need to
produce sparse vectors for the LDA analysis - im happy to write a document
parser and have setup my IntelliJ/maven env with the necessary dependencies.
Could someone please share some code from a similar requirement to get me
started - an example reading a csv file for example. All help appreciated.
Thanks
Dale
--
View this message in context: http://lucene.472066.n3.nabble.com/Generating-vectors-from-custom-source-tp3592300p3592300.html
Sent from the Mahout User List mailing list archive at Nabble.com.
Re: Generating vectors from custom source
Posted by tanzek <ta...@gmail.com>.
Beside above, I wanna to know when I need a FeatureVector and why.
Can you give me an example with source code?
2011/12/22 Isabel Drost <is...@apache.org>
> On 16.12.2011 Dale McDiarmid wrote:
> > Could someone please share some code from a similar
> > requirement to get me started - an example reading a csv file for
> example.
>
> There is also some documentation on the wiki:
>
> https://cwiki.apache.org/MAHOUT/creating-vectors-from-text.html
> https://cwiki.apache.org/MAHOUT/lda-commandline.html
>
> Isabel
>
Re: Generating vectors from custom source
Posted by Isabel Drost <is...@apache.org>.
On 16.12.2011 Dale McDiarmid wrote:
> Could someone please share some code from a similar
> requirement to get me started - an example reading a csv file for example.
There is also some documentation on the wiki:
https://cwiki.apache.org/MAHOUT/creating-vectors-from-text.html
https://cwiki.apache.org/MAHOUT/lda-commandline.html
Isabel
Re: Generating vectors from custom source
Posted by Dale McDiarmid <da...@ravn.co.uk>.
Looking through the utils package it seems i need to be
SequenceFilesFromDirectoryFilter is what i need to look at. Any examples
appreciated but it looks pretty straight fwd.
--
View this message in context: http://lucene.472066.n3.nabble.com/Generating-vectors-from-custom-source-tp3592300p3592410.html
Sent from the Mahout User List mailing list archive at Nabble.com.
Re: Generating vectors from custom source
Posted by Timothy Potter <th...@gmail.com>.
Hi Dale,
Have a look at:
integration/src/main/java/org/apache/mahout/text/MailArchivesClusteringAnalyzer.java
integration/src/main/java/org/apache/mahout/text/SequenceFilesFromMailArchives.java
Cheers,
Tim
On Fri, Dec 16, 2011 at 12:04 PM, Dale McDiarmid <da...@ravn.co.uk> wrote:
> Hi,
> Im new to the community, mahout and machine learning. Upto now i've been
> experimenting the Gensim libraries to perform LDA analysis on a large
> document corpus. I do, however, have an hadoop cluster to hand and would
> like to explore the capabilities of Mahout. I
> have my files in a proprietary format - structured text effectively where
> a
> text file contains multiple document entries. I understand i need to
> produce sparse vectors for the LDA analysis - im happy to write a document
> parser and have setup my IntelliJ/maven env with the necessary
> dependencies.
> Could someone please share some code from a similar requirement to get me
> started - an example reading a csv file for example. All help appreciated.
>
> Thanks
> Dale
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Generating-vectors-from-custom-source-tp3592300p3592300.html
> Sent from the Mahout User List mailing list archive at Nabble.com.
>