You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@opennlp.apache.org by William Colen <wi...@gmail.com> on 2016/06/28 21:23:53 UTC
DeepLearning4J as a ML for OpenNLP
Hi,
Do you think it would be possible to implement a ML based on DL4J?
http://deeplearning4j.org/
Thank you
William
Re: DeepLearning4J as a ML for OpenNLP
Posted by Anthony Beylerian <an...@gmail.com>.
There's also Doc2vec ::
http://deeplearning4j.org/doc2vec.html
Which could work as well.
On Wed, Jun 29, 2016 at 8:43 PM, William Colen <wi...@gmail.com>
wrote:
> Thank you, Boris. I am new to DeepLearning, so I have no idea the issues we
> would face. I was wondering if we can use Features2Vec instead of Word2Vec,
> does it make any sense?
> The idea was to use DL in low level NLP tasks where we don't have parse
> trees yet.
>
>
> 2016-06-29 6:34 GMT-03:00 Boris Galitsky <bg...@hotmail.com>:
>
> > Hi guys
> >
> > I should mention how we used DeepLearning4J for the OpenNLP.Similarity
> > project at
> >
> > https://github.com/bgalitsky/relevance-based-on-parse-trees
> >
> >
> > The main question is how word2vec models and linguistic information such
> > as part trees complement each other. In a word2vec approach any two words
> > can be compared. The weakness here is that when learning is based on
> > computing a distance between totally unrelated words like 'cat' and 'fly'
> > can be meaningless, uninformative and can corrupt a learning model.
> >
> >
> > In OpenNLP.Similarity component similarity is defined in terms of parse
> > trees. When word2vec is applied on top of parse trees and not as a
> > bag-of-words, we only compute the distance between the words with the
> same
> > semantic role, so the model becomes more accurate.
> >
> >
> > There's a paper on the way which does the assessment of relevance
> > improvent for
> >
> >
> > word2vec (bag-of-words) [traditional] vs word2vec (parse-trees)
> >
> >
> > Regards
> >
> > Boris
> >
> > [https://avatars3.githubusercontent.com/u/1051120?v=3&s=400]<
> > https://github.com/bgalitsky/relevance-based-on-parse-trees>
> >
> > bgalitsky/relevance-based-on-parse-trees<
> > https://github.com/bgalitsky/relevance-based-on-parse-trees>
> > github.com
> > Automatically exported from
> > code.google.com/p/relevance-based-on-parse-trees
> >
> >
> >
> >
> > ________________________________
> > From: Anthony Beylerian <an...@gmail.com>
> > Sent: Wednesday, June 29, 2016 2:13:38 AM
> > To: dev@opennlp.apache.org
> > Subject: Re: DeepLearning4J as a ML for OpenNLP
> >
> > +1 would be willing to help out when possible
> >
>
Re: DeepLearning4J as a ML for OpenNLP
Posted by Boris Galitsky <bg...@hotmail.com>.
Hi Anthony
My interest lies in the question you raised - how to machine learn the structure of a paragraph (not a document yet), given parse trees of individual sentences.
Doc2vec is one direction, but my personal preference is more explicit, structure-based. In my opinion, deep learning family of approaches leverages a huge training dataset they train from, but lacks representing of logical structure of a given document. On the other hand, a discourse tree of a paragraph is a good way to link individual parse trees in a structure to represent a paragraph of text, but lacks extensive knowledge for how n-grams form "meanings" in documents. Therefore I believe doc2vec and learning of discourse trees complement each other.
To systematically learn discourse tree in addition to parse trees, we use tree kernel learning. It forms the space of all sub-trees of trees with abstract labels, and does SVM learning in it. We combine regular parse trees and links between sentences such as rhetoric relations.
The application areas are:
- answering multi-sentence questions
- document-level classification, text style recognition, e.g. for security domain - where documents include the same words but need to be classified by style.
- content generation where maintaining of rhetoric structure is important.
>The generalizations could hurt the classification performance in some
>tasks, but seem to be more useful when the target documents are larger.
Yes, in this case discourse trees are less plausible.
>It could also be possible to chose the "document" to be a single word as
>well, reducing the underlying matrix to an array, does that make sense?
Have not thought about it / have not tried it either
>Therefore, we could also use document based vectors for mid to high-layer
>tasks (doc cat, sentiment, profile etc..). What do you think?
I think for doc classification yes, for sentiments - I am more skeptical, although it is the evaluation area of (Mikolov et al).
Do you have a particular problem in mind?
I can share code on git / papers on the above.
Another way to look at deep learning for NLP : deep learning kind of takes science away from linguistics and makes it more like engineering, I am not sure it is a direction for openNLP?
Regards
Boris
________________________________
From: Anthony Beylerian <an...@gmail.com>
Sent: Wednesday, June 29, 2016 11:24:02 AM
To: dev@opennlp.apache.org
Subject: Re: DeepLearning4J as a ML for OpenNLP
Hi Boris,
Thank you very much for sharing your experience with us!
Is it possible to ask you for more information?
I have only just recently used d4lj with some introductory material,
however I have also felt doc2vec could also be quite useful, although my
understanding of it is still limited.
My current understanding is that doc2vec as an extension of word2vec, can
capture a more generalized context (the document) instead of just focusing
on the context of a single word, in order to provide features useful to
classify that document.
The advantage would be to better capture latent information that exist in
the document (such as the order of words), instead of just averaging word
vectors, or through other approaches on the document level (would love some
feedback on this)
The generalizations could hurt the classification performance in some
tasks, but seem to be more useful when the target documents are larger.
It could also be possible to chose the "document" to be a single word as
well, reducing the underlying matrix to an array, does that make sense?
Therefore, we could also use document based vectors for mid to high-layer
tasks (doc cat, sentiment, profile etc..). What do you think?
It would be fantastic to clarify, I believe that would also motivate more
people to pitch in and better assist with this.
Thanks,
Anthony
Hi William
I have never heard of Features2Vec.
I think for low-level tasks, pre-linguistic tasks such as text
classification where we don't want to build models and have a one-fits-all
solution, Word2Vec works well. I used it in industrial environment for text
classification, some information extraction and content generation tasks.
So I think it should also work for low-level OpenNLP tasks.
Regards
Boris
________________________________
From: William Colen <wi...@gmail.com>
Sent: Wednesday, June 29, 2016 4:43:25 AM
To: dev@opennlp.apache.org
Subject: Re: DeepLearning4J as a ML for OpenNLP
Thank you, Boris. I am new to DeepLearning, so I have no idea the issues we
would face. I was wondering if we can use Features2Vec instead of Word2Vec,
does it make any sense?
The idea was to use DL in low level NLP tasks where we don't have parse
trees yet.
2016-06-29 6:34 GMT-03:00 Boris Galitsky <bg...@hotmail.com>:
> Hi guys
>
> I should mention how we used DeepLearning4J for the OpenNLP.Similarity
> project at
>
> https://github.com/bgalitsky/relevance-based-on-parse-trees
>
>
> The main question is how word2vec models and linguistic information such
> as part trees complement each other. In a word2vec approach any two words
> can be compared. The weakness here is that when learning is based on
> computing a distance between totally unrelated words like 'cat' and 'fly'
> can be meaningless, uninformative and can corrupt a learning model.
>
>
> In OpenNLP.Similarity component similarity is defined in terms of parse
> trees. When word2vec is applied on top of parse trees and not as a
> bag-of-words, we only compute the distance between the words with the same
> semantic role, so the model becomes more accurate.
>
>
> There's a paper on the way which does the assessment of relevance
> improvent for
>
>
> word2vec (bag-of-words) [traditional] vs word2vec (parse-trees)
>
>
> Regards
>
> Boris
>
> [https://avatars3.githubusercontent.com/u/1051120?v=3&s=400]<
> https://github.com/bgalitsky/relevance-based-on-parse-trees>
>
> bgalitsky/relevance-based-on-parse-trees<
> https://github.com/bgalitsky/relevance-based-on-parse-trees>
> github.com
> Automatically exported from
> code.google.com/p/relevance-based-on-parse-trees
>
>
>
>
> ________________________________
> From: Anthony Beylerian <an...@gmail.com>
> Sent: Wednesday, June 29, 2016 2:13:38 AM
> To: dev@opennlp.apache.org
> Subject: Re: DeepLearning4J as a ML for OpenNLP
>
> +1 would be willing to help out when possible
>
Re: DeepLearning4J as a ML for OpenNLP
Posted by Anthony Beylerian <an...@gmail.com>.
Hi Boris,
Thank you very much for sharing your experience with us!
Is it possible to ask you for more information?
I have only just recently used d4lj with some introductory material,
however I have also felt doc2vec could also be quite useful, although my
understanding of it is still limited.
My current understanding is that doc2vec as an extension of word2vec, can
capture a more generalized context (the document) instead of just focusing
on the context of a single word, in order to provide features useful to
classify that document.
The advantage would be to better capture latent information that exist in
the document (such as the order of words), instead of just averaging word
vectors, or through other approaches on the document level (would love some
feedback on this)
The generalizations could hurt the classification performance in some
tasks, but seem to be more useful when the target documents are larger.
It could also be possible to chose the "document" to be a single word as
well, reducing the underlying matrix to an array, does that make sense?
Therefore, we could also use document based vectors for mid to high-layer
tasks (doc cat, sentiment, profile etc..). What do you think?
It would be fantastic to clarify, I believe that would also motivate more
people to pitch in and better assist with this.
Thanks,
Anthony
Hi William
I have never heard of Features2Vec.
I think for low-level tasks, pre-linguistic tasks such as text
classification where we don't want to build models and have a one-fits-all
solution, Word2Vec works well. I used it in industrial environment for text
classification, some information extraction and content generation tasks.
So I think it should also work for low-level OpenNLP tasks.
Regards
Boris
________________________________
From: William Colen <wi...@gmail.com>
Sent: Wednesday, June 29, 2016 4:43:25 AM
To: dev@opennlp.apache.org
Subject: Re: DeepLearning4J as a ML for OpenNLP
Thank you, Boris. I am new to DeepLearning, so I have no idea the issues we
would face. I was wondering if we can use Features2Vec instead of Word2Vec,
does it make any sense?
The idea was to use DL in low level NLP tasks where we don't have parse
trees yet.
2016-06-29 6:34 GMT-03:00 Boris Galitsky <bg...@hotmail.com>:
> Hi guys
>
> I should mention how we used DeepLearning4J for the OpenNLP.Similarity
> project at
>
> https://github.com/bgalitsky/relevance-based-on-parse-trees
>
>
> The main question is how word2vec models and linguistic information such
> as part trees complement each other. In a word2vec approach any two words
> can be compared. The weakness here is that when learning is based on
> computing a distance between totally unrelated words like 'cat' and 'fly'
> can be meaningless, uninformative and can corrupt a learning model.
>
>
> In OpenNLP.Similarity component similarity is defined in terms of parse
> trees. When word2vec is applied on top of parse trees and not as a
> bag-of-words, we only compute the distance between the words with the same
> semantic role, so the model becomes more accurate.
>
>
> There's a paper on the way which does the assessment of relevance
> improvent for
>
>
> word2vec (bag-of-words) [traditional] vs word2vec (parse-trees)
>
>
> Regards
>
> Boris
>
> [https://avatars3.githubusercontent.com/u/1051120?v=3&s=400]<
> https://github.com/bgalitsky/relevance-based-on-parse-trees>
>
> bgalitsky/relevance-based-on-parse-trees<
> https://github.com/bgalitsky/relevance-based-on-parse-trees>
> github.com
> Automatically exported from
> code.google.com/p/relevance-based-on-parse-trees
>
>
>
>
> ________________________________
> From: Anthony Beylerian <an...@gmail.com>
> Sent: Wednesday, June 29, 2016 2:13:38 AM
> To: dev@opennlp.apache.org
> Subject: Re: DeepLearning4J as a ML for OpenNLP
>
> +1 would be willing to help out when possible
>
Re: DeepLearning4J as a ML for OpenNLP
Posted by Boris Galitsky <bg...@hotmail.com>.
Hi William
I have never heard of Features2Vec.
I think for low-level tasks, pre-linguistic tasks such as text classification where we don't want to build models and have a one-fits-all solution, Word2Vec works well. I used it in industrial environment for text classification, some information extraction and content generation tasks. So I think it should also work for low-level OpenNLP tasks.
Regards
Boris
________________________________
From: William Colen <wi...@gmail.com>
Sent: Wednesday, June 29, 2016 4:43:25 AM
To: dev@opennlp.apache.org
Subject: Re: DeepLearning4J as a ML for OpenNLP
Thank you, Boris. I am new to DeepLearning, so I have no idea the issues we
would face. I was wondering if we can use Features2Vec instead of Word2Vec,
does it make any sense?
The idea was to use DL in low level NLP tasks where we don't have parse
trees yet.
2016-06-29 6:34 GMT-03:00 Boris Galitsky <bg...@hotmail.com>:
> Hi guys
>
> I should mention how we used DeepLearning4J for the OpenNLP.Similarity
> project at
>
> https://github.com/bgalitsky/relevance-based-on-parse-trees
>
>
> The main question is how word2vec models and linguistic information such
> as part trees complement each other. In a word2vec approach any two words
> can be compared. The weakness here is that when learning is based on
> computing a distance between totally unrelated words like 'cat' and 'fly'
> can be meaningless, uninformative and can corrupt a learning model.
>
>
> In OpenNLP.Similarity component similarity is defined in terms of parse
> trees. When word2vec is applied on top of parse trees and not as a
> bag-of-words, we only compute the distance between the words with the same
> semantic role, so the model becomes more accurate.
>
>
> There's a paper on the way which does the assessment of relevance
> improvent for
>
>
> word2vec (bag-of-words) [traditional] vs word2vec (parse-trees)
>
>
> Regards
>
> Boris
>
> [https://avatars3.githubusercontent.com/u/1051120?v=3&s=400]<
> https://github.com/bgalitsky/relevance-based-on-parse-trees>
>
> bgalitsky/relevance-based-on-parse-trees<
> https://github.com/bgalitsky/relevance-based-on-parse-trees>
> github.com
> Automatically exported from
> code.google.com/p/relevance-based-on-parse-trees
>
>
>
>
> ________________________________
> From: Anthony Beylerian <an...@gmail.com>
> Sent: Wednesday, June 29, 2016 2:13:38 AM
> To: dev@opennlp.apache.org
> Subject: Re: DeepLearning4J as a ML for OpenNLP
>
> +1 would be willing to help out when possible
>
Re: DeepLearning4J as a ML for OpenNLP
Posted by William Colen <wi...@gmail.com>.
Thank you, Boris. I am new to DeepLearning, so I have no idea the issues we
would face. I was wondering if we can use Features2Vec instead of Word2Vec,
does it make any sense?
The idea was to use DL in low level NLP tasks where we don't have parse
trees yet.
2016-06-29 6:34 GMT-03:00 Boris Galitsky <bg...@hotmail.com>:
> Hi guys
>
> I should mention how we used DeepLearning4J for the OpenNLP.Similarity
> project at
>
> https://github.com/bgalitsky/relevance-based-on-parse-trees
>
>
> The main question is how word2vec models and linguistic information such
> as part trees complement each other. In a word2vec approach any two words
> can be compared. The weakness here is that when learning is based on
> computing a distance between totally unrelated words like 'cat' and 'fly'
> can be meaningless, uninformative and can corrupt a learning model.
>
>
> In OpenNLP.Similarity component similarity is defined in terms of parse
> trees. When word2vec is applied on top of parse trees and not as a
> bag-of-words, we only compute the distance between the words with the same
> semantic role, so the model becomes more accurate.
>
>
> There's a paper on the way which does the assessment of relevance
> improvent for
>
>
> word2vec (bag-of-words) [traditional] vs word2vec (parse-trees)
>
>
> Regards
>
> Boris
>
> [https://avatars3.githubusercontent.com/u/1051120?v=3&s=400]<
> https://github.com/bgalitsky/relevance-based-on-parse-trees>
>
> bgalitsky/relevance-based-on-parse-trees<
> https://github.com/bgalitsky/relevance-based-on-parse-trees>
> github.com
> Automatically exported from
> code.google.com/p/relevance-based-on-parse-trees
>
>
>
>
> ________________________________
> From: Anthony Beylerian <an...@gmail.com>
> Sent: Wednesday, June 29, 2016 2:13:38 AM
> To: dev@opennlp.apache.org
> Subject: Re: DeepLearning4J as a ML for OpenNLP
>
> +1 would be willing to help out when possible
>
Re: DeepLearning4J as a ML for OpenNLP
Posted by Boris Galitsky <bg...@hotmail.com>.
Hi guys
I should mention how we used DeepLearning4J for the OpenNLP.Similarity project at
https://github.com/bgalitsky/relevance-based-on-parse-trees
The main question is how word2vec models and linguistic information such as part trees complement each other. In a word2vec approach any two words can be compared. The weakness here is that when learning is based on computing a distance between totally unrelated words like 'cat' and 'fly' can be meaningless, uninformative and can corrupt a learning model.
In OpenNLP.Similarity component similarity is defined in terms of parse trees. When word2vec is applied on top of parse trees and not as a bag-of-words, we only compute the distance between the words with the same semantic role, so the model becomes more accurate.
There's a paper on the way which does the assessment of relevance improvent for
word2vec (bag-of-words) [traditional] vs word2vec (parse-trees)
Regards
Boris
[https://avatars3.githubusercontent.com/u/1051120?v=3&s=400]<https://github.com/bgalitsky/relevance-based-on-parse-trees>
bgalitsky/relevance-based-on-parse-trees<https://github.com/bgalitsky/relevance-based-on-parse-trees>
github.com
Automatically exported from code.google.com/p/relevance-based-on-parse-trees
________________________________
From: Anthony Beylerian <an...@gmail.com>
Sent: Wednesday, June 29, 2016 2:13:38 AM
To: dev@opennlp.apache.org
Subject: Re: DeepLearning4J as a ML for OpenNLP
+1 would be willing to help out when possible
Re: DeepLearning4J as a ML for OpenNLP
Posted by Anthony Beylerian <an...@gmail.com>.
+1 would be willing to help out when possible
Re: DeepLearning4J as a ML for OpenNLP
Posted by "Mattmann, Chris A (3980)" <ch...@jpl.nasa.gov>.
Thanks check out http://github.com/SciSpark/scispark
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Chief Architect
Instrument Software and Science Data Systems Section (398)
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 168-519, Mailstop: 168-527
Email: chris.a.mattmann@nasa.gov
WWW: http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Director, Information Retrieval and Data Science Group (IRDS)
Adjunct Associate Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
WWW: http://irds.usc.edu/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
On 6/28/16, 2:43 PM, "William Colen" <wi...@gmail.com> wrote:
>Thank you for pointing, Prof. Chris. Can you please point me the exact
>project at http://scispark.jpl.nasa.gov/ I should look at? It is huge.
>
>Thank you again.
>William
>
>William Colen
>
>2016-06-28 18:26 GMT-03:00 Mattmann, Chris A (3980) <
>chris.a.mattmann@jpl.nasa.gov>:
>
>> Yep I think so - you may also look at SciSpark
>> http://scispark.jpl.nasa.gov
>> where we are using DL4J/ND4J and Breeze interchangeably here.
>>
>> Cheers,
>> Chris
>>
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> Chris Mattmann, Ph.D.
>> Chief Architect
>> Instrument Software and Science Data Systems Section (398)
>> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>> Office: 168-519, Mailstop: 168-527
>> Email: chris.a.mattmann@nasa.gov
>> WWW: http://sunset.usc.edu/~mattmann/
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> Director, Information Retrieval and Data Science Group (IRDS)
>> Adjunct Associate Professor, Computer Science Department
>> University of Southern California, Los Angeles, CA 90089 USA
>> WWW: http://irds.usc.edu/
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> On 6/28/16, 2:23 PM, "William Colen" <wi...@gmail.com> wrote:
>>
>> >Hi,
>> >
>> >Do you think it would be possible to implement a ML based on DL4J?
>> >
>> >http://deeplearning4j.org/
>> >
>> >Thank you
>> >William
>>
Re: DeepLearning4J as a ML for OpenNLP
Posted by William Colen <wi...@gmail.com>.
Thank you for pointing, Prof. Chris. Can you please point me the exact
project at http://scispark.jpl.nasa.gov/ I should look at? It is huge.
Thank you again.
William
William Colen
2016-06-28 18:26 GMT-03:00 Mattmann, Chris A (3980) <
chris.a.mattmann@jpl.nasa.gov>:
> Yep I think so - you may also look at SciSpark
> http://scispark.jpl.nasa.gov
> where we are using DL4J/ND4J and Breeze interchangeably here.
>
> Cheers,
> Chris
>
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Chris Mattmann, Ph.D.
> Chief Architect
> Instrument Software and Science Data Systems Section (398)
> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
> Office: 168-519, Mailstop: 168-527
> Email: chris.a.mattmann@nasa.gov
> WWW: http://sunset.usc.edu/~mattmann/
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Director, Information Retrieval and Data Science Group (IRDS)
> Adjunct Associate Professor, Computer Science Department
> University of Southern California, Los Angeles, CA 90089 USA
> WWW: http://irds.usc.edu/
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>
>
>
>
>
>
>
>
>
>
> On 6/28/16, 2:23 PM, "William Colen" <wi...@gmail.com> wrote:
>
> >Hi,
> >
> >Do you think it would be possible to implement a ML based on DL4J?
> >
> >http://deeplearning4j.org/
> >
> >Thank you
> >William
>
Re: DeepLearning4J as a ML for OpenNLP
Posted by Tommaso Teofili <to...@gmail.com>.
I had briefly looked into it a while ago, would be nice to collaborate
there.
Tommaso
Il giorno mar 28 giu 2016 alle 23:26 Mattmann, Chris A (3980) <
chris.a.mattmann@jpl.nasa.gov> ha scritto:
> Yep I think so - you may also look at SciSpark
> http://scispark.jpl.nasa.gov
> where we are using DL4J/ND4J and Breeze interchangeably here.
>
> Cheers,
> Chris
>
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Chris Mattmann, Ph.D.
> Chief Architect
> Instrument Software and Science Data Systems Section (398)
> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
> Office: 168-519, Mailstop: 168-527
> Email: chris.a.mattmann@nasa.gov
> WWW: http://sunset.usc.edu/~mattmann/
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Director, Information Retrieval and Data Science Group (IRDS)
> Adjunct Associate Professor, Computer Science Department
> University of Southern California, Los Angeles, CA 90089 USA
> WWW: http://irds.usc.edu/
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>
>
>
>
>
>
>
>
>
>
> On 6/28/16, 2:23 PM, "William Colen" <wi...@gmail.com> wrote:
>
> >Hi,
> >
> >Do you think it would be possible to implement a ML based on DL4J?
> >
> >http://deeplearning4j.org/
> >
> >Thank you
> >William
>
Re: DeepLearning4J as a ML for OpenNLP
Posted by "Mattmann, Chris A (3980)" <ch...@jpl.nasa.gov>.
Yep I think so - you may also look at SciSpark http://scispark.jpl.nasa.gov
where we are using DL4J/ND4J and Breeze interchangeably here.
Cheers,
Chris
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Chief Architect
Instrument Software and Science Data Systems Section (398)
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 168-519, Mailstop: 168-527
Email: chris.a.mattmann@nasa.gov
WWW: http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Director, Information Retrieval and Data Science Group (IRDS)
Adjunct Associate Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
WWW: http://irds.usc.edu/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
On 6/28/16, 2:23 PM, "William Colen" <wi...@gmail.com> wrote:
>Hi,
>
>Do you think it would be possible to implement a ML based on DL4J?
>
>http://deeplearning4j.org/
>
>Thank you
>William
Re: DeepLearning4J as a ML for OpenNLP
Posted by Anthony Beylerian <an...@gmail.com>.
@William
I think what you meant previously by feature2vec would be to deep-learn
with any discrete state, not just with words, am I right?
Extra side-information could possibly help improve some results, but this
would make things overly complicated in my opinion.
@Boris,
Thank you very much, I see what you mean, yes they do complement each other
in that sense.
>> Do you have a particular problem in mind?
No particular problem, but it was very nice of you to clarify what you
meant by parse tree based approaches.
On the other hand, I am not currently aware of any studies comparing both
doc2vec and discourse trees for particular problems.
But it will be useful to know what to use in each case, since we are
considering deep learning support, the application areas you mentioned are
also quite interesting.
Otherwise, as you may already know there are a couple of projects currently
in progress for the toolkit: sentiment analysis and author profiling.
I think it would be good to use deep learning with these tools (as well as
others).
>> I can share code on git / papers on the above.
Yes I would love to check those out.
>> I am not sure it is a direction for openNLP?
I think Jörn answered that, it would be great to have the library offer
even more tools, using dl4j would be a nice to have (since it also offers
different neural net classifiers).
@Jörn
We could try looking into some existing publications about training models
(if any), unless someone can point us in the right direction, that would
really help.
Otherwise, although we can use other classifiers, dl4j team also has a page
for some recommended neural nets to use for classification in the next step:
http://deeplearning4j.org/neuralnetworktable
Best,
Anthony
On Fri, Jul 1, 2016 at 10:12 PM, Joern Kottmann <ko...@gmail.com> wrote:
> Hello,
>
> the people from deeplearning4j are rather nice and I discussed with them
> for a while how
> it can be used for OpenNLP. The state back then was that they don't
> properly support the
> sparse feature vectors we use in OpenNLP today. Instead we would need to
> use word embeddings.
> In the end I never tried it out but I think it might not be very difficult
> to get everything wired together,
> the most difficult part is probably to find a deep learning model setup
> which works well.
>
> Jörn
>
> On Tue, Jun 28, 2016 at 11:23 PM, William Colen <wi...@gmail.com>
> wrote:
>
> > Hi,
> >
> > Do you think it would be possible to implement a ML based on DL4J?
> >
> > http://deeplearning4j.org/
> >
> > Thank you
> > William
> >
>
Re: DeepLearning4J as a ML for OpenNLP
Posted by Joern Kottmann <ko...@gmail.com>.
Hello,
the people from deeplearning4j are rather nice and I discussed with them
for a while how
it can be used for OpenNLP. The state back then was that they don't
properly support the
sparse feature vectors we use in OpenNLP today. Instead we would need to
use word embeddings.
In the end I never tried it out but I think it might not be very difficult
to get everything wired together,
the most difficult part is probably to find a deep learning model setup
which works well.
Jörn
On Tue, Jun 28, 2016 at 11:23 PM, William Colen <wi...@gmail.com>
wrote:
> Hi,
>
> Do you think it would be possible to implement a ML based on DL4J?
>
> http://deeplearning4j.org/
>
> Thank you
> William
>
Re: DeepLearning4J as a ML for OpenNLP
Posted by William Colen <wi...@gmail.com>.
Suneel,
I mean an implementation so we can use DL4J to train the OpenNLP models,
just like we already do in opennlp.tools.ml package with Maxent,
Perceptron, NayveBayes. I think it was Jörn who also did a few others that
are in the SandBox: Mallet and Mahout.
Thank you!
William
2016-06-28 18:27 GMT-03:00 Suneel Marthi <su...@yahoo.com.invalid>:
> Are u looking at using ND4J (from Deeplearning4j project) as the Math
> backend for ML work? If so, yes.
>
>
> From: William Colen <wi...@gmail.com>
> To: "dev@opennlp.apache.org" <de...@opennlp.apache.org>
> Sent: Tuesday, June 28, 2016 5:23 PM
> Subject: DeepLearning4J as a ML for OpenNLP
>
> Hi,
>
> Do you think it would be possible to implement a ML based on DL4J?
>
> http://deeplearning4j.org/
>
> Thank you
> William
>
>
>
>
Re: DeepLearning4J as a ML for OpenNLP
Posted by Suneel Marthi <su...@yahoo.com.INVALID>.
Are u looking at using ND4J (from Deeplearning4j project) as the Math backend for ML work? If so, yes.
From: William Colen <wi...@gmail.com>
To: "dev@opennlp.apache.org" <de...@opennlp.apache.org>
Sent: Tuesday, June 28, 2016 5:23 PM
Subject: DeepLearning4J as a ML for OpenNLP
Hi,
Do you think it would be possible to implement a ML based on DL4J?
http://deeplearning4j.org/
Thank you
William