You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Puneet Pawaia <pu...@gmail.com> on 2016/07/07 17:09:54 UTC
Integrating Stanford NLP or any other NLP for Natural Language Query
Hi
I am currently using Solr 5.5.x to test but can upgrade to Solr 6.x if
required.
I am working on a POC for natural language query using Solr. Should I use
the Stanford libraries or are there any other libraries having integration
with Solr already available.
Any direction in how to do this would be most appreciated. How should I
process the query to give relevant results.
Regards
Puneet
Re: Integrating Stanford NLP or any other NLP for Natural Language Query
Posted by Puneet Pawaia <pu...@gmail.com>.
Thanks for the link.
I'll take a look at it later in the day once I am at office.
Puneet
On 8 Jul 2016 08:19, "Yangrui Guo" <gu...@gmail.com> wrote:
https://github.com/guoyangrui/squery
It's not well documented yet but the idea was simple. Users should first
format their database tables into triples by creating view, then Solr and
Stanford NLP handles the data retrieval part. I hope someone could continue
contribute to its developement.
Yangrui
On Thursday, July 7, 2016, John Blythe <jo...@curvolabs.com> wrote:
> can you share a link, i'd be interested in checking it out.
>
> thanks-
>
> --
> *John Blythe*
> Product Manager & Lead Developer
>
> 251.605.3071 | john@curvolabs.com <javascript:;>
> www.curvolabs.com
>
> 58 Adams Ave
> Evansville, IN 47713
>
> On Thu, Jul 7, 2016 at 4:32 PM, Yangrui Guo <guoyangrui@gmail.com
> <javascript:;>> wrote:
>
> > What is your NLP search like? I have a NLP solution for Solr and just
> open
> > sourced it. Not sure if it fits your need
> >
> > Yangrui
> >
> > On Thursday, July 7, 2016, Puneet Pawaia <puneet.pawaia@gmail.com
> <javascript:;>> wrote:
> >
> > > Hi
> > >
> > > I am currently using Solr 5.5.x to test but can upgrade to Solr 6.x if
> > > required.
> > > I am working on a POC for natural language query using Solr. Should I
> use
> > > the Stanford libraries or are there any other libraries having
> > integration
> > > with Solr already available.
> > > Any direction in how to do this would be most appreciated. How should
I
> > > process the query to give relevant results.
> > >
> > > Regards
> > > Puneet
> > >
> >
>
Re: Integrating Stanford NLP or any other NLP for Natural Language Query
Posted by Yangrui Guo <gu...@gmail.com>.
https://github.com/guoyangrui/squery
It's not well documented yet but the idea was simple. Users should first
format their database tables into triples by creating view, then Solr and
Stanford NLP handles the data retrieval part. I hope someone could continue
contribute to its developement.
Yangrui
On Thursday, July 7, 2016, John Blythe <jo...@curvolabs.com> wrote:
> can you share a link, i'd be interested in checking it out.
>
> thanks-
>
> --
> *John Blythe*
> Product Manager & Lead Developer
>
> 251.605.3071 | john@curvolabs.com <javascript:;>
> www.curvolabs.com
>
> 58 Adams Ave
> Evansville, IN 47713
>
> On Thu, Jul 7, 2016 at 4:32 PM, Yangrui Guo <guoyangrui@gmail.com
> <javascript:;>> wrote:
>
> > What is your NLP search like? I have a NLP solution for Solr and just
> open
> > sourced it. Not sure if it fits your need
> >
> > Yangrui
> >
> > On Thursday, July 7, 2016, Puneet Pawaia <puneet.pawaia@gmail.com
> <javascript:;>> wrote:
> >
> > > Hi
> > >
> > > I am currently using Solr 5.5.x to test but can upgrade to Solr 6.x if
> > > required.
> > > I am working on a POC for natural language query using Solr. Should I
> use
> > > the Stanford libraries or are there any other libraries having
> > integration
> > > with Solr already available.
> > > Any direction in how to do this would be most appreciated. How should I
> > > process the query to give relevant results.
> > >
> > > Regards
> > > Puneet
> > >
> >
>
Re: Integrating Stanford NLP or any other NLP for Natural Language Query
Posted by John Blythe <jo...@curvolabs.com>.
can you share a link, i'd be interested in checking it out.
thanks-
--
*John Blythe*
Product Manager & Lead Developer
251.605.3071 | john@curvolabs.com
www.curvolabs.com
58 Adams Ave
Evansville, IN 47713
On Thu, Jul 7, 2016 at 4:32 PM, Yangrui Guo <gu...@gmail.com> wrote:
> What is your NLP search like? I have a NLP solution for Solr and just open
> sourced it. Not sure if it fits your need
>
> Yangrui
>
> On Thursday, July 7, 2016, Puneet Pawaia <pu...@gmail.com> wrote:
>
> > Hi
> >
> > I am currently using Solr 5.5.x to test but can upgrade to Solr 6.x if
> > required.
> > I am working on a POC for natural language query using Solr. Should I use
> > the Stanford libraries or are there any other libraries having
> integration
> > with Solr already available.
> > Any direction in how to do this would be most appreciated. How should I
> > process the query to give relevant results.
> >
> > Regards
> > Puneet
> >
>
Re: Integrating Stanford NLP or any other NLP for Natural Language Query
Posted by Yangrui Guo <gu...@gmail.com>.
My solution lets users retrieve data entities using queries like "find me a
job that only requires a high school degree" and "I want a car from
American with alloy wheels". It can also be expanded to perform other
database queries, like date time or price range searches. I use Stanford
NLP to identify the main entity and its related attributes in a user
sentence.
Yangrui
On Thursday, July 7, 2016, Puneet Pawaia <pu...@gmail.com> wrote:
> Hi Yangrui
> I would like users to be able to write queries in natural language rather
> than keyword based search.
> A link to your solution would be worth looking at.
> Regards
> Puneet
>
> On 8 Jul 2016 03:02, "Yangrui Guo" <guoyangrui@gmail.com <javascript:;>>
> wrote:
>
> What is your NLP search like? I have a NLP solution for Solr and just open
> sourced it. Not sure if it fits your need
>
> Yangrui
>
> On Thursday, July 7, 2016, Puneet Pawaia <puneet.pawaia@gmail.com
> <javascript:;>> wrote:
>
> > Hi
> >
> > I am currently using Solr 5.5.x to test but can upgrade to Solr 6.x if
> > required.
> > I am working on a POC for natural language query using Solr. Should I use
> > the Stanford libraries or are there any other libraries having
> integration
> > with Solr already available.
> > Any direction in how to do this would be most appreciated. How should I
> > process the query to give relevant results.
> >
> > Regards
> > Puneet
> >
>
Re: Integrating Stanford NLP or any other NLP for Natural Language Query
Posted by Puneet Pawaia <pu...@gmail.com>.
Hi Yangrui
I would like users to be able to write queries in natural language rather
than keyword based search.
A link to your solution would be worth looking at.
Regards
Puneet
On 8 Jul 2016 03:02, "Yangrui Guo" <gu...@gmail.com> wrote:
What is your NLP search like? I have a NLP solution for Solr and just open
sourced it. Not sure if it fits your need
Yangrui
On Thursday, July 7, 2016, Puneet Pawaia <pu...@gmail.com> wrote:
> Hi
>
> I am currently using Solr 5.5.x to test but can upgrade to Solr 6.x if
> required.
> I am working on a POC for natural language query using Solr. Should I use
> the Stanford libraries or are there any other libraries having integration
> with Solr already available.
> Any direction in how to do this would be most appreciated. How should I
> process the query to give relevant results.
>
> Regards
> Puneet
>
Re: Integrating Stanford NLP or any other NLP for Natural Language Query
Posted by Yangrui Guo <gu...@gmail.com>.
What is your NLP search like? I have a NLP solution for Solr and just open
sourced it. Not sure if it fits your need
Yangrui
On Thursday, July 7, 2016, Puneet Pawaia <pu...@gmail.com> wrote:
> Hi
>
> I am currently using Solr 5.5.x to test but can upgrade to Solr 6.x if
> required.
> I am working on a POC for natural language query using Solr. Should I use
> the Stanford libraries or are there any other libraries having integration
> with Solr already available.
> Any direction in how to do this would be most appreciated. How should I
> process the query to give relevant results.
>
> Regards
> Puneet
>
Re: Integrating Stanford NLP or any other NLP for Natural Language Query
Posted by Joel Bernstein <jo...@gmail.com>.
You may want to take a look at NLP4J. There is no integration yet with
Solr, but it seems like it would be a good fit.
Joel Bernstein
http://joelsolr.blogspot.com/
On Thu, Jul 7, 2016 at 1:09 PM, Puneet Pawaia <pu...@gmail.com>
wrote:
> Hi
>
> I am currently using Solr 5.5.x to test but can upgrade to Solr 6.x if
> required.
> I am working on a POC for natural language query using Solr. Should I use
> the Stanford libraries or are there any other libraries having integration
> with Solr already available.
> Any direction in how to do this would be most appreciated. How should I
> process the query to give relevant results.
>
> Regards
> Puneet
>
Re: Integrating Stanford NLP or any other NLP for Natural Language Query
Posted by Yangrui Guo <gu...@gmail.com>.
Hi my solution uses multi value text fields for storing data objects. It
works best with relational search using natural language. For example,
"car", "automobile", "vehicle" may denote the same class, but they are not
equivalent under certain contexts. Multiple value attributes could help the
search engine better interpret different expression by user referring to
the same concept.
On Sunday, July 10, 2016, Puneet Pawaia <pu...@gmail.com> wrote:
> Hi Yangrui
> We are testing the Rank and Retrieve as well as the NLP interface. However
> this is being done by another team and so I would not be able to comment
> further on it.
> I would like to know what kind of Solr field you are using for storing the
> output from your classes. And also what function output you are putting
> into that field.
> Thanks
> Puneet
>
> On 10 Jul 2016 00:17, "Yangrui Guo" <guoyangrui@gmail.com <javascript:;>>
> wrote:
>
> > Hi Puneet,
> >
> > I only use Watson's text to speech as user interface, because a lot of
> > people think NLP is the same as voice recognition. If you don't need
> voice
> > recognition you could remove Watson from it. Stanford has better
> dependency
> > parsing and can be used offline. However it seems you are using Watson's
> > retrieve and rank API, which is based on Solr, am I correct?
> >
> > Yangrui
> >
> > On Saturday, July 9, 2016, Puneet Pawaia <puneet.pawaia@gmail.com
> <javascript:;>> wrote:
> >
> > > Hi Yangrui,
> > >
> > > I have been looking at your code for squery.
> > > Unfortunately, I am not very conversant with SolrJ. I seem to be
> missing
> > > how and what data is added to the Solr index.
> > > Also, I see some references to IBM Watson in your code. Are you using
> IBM
> > > Watson? If yes, then why use the Stanford NLP if you can use the Watson
> > > NLP?
> > >
> > > Regards
> > > Puneet
> > >
> > >
> > > On Sat, Jul 9, 2016 at 11:37 AM, Puneet Pawaia <
> puneet.pawaia@gmail.com <javascript:;>
> > > <javascript:;>>
> > > wrote:
> > >
> > > > Hi Alessandro
> > > >
> > > > I am looking at being able to answer questions like "Can a
> non-compete
> > > > clause in an employment agreement be enforced after the expiry of the
> > > > agreement?"
> > > > We are doing some testing with IBM Watson and with a sample test
> data,
> > we
> > > > are able to get relevant replies to the above question. Since IBM
> > Watson
> > > > uses Solr at its backend, I was wondering if we can get the same
> > working
> > > at
> > > > the Solr level without having to use Watson.
> > > >
> > > > Regards
> > > > Puneet
> > > >
> > > > On Sat, Jul 9, 2016 at 11:34 AM, Puneet Pawaia <
> > puneet.pawaia@gmail.com <javascript:;>
> > > <javascript:;>>
> > > > wrote:
> > > >
> > > >> Hi Alessandro
> > > >>
> > > >> I am looking at being able to answer questions like "Can a
> non-compete
> > > >> clause in an employment agreement be enforced after the expiry of
> the
> > > >> agreement?"
> > > >>
> > > >> On Sat, Jul 9, 2016 at 4:34 AM, Alessandro Benedetti <
> > > >> abenedetti@apache.org <javascript:;> <javascript:;>> wrote:
> > > >>
> > > >>> Hi Puneet,
> > > >>> your requirement :
> > > >>> "I would like users to be able to write queries in natural language
> > > >>> rather
> > > >>> than keyword based search."
> > > >>>
> > > >>> Is really really vague :(
> > > >>> Can you try to help us with some specific example, starting of
> course
> > > >>> from
> > > >>> the simplest use cases you have initially in mind ?
> > > >>>
> > > >>> Moving from keyword based search to natural language is a really
> > > complex
> > > >>> task.
> > > >>> Proceeding step by step can help you.
> > > >>>
> > > >>> Do you want for example to set up a Q&A basic system ?
> > > >>> In that case you should take care of query rewriting.
> > > >>> You need basically to identify your base requirement and then
> build a
> > > >>> specific parser for that.
> > > >>> You can use triple stores and knowledge bases to enrich both your
> > query
> > > >>> and
> > > >>> your index, but let's start from the basis, what is your simplest
> > > >>> requirement ?
> > > >>>
> > > >>> On Fri, Jul 8, 2016 at 1:56 PM, Jay Urbain <jay.urbain@gmail.com
> <javascript:;>
> > > <javascript:;>> wrote:
> > > >>>
> > > >>> > I've added multivalued fields within my SOLR schema for indexing
> > > >>> entities
> > > >>> > extracted using NLP methods applied to the text I'm indexing,
> along
> > > >>> with
> > > >>> > fields for other discrete data extracted from relational
> databases.
> > > >>> >
> > > >>> > A Java application reads data out of multiple relational
> databases,
> > > >>> uses
> > > >>> > NLP on the text and indexes each document (de-normalized) using
> > > SOLRJ.
> > > >>> >
> > > >>> > I initially tried doing this with content handlers, but found it
> > much
> > > >>> > easier to just write a Java application.
> > > >>> >
> > > >>> > SOLRJ Java API reference:
> > > >>> > https://cwiki.apache.org/confluence/display/solr/Using+SolrJ
> > > >>> >
> > > >>> > Stanford NLP:
> > > >>> > http://stanfordnlp.github.io/CoreNLP/
> > > >>> >
> > > >>> > Best,
> > > >>> > Jay
> > > >>> >
> > > >>> >
> > > >>> > On Thu, Jul 7, 2016 at 9:52 PM, Puneet Pawaia <
> > > puneet.pawaia@gmail.com <javascript:;> <javascript:;>
> > > >>> >
> > > >>> > wrote:
> > > >>> >
> > > >>> > > Hi Jay
> > > >>> > > Any place I can learn more on this method of integration?
> > > >>> > > Thanks
> > > >>> > > Puneet
> > > >>> > >
> > > >>> > > On 8 Jul 2016 02:58, "Jay Urbain" <jay.urbain@gmail.com
> <javascript:;>
> > > <javascript:;>> wrote:
> > > >>> > >
> > > >>> > > > I use Stanford NLP and cTakes (based on OpenNLP) while
> indexing
> > > >>> with a
> > > >>> > > > SOLRJ application.
> > > >>> > > >
> > > >>> > > > Best,
> > > >>> > > > Jay
> > > >>> > > >
> > > >>> > > > On Thu, Jul 7, 2016 at 12:09 PM, Puneet Pawaia <
> > > >>> > puneet.pawaia@gmail.com <javascript:;> <javascript:;>>
> > > >>> > > > wrote:
> > > >>> > > >
> > > >>> > > > > Hi
> > > >>> > > > >
> > > >>> > > > > I am currently using Solr 5.5.x to test but can upgrade to
> > Solr
> > > >>> 6.x
> > > >>> > if
> > > >>> > > > > required.
> > > >>> > > > > I am working on a POC for natural language query using
> Solr.
> > > >>> Should I
> > > >>> > > use
> > > >>> > > > > the Stanford libraries or are there any other libraries
> > having
> > > >>> > > > integration
> > > >>> > > > > with Solr already available.
> > > >>> > > > > Any direction in how to do this would be most appreciated.
> > How
> > > >>> > should I
> > > >>> > > > > process the query to give relevant results.
> > > >>> > > > >
> > > >>> > > > > Regards
> > > >>> > > > > Puneet
> > > >>> > > > >
> > > >>> > > >
> > > >>> > >
> > > >>> >
> > > >>>
> > > >>>
> > > >>>
> > > >>> --
> > > >>> --------------------------
> > > >>>
> > > >>> Benedetti Alessandro
> > > >>> Visiting card : http://about.me/alessandro_benedetti
> > > >>>
> > > >>> "Tyger, tyger burning bright
> > > >>> In the forests of the night,
> > > >>> What immortal hand or eye
> > > >>> Could frame thy fearful symmetry?"
> > > >>>
> > > >>> William Blake - Songs of Experience -1794 England
> > > >>>
> > > >>
> > > >>
> > > >
> > >
> >
>
Re: Integrating Stanford NLP or any other NLP for Natural Language Query
Posted by Puneet Pawaia <pu...@gmail.com>.
Hi Yangrui
We are testing the Rank and Retrieve as well as the NLP interface. However
this is being done by another team and so I would not be able to comment
further on it.
I would like to know what kind of Solr field you are using for storing the
output from your classes. And also what function output you are putting
into that field.
Thanks
Puneet
On 10 Jul 2016 00:17, "Yangrui Guo" <gu...@gmail.com> wrote:
> Hi Puneet,
>
> I only use Watson's text to speech as user interface, because a lot of
> people think NLP is the same as voice recognition. If you don't need voice
> recognition you could remove Watson from it. Stanford has better dependency
> parsing and can be used offline. However it seems you are using Watson's
> retrieve and rank API, which is based on Solr, am I correct?
>
> Yangrui
>
> On Saturday, July 9, 2016, Puneet Pawaia <pu...@gmail.com> wrote:
>
> > Hi Yangrui,
> >
> > I have been looking at your code for squery.
> > Unfortunately, I am not very conversant with SolrJ. I seem to be missing
> > how and what data is added to the Solr index.
> > Also, I see some references to IBM Watson in your code. Are you using IBM
> > Watson? If yes, then why use the Stanford NLP if you can use the Watson
> > NLP?
> >
> > Regards
> > Puneet
> >
> >
> > On Sat, Jul 9, 2016 at 11:37 AM, Puneet Pawaia <puneet.pawaia@gmail.com
> > <javascript:;>>
> > wrote:
> >
> > > Hi Alessandro
> > >
> > > I am looking at being able to answer questions like "Can a non-compete
> > > clause in an employment agreement be enforced after the expiry of the
> > > agreement?"
> > > We are doing some testing with IBM Watson and with a sample test data,
> we
> > > are able to get relevant replies to the above question. Since IBM
> Watson
> > > uses Solr at its backend, I was wondering if we can get the same
> working
> > at
> > > the Solr level without having to use Watson.
> > >
> > > Regards
> > > Puneet
> > >
> > > On Sat, Jul 9, 2016 at 11:34 AM, Puneet Pawaia <
> puneet.pawaia@gmail.com
> > <javascript:;>>
> > > wrote:
> > >
> > >> Hi Alessandro
> > >>
> > >> I am looking at being able to answer questions like "Can a non-compete
> > >> clause in an employment agreement be enforced after the expiry of the
> > >> agreement?"
> > >>
> > >> On Sat, Jul 9, 2016 at 4:34 AM, Alessandro Benedetti <
> > >> abenedetti@apache.org <javascript:;>> wrote:
> > >>
> > >>> Hi Puneet,
> > >>> your requirement :
> > >>> "I would like users to be able to write queries in natural language
> > >>> rather
> > >>> than keyword based search."
> > >>>
> > >>> Is really really vague :(
> > >>> Can you try to help us with some specific example, starting of course
> > >>> from
> > >>> the simplest use cases you have initially in mind ?
> > >>>
> > >>> Moving from keyword based search to natural language is a really
> > complex
> > >>> task.
> > >>> Proceeding step by step can help you.
> > >>>
> > >>> Do you want for example to set up a Q&A basic system ?
> > >>> In that case you should take care of query rewriting.
> > >>> You need basically to identify your base requirement and then build a
> > >>> specific parser for that.
> > >>> You can use triple stores and knowledge bases to enrich both your
> query
> > >>> and
> > >>> your index, but let's start from the basis, what is your simplest
> > >>> requirement ?
> > >>>
> > >>> On Fri, Jul 8, 2016 at 1:56 PM, Jay Urbain <jay.urbain@gmail.com
> > <javascript:;>> wrote:
> > >>>
> > >>> > I've added multivalued fields within my SOLR schema for indexing
> > >>> entities
> > >>> > extracted using NLP methods applied to the text I'm indexing, along
> > >>> with
> > >>> > fields for other discrete data extracted from relational databases.
> > >>> >
> > >>> > A Java application reads data out of multiple relational databases,
> > >>> uses
> > >>> > NLP on the text and indexes each document (de-normalized) using
> > SOLRJ.
> > >>> >
> > >>> > I initially tried doing this with content handlers, but found it
> much
> > >>> > easier to just write a Java application.
> > >>> >
> > >>> > SOLRJ Java API reference:
> > >>> > https://cwiki.apache.org/confluence/display/solr/Using+SolrJ
> > >>> >
> > >>> > Stanford NLP:
> > >>> > http://stanfordnlp.github.io/CoreNLP/
> > >>> >
> > >>> > Best,
> > >>> > Jay
> > >>> >
> > >>> >
> > >>> > On Thu, Jul 7, 2016 at 9:52 PM, Puneet Pawaia <
> > puneet.pawaia@gmail.com <javascript:;>
> > >>> >
> > >>> > wrote:
> > >>> >
> > >>> > > Hi Jay
> > >>> > > Any place I can learn more on this method of integration?
> > >>> > > Thanks
> > >>> > > Puneet
> > >>> > >
> > >>> > > On 8 Jul 2016 02:58, "Jay Urbain" <jay.urbain@gmail.com
> > <javascript:;>> wrote:
> > >>> > >
> > >>> > > > I use Stanford NLP and cTakes (based on OpenNLP) while indexing
> > >>> with a
> > >>> > > > SOLRJ application.
> > >>> > > >
> > >>> > > > Best,
> > >>> > > > Jay
> > >>> > > >
> > >>> > > > On Thu, Jul 7, 2016 at 12:09 PM, Puneet Pawaia <
> > >>> > puneet.pawaia@gmail.com <javascript:;>>
> > >>> > > > wrote:
> > >>> > > >
> > >>> > > > > Hi
> > >>> > > > >
> > >>> > > > > I am currently using Solr 5.5.x to test but can upgrade to
> Solr
> > >>> 6.x
> > >>> > if
> > >>> > > > > required.
> > >>> > > > > I am working on a POC for natural language query using Solr.
> > >>> Should I
> > >>> > > use
> > >>> > > > > the Stanford libraries or are there any other libraries
> having
> > >>> > > > integration
> > >>> > > > > with Solr already available.
> > >>> > > > > Any direction in how to do this would be most appreciated.
> How
> > >>> > should I
> > >>> > > > > process the query to give relevant results.
> > >>> > > > >
> > >>> > > > > Regards
> > >>> > > > > Puneet
> > >>> > > > >
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> > >>>
> > >>>
> > >>> --
> > >>> --------------------------
> > >>>
> > >>> Benedetti Alessandro
> > >>> Visiting card : http://about.me/alessandro_benedetti
> > >>>
> > >>> "Tyger, tyger burning bright
> > >>> In the forests of the night,
> > >>> What immortal hand or eye
> > >>> Could frame thy fearful symmetry?"
> > >>>
> > >>> William Blake - Songs of Experience -1794 England
> > >>>
> > >>
> > >>
> > >
> >
>
Re: Integrating Stanford NLP or any other NLP for Natural Language Query
Posted by Yangrui Guo <gu...@gmail.com>.
Hi Puneet,
I only use Watson's text to speech as user interface, because a lot of
people think NLP is the same as voice recognition. If you don't need voice
recognition you could remove Watson from it. Stanford has better dependency
parsing and can be used offline. However it seems you are using Watson's
retrieve and rank API, which is based on Solr, am I correct?
Yangrui
On Saturday, July 9, 2016, Puneet Pawaia <pu...@gmail.com> wrote:
> Hi Yangrui,
>
> I have been looking at your code for squery.
> Unfortunately, I am not very conversant with SolrJ. I seem to be missing
> how and what data is added to the Solr index.
> Also, I see some references to IBM Watson in your code. Are you using IBM
> Watson? If yes, then why use the Stanford NLP if you can use the Watson
> NLP?
>
> Regards
> Puneet
>
>
> On Sat, Jul 9, 2016 at 11:37 AM, Puneet Pawaia <puneet.pawaia@gmail.com
> <javascript:;>>
> wrote:
>
> > Hi Alessandro
> >
> > I am looking at being able to answer questions like "Can a non-compete
> > clause in an employment agreement be enforced after the expiry of the
> > agreement?"
> > We are doing some testing with IBM Watson and with a sample test data, we
> > are able to get relevant replies to the above question. Since IBM Watson
> > uses Solr at its backend, I was wondering if we can get the same working
> at
> > the Solr level without having to use Watson.
> >
> > Regards
> > Puneet
> >
> > On Sat, Jul 9, 2016 at 11:34 AM, Puneet Pawaia <puneet.pawaia@gmail.com
> <javascript:;>>
> > wrote:
> >
> >> Hi Alessandro
> >>
> >> I am looking at being able to answer questions like "Can a non-compete
> >> clause in an employment agreement be enforced after the expiry of the
> >> agreement?"
> >>
> >> On Sat, Jul 9, 2016 at 4:34 AM, Alessandro Benedetti <
> >> abenedetti@apache.org <javascript:;>> wrote:
> >>
> >>> Hi Puneet,
> >>> your requirement :
> >>> "I would like users to be able to write queries in natural language
> >>> rather
> >>> than keyword based search."
> >>>
> >>> Is really really vague :(
> >>> Can you try to help us with some specific example, starting of course
> >>> from
> >>> the simplest use cases you have initially in mind ?
> >>>
> >>> Moving from keyword based search to natural language is a really
> complex
> >>> task.
> >>> Proceeding step by step can help you.
> >>>
> >>> Do you want for example to set up a Q&A basic system ?
> >>> In that case you should take care of query rewriting.
> >>> You need basically to identify your base requirement and then build a
> >>> specific parser for that.
> >>> You can use triple stores and knowledge bases to enrich both your query
> >>> and
> >>> your index, but let's start from the basis, what is your simplest
> >>> requirement ?
> >>>
> >>> On Fri, Jul 8, 2016 at 1:56 PM, Jay Urbain <jay.urbain@gmail.com
> <javascript:;>> wrote:
> >>>
> >>> > I've added multivalued fields within my SOLR schema for indexing
> >>> entities
> >>> > extracted using NLP methods applied to the text I'm indexing, along
> >>> with
> >>> > fields for other discrete data extracted from relational databases.
> >>> >
> >>> > A Java application reads data out of multiple relational databases,
> >>> uses
> >>> > NLP on the text and indexes each document (de-normalized) using
> SOLRJ.
> >>> >
> >>> > I initially tried doing this with content handlers, but found it much
> >>> > easier to just write a Java application.
> >>> >
> >>> > SOLRJ Java API reference:
> >>> > https://cwiki.apache.org/confluence/display/solr/Using+SolrJ
> >>> >
> >>> > Stanford NLP:
> >>> > http://stanfordnlp.github.io/CoreNLP/
> >>> >
> >>> > Best,
> >>> > Jay
> >>> >
> >>> >
> >>> > On Thu, Jul 7, 2016 at 9:52 PM, Puneet Pawaia <
> puneet.pawaia@gmail.com <javascript:;>
> >>> >
> >>> > wrote:
> >>> >
> >>> > > Hi Jay
> >>> > > Any place I can learn more on this method of integration?
> >>> > > Thanks
> >>> > > Puneet
> >>> > >
> >>> > > On 8 Jul 2016 02:58, "Jay Urbain" <jay.urbain@gmail.com
> <javascript:;>> wrote:
> >>> > >
> >>> > > > I use Stanford NLP and cTakes (based on OpenNLP) while indexing
> >>> with a
> >>> > > > SOLRJ application.
> >>> > > >
> >>> > > > Best,
> >>> > > > Jay
> >>> > > >
> >>> > > > On Thu, Jul 7, 2016 at 12:09 PM, Puneet Pawaia <
> >>> > puneet.pawaia@gmail.com <javascript:;>>
> >>> > > > wrote:
> >>> > > >
> >>> > > > > Hi
> >>> > > > >
> >>> > > > > I am currently using Solr 5.5.x to test but can upgrade to Solr
> >>> 6.x
> >>> > if
> >>> > > > > required.
> >>> > > > > I am working on a POC for natural language query using Solr.
> >>> Should I
> >>> > > use
> >>> > > > > the Stanford libraries or are there any other libraries having
> >>> > > > integration
> >>> > > > > with Solr already available.
> >>> > > > > Any direction in how to do this would be most appreciated. How
> >>> > should I
> >>> > > > > process the query to give relevant results.
> >>> > > > >
> >>> > > > > Regards
> >>> > > > > Puneet
> >>> > > > >
> >>> > > >
> >>> > >
> >>> >
> >>>
> >>>
> >>>
> >>> --
> >>> --------------------------
> >>>
> >>> Benedetti Alessandro
> >>> Visiting card : http://about.me/alessandro_benedetti
> >>>
> >>> "Tyger, tyger burning bright
> >>> In the forests of the night,
> >>> What immortal hand or eye
> >>> Could frame thy fearful symmetry?"
> >>>
> >>> William Blake - Songs of Experience -1794 England
> >>>
> >>
> >>
> >
>
Re: Integrating Stanford NLP or any other NLP for Natural Language Query
Posted by Puneet Pawaia <pu...@gmail.com>.
Hi Yangrui,
I have been looking at your code for squery.
Unfortunately, I am not very conversant with SolrJ. I seem to be missing
how and what data is added to the Solr index.
Also, I see some references to IBM Watson in your code. Are you using IBM
Watson? If yes, then why use the Stanford NLP if you can use the Watson NLP?
Regards
Puneet
On Sat, Jul 9, 2016 at 11:37 AM, Puneet Pawaia <pu...@gmail.com>
wrote:
> Hi Alessandro
>
> I am looking at being able to answer questions like "Can a non-compete
> clause in an employment agreement be enforced after the expiry of the
> agreement?"
> We are doing some testing with IBM Watson and with a sample test data, we
> are able to get relevant replies to the above question. Since IBM Watson
> uses Solr at its backend, I was wondering if we can get the same working at
> the Solr level without having to use Watson.
>
> Regards
> Puneet
>
> On Sat, Jul 9, 2016 at 11:34 AM, Puneet Pawaia <pu...@gmail.com>
> wrote:
>
>> Hi Alessandro
>>
>> I am looking at being able to answer questions like "Can a non-compete
>> clause in an employment agreement be enforced after the expiry of the
>> agreement?"
>>
>> On Sat, Jul 9, 2016 at 4:34 AM, Alessandro Benedetti <
>> abenedetti@apache.org> wrote:
>>
>>> Hi Puneet,
>>> your requirement :
>>> "I would like users to be able to write queries in natural language
>>> rather
>>> than keyword based search."
>>>
>>> Is really really vague :(
>>> Can you try to help us with some specific example, starting of course
>>> from
>>> the simplest use cases you have initially in mind ?
>>>
>>> Moving from keyword based search to natural language is a really complex
>>> task.
>>> Proceeding step by step can help you.
>>>
>>> Do you want for example to set up a Q&A basic system ?
>>> In that case you should take care of query rewriting.
>>> You need basically to identify your base requirement and then build a
>>> specific parser for that.
>>> You can use triple stores and knowledge bases to enrich both your query
>>> and
>>> your index, but let's start from the basis, what is your simplest
>>> requirement ?
>>>
>>> On Fri, Jul 8, 2016 at 1:56 PM, Jay Urbain <ja...@gmail.com> wrote:
>>>
>>> > I've added multivalued fields within my SOLR schema for indexing
>>> entities
>>> > extracted using NLP methods applied to the text I'm indexing, along
>>> with
>>> > fields for other discrete data extracted from relational databases.
>>> >
>>> > A Java application reads data out of multiple relational databases,
>>> uses
>>> > NLP on the text and indexes each document (de-normalized) using SOLRJ.
>>> >
>>> > I initially tried doing this with content handlers, but found it much
>>> > easier to just write a Java application.
>>> >
>>> > SOLRJ Java API reference:
>>> > https://cwiki.apache.org/confluence/display/solr/Using+SolrJ
>>> >
>>> > Stanford NLP:
>>> > http://stanfordnlp.github.io/CoreNLP/
>>> >
>>> > Best,
>>> > Jay
>>> >
>>> >
>>> > On Thu, Jul 7, 2016 at 9:52 PM, Puneet Pawaia <puneet.pawaia@gmail.com
>>> >
>>> > wrote:
>>> >
>>> > > Hi Jay
>>> > > Any place I can learn more on this method of integration?
>>> > > Thanks
>>> > > Puneet
>>> > >
>>> > > On 8 Jul 2016 02:58, "Jay Urbain" <ja...@gmail.com> wrote:
>>> > >
>>> > > > I use Stanford NLP and cTakes (based on OpenNLP) while indexing
>>> with a
>>> > > > SOLRJ application.
>>> > > >
>>> > > > Best,
>>> > > > Jay
>>> > > >
>>> > > > On Thu, Jul 7, 2016 at 12:09 PM, Puneet Pawaia <
>>> > puneet.pawaia@gmail.com>
>>> > > > wrote:
>>> > > >
>>> > > > > Hi
>>> > > > >
>>> > > > > I am currently using Solr 5.5.x to test but can upgrade to Solr
>>> 6.x
>>> > if
>>> > > > > required.
>>> > > > > I am working on a POC for natural language query using Solr.
>>> Should I
>>> > > use
>>> > > > > the Stanford libraries or are there any other libraries having
>>> > > > integration
>>> > > > > with Solr already available.
>>> > > > > Any direction in how to do this would be most appreciated. How
>>> > should I
>>> > > > > process the query to give relevant results.
>>> > > > >
>>> > > > > Regards
>>> > > > > Puneet
>>> > > > >
>>> > > >
>>> > >
>>> >
>>>
>>>
>>>
>>> --
>>> --------------------------
>>>
>>> Benedetti Alessandro
>>> Visiting card : http://about.me/alessandro_benedetti
>>>
>>> "Tyger, tyger burning bright
>>> In the forests of the night,
>>> What immortal hand or eye
>>> Could frame thy fearful symmetry?"
>>>
>>> William Blake - Songs of Experience -1794 England
>>>
>>
>>
>
Re: Integrating Stanford NLP or any other NLP for Natural Language Query
Posted by Puneet Pawaia <pu...@gmail.com>.
Hi Alessandro
I am looking at being able to answer questions like "Can a non-compete
clause in an employment agreement be enforced after the expiry of the
agreement?"
We are doing some testing with IBM Watson and with a sample test data, we
are able to get relevant replies to the above question. Since IBM Watson
uses Solr at its backend, I was wondering if we can get the same working at
the Solr level without having to use Watson.
Regards
Puneet
On Sat, Jul 9, 2016 at 11:34 AM, Puneet Pawaia <pu...@gmail.com>
wrote:
> Hi Alessandro
>
> I am looking at being able to answer questions like "Can a non-compete
> clause in an employment agreement be enforced after the expiry of the
> agreement?"
>
> On Sat, Jul 9, 2016 at 4:34 AM, Alessandro Benedetti <
> abenedetti@apache.org> wrote:
>
>> Hi Puneet,
>> your requirement :
>> "I would like users to be able to write queries in natural language rather
>> than keyword based search."
>>
>> Is really really vague :(
>> Can you try to help us with some specific example, starting of course from
>> the simplest use cases you have initially in mind ?
>>
>> Moving from keyword based search to natural language is a really complex
>> task.
>> Proceeding step by step can help you.
>>
>> Do you want for example to set up a Q&A basic system ?
>> In that case you should take care of query rewriting.
>> You need basically to identify your base requirement and then build a
>> specific parser for that.
>> You can use triple stores and knowledge bases to enrich both your query
>> and
>> your index, but let's start from the basis, what is your simplest
>> requirement ?
>>
>> On Fri, Jul 8, 2016 at 1:56 PM, Jay Urbain <ja...@gmail.com> wrote:
>>
>> > I've added multivalued fields within my SOLR schema for indexing
>> entities
>> > extracted using NLP methods applied to the text I'm indexing, along with
>> > fields for other discrete data extracted from relational databases.
>> >
>> > A Java application reads data out of multiple relational databases, uses
>> > NLP on the text and indexes each document (de-normalized) using SOLRJ.
>> >
>> > I initially tried doing this with content handlers, but found it much
>> > easier to just write a Java application.
>> >
>> > SOLRJ Java API reference:
>> > https://cwiki.apache.org/confluence/display/solr/Using+SolrJ
>> >
>> > Stanford NLP:
>> > http://stanfordnlp.github.io/CoreNLP/
>> >
>> > Best,
>> > Jay
>> >
>> >
>> > On Thu, Jul 7, 2016 at 9:52 PM, Puneet Pawaia <pu...@gmail.com>
>> > wrote:
>> >
>> > > Hi Jay
>> > > Any place I can learn more on this method of integration?
>> > > Thanks
>> > > Puneet
>> > >
>> > > On 8 Jul 2016 02:58, "Jay Urbain" <ja...@gmail.com> wrote:
>> > >
>> > > > I use Stanford NLP and cTakes (based on OpenNLP) while indexing
>> with a
>> > > > SOLRJ application.
>> > > >
>> > > > Best,
>> > > > Jay
>> > > >
>> > > > On Thu, Jul 7, 2016 at 12:09 PM, Puneet Pawaia <
>> > puneet.pawaia@gmail.com>
>> > > > wrote:
>> > > >
>> > > > > Hi
>> > > > >
>> > > > > I am currently using Solr 5.5.x to test but can upgrade to Solr
>> 6.x
>> > if
>> > > > > required.
>> > > > > I am working on a POC for natural language query using Solr.
>> Should I
>> > > use
>> > > > > the Stanford libraries or are there any other libraries having
>> > > > integration
>> > > > > with Solr already available.
>> > > > > Any direction in how to do this would be most appreciated. How
>> > should I
>> > > > > process the query to give relevant results.
>> > > > >
>> > > > > Regards
>> > > > > Puneet
>> > > > >
>> > > >
>> > >
>> >
>>
>>
>>
>> --
>> --------------------------
>>
>> Benedetti Alessandro
>> Visiting card : http://about.me/alessandro_benedetti
>>
>> "Tyger, tyger burning bright
>> In the forests of the night,
>> What immortal hand or eye
>> Could frame thy fearful symmetry?"
>>
>> William Blake - Songs of Experience -1794 England
>>
>
>
Re: Integrating Stanford NLP or any other NLP for Natural Language Query
Posted by Puneet Pawaia <pu...@gmail.com>.
Hi Alessandro
I am looking at being able to answer questions like "Can a non-compete
clause in an employment agreement be enforced after the expiry of the
agreement?"
On Sat, Jul 9, 2016 at 4:34 AM, Alessandro Benedetti <ab...@apache.org>
wrote:
> Hi Puneet,
> your requirement :
> "I would like users to be able to write queries in natural language rather
> than keyword based search."
>
> Is really really vague :(
> Can you try to help us with some specific example, starting of course from
> the simplest use cases you have initially in mind ?
>
> Moving from keyword based search to natural language is a really complex
> task.
> Proceeding step by step can help you.
>
> Do you want for example to set up a Q&A basic system ?
> In that case you should take care of query rewriting.
> You need basically to identify your base requirement and then build a
> specific parser for that.
> You can use triple stores and knowledge bases to enrich both your query and
> your index, but let's start from the basis, what is your simplest
> requirement ?
>
> On Fri, Jul 8, 2016 at 1:56 PM, Jay Urbain <ja...@gmail.com> wrote:
>
> > I've added multivalued fields within my SOLR schema for indexing entities
> > extracted using NLP methods applied to the text I'm indexing, along with
> > fields for other discrete data extracted from relational databases.
> >
> > A Java application reads data out of multiple relational databases, uses
> > NLP on the text and indexes each document (de-normalized) using SOLRJ.
> >
> > I initially tried doing this with content handlers, but found it much
> > easier to just write a Java application.
> >
> > SOLRJ Java API reference:
> > https://cwiki.apache.org/confluence/display/solr/Using+SolrJ
> >
> > Stanford NLP:
> > http://stanfordnlp.github.io/CoreNLP/
> >
> > Best,
> > Jay
> >
> >
> > On Thu, Jul 7, 2016 at 9:52 PM, Puneet Pawaia <pu...@gmail.com>
> > wrote:
> >
> > > Hi Jay
> > > Any place I can learn more on this method of integration?
> > > Thanks
> > > Puneet
> > >
> > > On 8 Jul 2016 02:58, "Jay Urbain" <ja...@gmail.com> wrote:
> > >
> > > > I use Stanford NLP and cTakes (based on OpenNLP) while indexing with
> a
> > > > SOLRJ application.
> > > >
> > > > Best,
> > > > Jay
> > > >
> > > > On Thu, Jul 7, 2016 at 12:09 PM, Puneet Pawaia <
> > puneet.pawaia@gmail.com>
> > > > wrote:
> > > >
> > > > > Hi
> > > > >
> > > > > I am currently using Solr 5.5.x to test but can upgrade to Solr 6.x
> > if
> > > > > required.
> > > > > I am working on a POC for natural language query using Solr.
> Should I
> > > use
> > > > > the Stanford libraries or are there any other libraries having
> > > > integration
> > > > > with Solr already available.
> > > > > Any direction in how to do this would be most appreciated. How
> > should I
> > > > > process the query to give relevant results.
> > > > >
> > > > > Regards
> > > > > Puneet
> > > > >
> > > >
> > >
> >
>
>
>
> --
> --------------------------
>
> Benedetti Alessandro
> Visiting card : http://about.me/alessandro_benedetti
>
> "Tyger, tyger burning bright
> In the forests of the night,
> What immortal hand or eye
> Could frame thy fearful symmetry?"
>
> William Blake - Songs of Experience -1794 England
>
Re: Integrating Stanford NLP or any other NLP for Natural Language Query
Posted by Alessandro Benedetti <ab...@apache.org>.
Hi Puneet,
your requirement :
"I would like users to be able to write queries in natural language rather
than keyword based search."
Is really really vague :(
Can you try to help us with some specific example, starting of course from
the simplest use cases you have initially in mind ?
Moving from keyword based search to natural language is a really complex
task.
Proceeding step by step can help you.
Do you want for example to set up a Q&A basic system ?
In that case you should take care of query rewriting.
You need basically to identify your base requirement and then build a
specific parser for that.
You can use triple stores and knowledge bases to enrich both your query and
your index, but let's start from the basis, what is your simplest
requirement ?
On Fri, Jul 8, 2016 at 1:56 PM, Jay Urbain <ja...@gmail.com> wrote:
> I've added multivalued fields within my SOLR schema for indexing entities
> extracted using NLP methods applied to the text I'm indexing, along with
> fields for other discrete data extracted from relational databases.
>
> A Java application reads data out of multiple relational databases, uses
> NLP on the text and indexes each document (de-normalized) using SOLRJ.
>
> I initially tried doing this with content handlers, but found it much
> easier to just write a Java application.
>
> SOLRJ Java API reference:
> https://cwiki.apache.org/confluence/display/solr/Using+SolrJ
>
> Stanford NLP:
> http://stanfordnlp.github.io/CoreNLP/
>
> Best,
> Jay
>
>
> On Thu, Jul 7, 2016 at 9:52 PM, Puneet Pawaia <pu...@gmail.com>
> wrote:
>
> > Hi Jay
> > Any place I can learn more on this method of integration?
> > Thanks
> > Puneet
> >
> > On 8 Jul 2016 02:58, "Jay Urbain" <ja...@gmail.com> wrote:
> >
> > > I use Stanford NLP and cTakes (based on OpenNLP) while indexing with a
> > > SOLRJ application.
> > >
> > > Best,
> > > Jay
> > >
> > > On Thu, Jul 7, 2016 at 12:09 PM, Puneet Pawaia <
> puneet.pawaia@gmail.com>
> > > wrote:
> > >
> > > > Hi
> > > >
> > > > I am currently using Solr 5.5.x to test but can upgrade to Solr 6.x
> if
> > > > required.
> > > > I am working on a POC for natural language query using Solr. Should I
> > use
> > > > the Stanford libraries or are there any other libraries having
> > > integration
> > > > with Solr already available.
> > > > Any direction in how to do this would be most appreciated. How
> should I
> > > > process the query to give relevant results.
> > > >
> > > > Regards
> > > > Puneet
> > > >
> > >
> >
>
--
--------------------------
Benedetti Alessandro
Visiting card : http://about.me/alessandro_benedetti
"Tyger, tyger burning bright
In the forests of the night,
What immortal hand or eye
Could frame thy fearful symmetry?"
William Blake - Songs of Experience -1794 England
Re: Integrating Stanford NLP or any other NLP for Natural Language Query
Posted by Jay Urbain <ja...@gmail.com>.
I've added multivalued fields within my SOLR schema for indexing entities
extracted using NLP methods applied to the text I'm indexing, along with
fields for other discrete data extracted from relational databases.
A Java application reads data out of multiple relational databases, uses
NLP on the text and indexes each document (de-normalized) using SOLRJ.
I initially tried doing this with content handlers, but found it much
easier to just write a Java application.
SOLRJ Java API reference:
https://cwiki.apache.org/confluence/display/solr/Using+SolrJ
Stanford NLP:
http://stanfordnlp.github.io/CoreNLP/
Best,
Jay
On Thu, Jul 7, 2016 at 9:52 PM, Puneet Pawaia <pu...@gmail.com>
wrote:
> Hi Jay
> Any place I can learn more on this method of integration?
> Thanks
> Puneet
>
> On 8 Jul 2016 02:58, "Jay Urbain" <ja...@gmail.com> wrote:
>
> > I use Stanford NLP and cTakes (based on OpenNLP) while indexing with a
> > SOLRJ application.
> >
> > Best,
> > Jay
> >
> > On Thu, Jul 7, 2016 at 12:09 PM, Puneet Pawaia <pu...@gmail.com>
> > wrote:
> >
> > > Hi
> > >
> > > I am currently using Solr 5.5.x to test but can upgrade to Solr 6.x if
> > > required.
> > > I am working on a POC for natural language query using Solr. Should I
> use
> > > the Stanford libraries or are there any other libraries having
> > integration
> > > with Solr already available.
> > > Any direction in how to do this would be most appreciated. How should I
> > > process the query to give relevant results.
> > >
> > > Regards
> > > Puneet
> > >
> >
>
Re: Integrating Stanford NLP or any other NLP for Natural Language Query
Posted by Puneet Pawaia <pu...@gmail.com>.
Hi Jay
Any place I can learn more on this method of integration?
Thanks
Puneet
On 8 Jul 2016 02:58, "Jay Urbain" <ja...@gmail.com> wrote:
> I use Stanford NLP and cTakes (based on OpenNLP) while indexing with a
> SOLRJ application.
>
> Best,
> Jay
>
> On Thu, Jul 7, 2016 at 12:09 PM, Puneet Pawaia <pu...@gmail.com>
> wrote:
>
> > Hi
> >
> > I am currently using Solr 5.5.x to test but can upgrade to Solr 6.x if
> > required.
> > I am working on a POC for natural language query using Solr. Should I use
> > the Stanford libraries or are there any other libraries having
> integration
> > with Solr already available.
> > Any direction in how to do this would be most appreciated. How should I
> > process the query to give relevant results.
> >
> > Regards
> > Puneet
> >
>
Re: Integrating Stanford NLP or any other NLP for Natural Language Query
Posted by Jay Urbain <ja...@gmail.com>.
I use Stanford NLP and cTakes (based on OpenNLP) while indexing with a
SOLRJ application.
Best,
Jay
On Thu, Jul 7, 2016 at 12:09 PM, Puneet Pawaia <pu...@gmail.com>
wrote:
> Hi
>
> I am currently using Solr 5.5.x to test but can upgrade to Solr 6.x if
> required.
> I am working on a POC for natural language query using Solr. Should I use
> the Stanford libraries or are there any other libraries having integration
> with Solr already available.
> Any direction in how to do this would be most appreciated. How should I
> process the query to give relevant results.
>
> Regards
> Puneet
>