You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by rohan rai <hi...@gmail.com> on 2011/02/03 19:02:38 UTC
Solr for finding similar word between two documents
Is there a way to use solr and get similar words between two document
(files).
Any ideas
Regards
Rohan
Re: Solr for finding similar word between two documents
Posted by Otis Gospodnetic <ot...@yahoo.com>.
Rohan,
You can really do that with Lucene's tokenizers to get individual tokens/words
and a HashMap where keys are those words/tokens from the first document. You
can then tokenize the second doc and check each of its words in the HashMap.
Our Key Phrase Extractor (
http://sematext.com/products/key-phrase-extractor/index.html ) includes similar
functionality that works with 2 corpora (or 2 pieces of text or 2 language
models) and gets you the "overlap". I think it also takes into consideration
term frequencies, which can be handy.
Otis
----
Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch
Lucene ecosystem search :: http://search-lucene.com/
----- Original Message ----
> From: rohan rai <hi...@gmail.com>
> To: solr-user@lucene.apache.org
> Sent: Thu, February 3, 2011 2:35:39 PM
> Subject: Re: Solr for finding similar word between two documents
>
> Lets say 1 have document(file) which is large and contains word inside it.
>
> And the 2nd document also is a text file.
>
> Problem is to find all those words in 2nd document which is present in first
> document
> when both of the files are large enough.
>
> Regards
> Rohan
>
> On Fri, Feb 4, 2011 at 1:01 AM, openvictor Open <op...@gmail.com>wrote:
>
> > Rohan : what you want to do can be done with quite little effort if your
> > document has a limited size (up to some Mo) with common and basic
> > structures
> > like Hasmap.
> >
> > Do you have any additional information on your problem so that we can give
> > you more useful inputs ?
> >
> > 2011/2/3 Gora Mohanty <go...@mimirtech.com>
> >
> > > On Thu, Feb 3, 2011 at 11:32 PM, rohan rai <hi...@gmail.com> wrote:
> > > > Is there a way to use solr and get similar words between two document
> > > > (files).
> > > [...]
> > >
> > > This is *way* too vague t make any sense out of. Could you elaborate,
> > > as I could have sworn that what you seem to want is the essential
> > > function of a search engine.
> > >
> > > Regards,
> > > Gora
> > >
> >
>
Re: Solr for finding similar word between two documents
Posted by rohan rai <hi...@gmail.com>.
Lets say 1 have document(file) which is large and contains word inside it.
And the 2nd document also is a text file.
Problem is to find all those words in 2nd document which is present in first
document
when both of the files are large enough.
Regards
Rohan
On Fri, Feb 4, 2011 at 1:01 AM, openvictor Open <op...@gmail.com>wrote:
> Rohan : what you want to do can be done with quite little effort if your
> document has a limited size (up to some Mo) with common and basic
> structures
> like Hasmap.
>
> Do you have any additional information on your problem so that we can give
> you more useful inputs ?
>
> 2011/2/3 Gora Mohanty <go...@mimirtech.com>
>
> > On Thu, Feb 3, 2011 at 11:32 PM, rohan rai <hi...@gmail.com> wrote:
> > > Is there a way to use solr and get similar words between two document
> > > (files).
> > [...]
> >
> > This is *way* too vague t make any sense out of. Could you elaborate,
> > as I could have sworn that what you seem to want is the essential
> > function of a search engine.
> >
> > Regards,
> > Gora
> >
>
Re: Solr for finding similar word between two documents
Posted by openvictor Open <op...@gmail.com>.
Rohan : what you want to do can be done with quite little effort if your
document has a limited size (up to some Mo) with common and basic structures
like Hasmap.
Do you have any additional information on your problem so that we can give
you more useful inputs ?
2011/2/3 Gora Mohanty <go...@mimirtech.com>
> On Thu, Feb 3, 2011 at 11:32 PM, rohan rai <hi...@gmail.com> wrote:
> > Is there a way to use solr and get similar words between two document
> > (files).
> [...]
>
> This is *way* too vague t make any sense out of. Could you elaborate,
> as I could have sworn that what you seem to want is the essential
> function of a search engine.
>
> Regards,
> Gora
>
Re: Solr for finding similar word between two documents
Posted by Gora Mohanty <go...@mimirtech.com>.
On Thu, Feb 3, 2011 at 11:32 PM, rohan rai <hi...@gmail.com> wrote:
> Is there a way to use solr and get similar words between two document
> (files).
[...]
This is *way* too vague t make any sense out of. Could you elaborate,
as I could have sworn that what you seem to want is the essential
function of a search engine.
Regards,
Gora