High Query Volume

[Sheetal <ri...@gmail.com>]

Hello,

I am using solr MoreLikeThis for finding similar result. I have all data
indexed into my Solr server. And the indexed data is also too huge. The data
ranges to millions. 

What I am trying to do is, given the "ID", it should check the contents of
that respective ID and give me the result similar to the contents of that
"ID".

What my problem is , since the contents of the ID is too large. The  term
vectors/Term frequency becomes too huge. Also the maximum number of query
terms (mlt.maxqt) that will be included in generated query depends upon the
ID , as some contents are hundreds and some are millions. As I have the ID
and its contents, I can find and pass the mlt.maxqt depending upon the ID.
So, depending upon the contents, my query limit is  sometimes mlt.maxqt=100
, sometimes mlt.maxqt=1000, and sometimes even mlt.maxqt=100000……..

If my mlt.maxqt=100 , then the result comes pretty fast. But when its
mlt.maxqt=1000 or more, its too too slow obviously…... Is there any way that
I can use to solve this issue...scale solr in anyway…Is there any way that I
can handle huge Query volume in searching. I know the default query term is
25 but I need lot more than that. Am I using the right tool(solr
morelikethis this) ? 

Also, I have my solr running with 2GB and my application running with 2GB.

Any thoughts and help would be real helpful.Thank you in advance.



--
View this message in context: http://lucene.472066.n3.nabble.com/High-Query-Volume-tp3172274p3172274.html
Sent from the Solr - User mailing list archive at Nabble.com.