You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Sathya <sa...@gmail.com> on 2014/04/04 09:31:02 UTC
How to reduce the search speed of solrcloud
Hi All,
Hi All, I am new to Solr. And i dont know how to increase the search speed
of solrcloud. I have indexed nearly 4 GB of data. When i am searching a
document using java with solrj, solr takes more 6 seconds to return a query
result. Any one please help me to reduce the search query time to less than
500 ms. i have allocate the 4 GB ram for solr. Please let me know for
further details about solrcloud config.
--
View this message in context: http://lucene.472066.n3.nabble.com/How-to-reduce-the-search-speed-of-solrcloud-tp4129067.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: How to reduce the search speed of solrcloud
Posted by Sathya <sa...@gmail.com>.
Hi Anshum,
I am using Solr 4.7. And i follow this
tutorial<http://myjeeva.com/solrcloud-cluster-single-collection-deployment.html>to
setup a solrcloud. I have only one collection in my solr. Kindly let
me,
if u need more details.
On Fri, Apr 4, 2014 at 11:19 PM, Anshum Gupta [via Lucene] <
ml-node+s472066n4129190h65@n3.nabble.com> wrote:
> I am not sure if you setup your SolrCloud right. Can you also provide me
> with the version of Solr that you're running.
> Also, if you could tell me about how did you setup your SolrCloud cluster.
> Are the times consistent? Is this the only collection on the cluster?
>
> Also, if I am getting it right, you have 15 ZKs running. Correct me if I'm
> wrong, but if I'm not, you don't need that kind of a zk setup.
>
>
> On Fri, Apr 4, 2014 at 9:39 AM, Sathya <[hidden email]<http://user/SendEmail.jtp?type=node&node=4129190&i=0>>
> wrote:
>
> > Hi shawn,
> >
> > I have indexed 50 million data in 5 servers. 3 servers have 8gb ram. One
> > have 24gb and another one have 64gb ram. I was allocate 4 gb ram to solr
> in
> > each machine. I am using solrcloud. My total index size is 50gb
> including 5
> > servers. Each server have 3 zookeepers. Still I didnt check about OS
> disk
> > cache and heap memory. I will check and let u know shawn. If anything,
> pls
> > let me know.
> >
> > Thank u shawn.
> >
> > On Friday, April 4, 2014, Shawn Heisey-4 [via Lucene] <
> > [hidden email] <http://user/SendEmail.jtp?type=node&node=4129190&i=1>>
> wrote:
> > > On 4/4/2014 1:31 AM, Sathya wrote:
> > >> Hi All,
> > >>
> > >> Hi All, I am new to Solr. And i dont know how to increase the search
> > speed
> > >> of solrcloud. I have indexed nearly 4 GB of data. When i am searching
> a
> > >> document using java with solrj, solr takes more 6 seconds to return a
> > query
> > >> result. Any one please help me to reduce the search query time to
> less
> > than
> > >> 500 ms. i have allocate the 4 GB ram for solr. Please let me know for
> > >> further details about solrcloud config.
> > >
> > > How much total RAM do you have on the system, and how much total index
> > > data is on that system (adding up all the Solr cores)? You've already
> > > said that you have allocated 4GB of RAM for Solr.
> > >
> > > Later you said you had 50 million documents, and then you showed us a
> > > URL that looks like SolrCloud.
> > >
> > > I suspect that you don't have enough RAM left over to cache your index
> > > effectively -- the OS Disk Cache is too small.
> > >
> > > http://wiki.apache.org/solr/SolrPerformanceProblems
> > >
> > > Another possible problem, also discussed on that page, is that your
> Java
> > > heap is too small.
> > >
> > > Thanks,
> > > Shawn
> > >
> > >
> > >
> > > ________________________________
> > > If you reply to this email, your message will be added to the
> discussion
> > below:
> > >
> >
> >
> http://lucene.472066.n3.nabble.com/How-to-reduce-the-search-speed-of-solrcloud-tp4129067p4129150.html
> > > To unsubscribe from How to reduce the search speed of solrcloud, click
> > here.
> > > NAML
> >
> >
> >
> >
> > --
> > View this message in context:
> >
> http://lucene.472066.n3.nabble.com/How-to-reduce-the-search-speed-of-solrcloud-tp4129067p4129173.html
>
> > Sent from the Solr - User mailing list archive at Nabble.com.
> >
>
>
>
> --
>
> Anshum Gupta
> http://www.anshumgupta.net
>
>
> ------------------------------
> If you reply to this email, your message will be added to the discussion
> below:
>
> http://lucene.472066.n3.nabble.com/How-to-reduce-the-search-speed-of-solrcloud-tp4129067p4129190.html
> To unsubscribe from How to reduce the search speed of solrcloud, click
> here<http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=4129067&code=c2F0aGlhLmJsYWNrc3RhckBnbWFpbC5jb218NDEyOTA2N3wtMjEyNDcwMTI5OA==>
> .
> NAML<http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>
--
View this message in context: http://lucene.472066.n3.nabble.com/How-to-reduce-the-search-speed-of-solrcloud-tp4129067p4129910.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: How to reduce the search speed of solrcloud
Posted by Sathya <sa...@gmail.com>.
Hi,
I use this link to setup a solrcloud
http://myjeeva.com/solrcloud-cluster-single-collection-deployment.html And
i use 5 different machine to setup this cloud. I use Unique id.
On Sat, Apr 5, 2014 at 6:30 AM, Alexandre Rafalovitch [via Lucene] <
ml-node+s472066n4129333h98@n3.nabble.com> wrote:
> And 50 million records of 3 fields each should not become 50Gb of
> data. Something smells wrong there. Do you have unique IDs setup?
>
> Regards,
> Alex.
> Personal website: http://www.outerthoughts.com/
> Current project: http://www.solr-start.com/ - Accelerating your Solr
> proficiency
>
>
> On Sat, Apr 5, 2014 at 12:48 AM, Anshum Gupta <[hidden email]<http://user/SendEmail.jtp?type=node&node=4129333&i=0>>
> wrote:
>
> > I am not sure if you setup your SolrCloud right. Can you also provide me
> > with the version of Solr that you're running.
> > Also, if you could tell me about how did you setup your SolrCloud
> cluster.
> > Are the times consistent? Is this the only collection on the cluster?
> >
> > Also, if I am getting it right, you have 15 ZKs running. Correct me if
> I'm
> > wrong, but if I'm not, you don't need that kind of a zk setup.
> >
> >
> > On Fri, Apr 4, 2014 at 9:39 AM, Sathya <[hidden email]<http://user/SendEmail.jtp?type=node&node=4129333&i=1>>
> wrote:
> >
> >> Hi shawn,
> >>
> >> I have indexed 50 million data in 5 servers. 3 servers have 8gb ram.
> One
> >> have 24gb and another one have 64gb ram. I was allocate 4 gb ram to
> solr in
> >> each machine. I am using solrcloud. My total index size is 50gb
> including 5
> >> servers. Each server have 3 zookeepers. Still I didnt check about OS
> disk
> >> cache and heap memory. I will check and let u know shawn. If anything,
> pls
> >> let me know.
> >>
> >> Thank u shawn.
> >>
> >> On Friday, April 4, 2014, Shawn Heisey-4 [via Lucene] <
> >> [hidden email] <http://user/SendEmail.jtp?type=node&node=4129333&i=2>>
> wrote:
> >> > On 4/4/2014 1:31 AM, Sathya wrote:
> >> >> Hi All,
> >> >>
> >> >> Hi All, I am new to Solr. And i dont know how to increase the search
> >> speed
> >> >> of solrcloud. I have indexed nearly 4 GB of data. When i am
> searching a
> >> >> document using java with solrj, solr takes more 6 seconds to return
> a
> >> query
> >> >> result. Any one please help me to reduce the search query time to
> less
> >> than
> >> >> 500 ms. i have allocate the 4 GB ram for solr. Please let me know
> for
> >> >> further details about solrcloud config.
> >> >
> >> > How much total RAM do you have on the system, and how much total
> index
> >> > data is on that system (adding up all the Solr cores)? You've
> already
> >> > said that you have allocated 4GB of RAM for Solr.
> >> >
> >> > Later you said you had 50 million documents, and then you showed us a
> >> > URL that looks like SolrCloud.
> >> >
> >> > I suspect that you don't have enough RAM left over to cache your
> index
> >> > effectively -- the OS Disk Cache is too small.
> >> >
> >> > http://wiki.apache.org/solr/SolrPerformanceProblems
> >> >
> >> > Another possible problem, also discussed on that page, is that your
> Java
> >> > heap is too small.
> >> >
> >> > Thanks,
> >> > Shawn
> >> >
> >> >
> >> >
> >> > ________________________________
> >> > If you reply to this email, your message will be added to the
> discussion
> >> below:
> >> >
> >>
> >>
> http://lucene.472066.n3.nabble.com/How-to-reduce-the-search-speed-of-solrcloud-tp4129067p4129150.html
> >> > To unsubscribe from How to reduce the search speed of solrcloud,
> click
> >> here.
> >> > NAML
> >>
> >>
> >>
> >>
> >> --
> >> View this message in context:
> >>
> http://lucene.472066.n3.nabble.com/How-to-reduce-the-search-speed-of-solrcloud-tp4129067p4129173.html
> >> Sent from the Solr - User mailing list archive at Nabble.com.
> >>
> >
> >
> >
> > --
> >
> > Anshum Gupta
> > http://www.anshumgupta.net
>
>
> ------------------------------
> If you reply to this email, your message will be added to the discussion
> below:
>
> http://lucene.472066.n3.nabble.com/How-to-reduce-the-search-speed-of-solrcloud-tp4129067p4129333.html
> To unsubscribe from How to reduce the search speed of solrcloud, click
> here<http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=4129067&code=c2F0aGlhLmJsYWNrc3RhckBnbWFpbC5jb218NDEyOTA2N3wtMjEyNDcwMTI5OA==>
> .
> NAML<http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>
--
View this message in context: http://lucene.472066.n3.nabble.com/How-to-reduce-the-search-speed-of-solrcloud-tp4129067p4129564.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: How to reduce the search speed of solrcloud
Posted by Alexandre Rafalovitch <ar...@gmail.com>.
And 50 million records of 3 fields each should not become 50Gb of
data. Something smells wrong there. Do you have unique IDs setup?
Regards,
Alex.
Personal website: http://www.outerthoughts.com/
Current project: http://www.solr-start.com/ - Accelerating your Solr proficiency
On Sat, Apr 5, 2014 at 12:48 AM, Anshum Gupta <an...@anshumgupta.net> wrote:
> I am not sure if you setup your SolrCloud right. Can you also provide me
> with the version of Solr that you're running.
> Also, if you could tell me about how did you setup your SolrCloud cluster.
> Are the times consistent? Is this the only collection on the cluster?
>
> Also, if I am getting it right, you have 15 ZKs running. Correct me if I'm
> wrong, but if I'm not, you don't need that kind of a zk setup.
>
>
> On Fri, Apr 4, 2014 at 9:39 AM, Sathya <sa...@gmail.com> wrote:
>
>> Hi shawn,
>>
>> I have indexed 50 million data in 5 servers. 3 servers have 8gb ram. One
>> have 24gb and another one have 64gb ram. I was allocate 4 gb ram to solr in
>> each machine. I am using solrcloud. My total index size is 50gb including 5
>> servers. Each server have 3 zookeepers. Still I didnt check about OS disk
>> cache and heap memory. I will check and let u know shawn. If anything, pls
>> let me know.
>>
>> Thank u shawn.
>>
>> On Friday, April 4, 2014, Shawn Heisey-4 [via Lucene] <
>> ml-node+s472066n4129150h25@n3.nabble.com> wrote:
>> > On 4/4/2014 1:31 AM, Sathya wrote:
>> >> Hi All,
>> >>
>> >> Hi All, I am new to Solr. And i dont know how to increase the search
>> speed
>> >> of solrcloud. I have indexed nearly 4 GB of data. When i am searching a
>> >> document using java with solrj, solr takes more 6 seconds to return a
>> query
>> >> result. Any one please help me to reduce the search query time to less
>> than
>> >> 500 ms. i have allocate the 4 GB ram for solr. Please let me know for
>> >> further details about solrcloud config.
>> >
>> > How much total RAM do you have on the system, and how much total index
>> > data is on that system (adding up all the Solr cores)? You've already
>> > said that you have allocated 4GB of RAM for Solr.
>> >
>> > Later you said you had 50 million documents, and then you showed us a
>> > URL that looks like SolrCloud.
>> >
>> > I suspect that you don't have enough RAM left over to cache your index
>> > effectively -- the OS Disk Cache is too small.
>> >
>> > http://wiki.apache.org/solr/SolrPerformanceProblems
>> >
>> > Another possible problem, also discussed on that page, is that your Java
>> > heap is too small.
>> >
>> > Thanks,
>> > Shawn
>> >
>> >
>> >
>> > ________________________________
>> > If you reply to this email, your message will be added to the discussion
>> below:
>> >
>>
>> http://lucene.472066.n3.nabble.com/How-to-reduce-the-search-speed-of-solrcloud-tp4129067p4129150.html
>> > To unsubscribe from How to reduce the search speed of solrcloud, click
>> here.
>> > NAML
>>
>>
>>
>>
>> --
>> View this message in context:
>> http://lucene.472066.n3.nabble.com/How-to-reduce-the-search-speed-of-solrcloud-tp4129067p4129173.html
>> Sent from the Solr - User mailing list archive at Nabble.com.
>>
>
>
>
> --
>
> Anshum Gupta
> http://www.anshumgupta.net
Re: How to reduce the search speed of solrcloud
Posted by Sathya <sa...@gmail.com>.
Hi All,
I found that which is taking more time. It is *server.query*
SolrDataDAO dataDao = new SolrDataDAO();
QueryResponse resp = dataDao.queryData(0, 1, subject);
SolrDocumentList data = resp.getResults();
System.out.println("len " + data.size());
System.out.println();
Subject is passed to here:
public QueryResponse queryData(int start, int rows, String query)
throws Exception {
ModifiableSolrParams solrParams = new ModifiableSolrParams();
try {
query = Clean(query);
query = escaping(query);
// String q = "\"";
// query = q + query + q;
// System.out.println("--------\t------------\t----------");
System.out.println("Checking :" + query);
solrParams.set("q", escaping(query));
solrParams.set("start", start);
solrParams.set("rows", rows);
// solrParams.set("facet", facet); // Facets if required
} catch (Exception e) {
e.printStackTrace();
// System.err.println("Error subject :"+query);
}
//return is taking more time...
return server.query(solrParams);
}
Here i am sending the subject to Solr for searching and it is searched
within 200 to 300 ms. But while returning the search result
to QueryResponse, Its getting delay. I dont know why and how to solve this.
Please help me friends. My project is nearly completing stage. Please help
me to complete. Also let me know more details.
On Tue, Apr 8, 2014 at 10:48 PM, Sathya Narayanan <
sathia.blackstar@gmail.com> wrote:
> Hi Anshum,
>
> I am using Solr 4.7. And i follow this tutorial<http://myjeeva.com/solrcloud-cluster-single-collection-deployment.html>to setup a solrcloud. I have only one collection in my solr. Kindly let me,
> if u need more details.
>
>
> On Fri, Apr 4, 2014 at 11:19 PM, Anshum Gupta [via Lucene] <
> ml-node+s472066n4129190h65@n3.nabble.com> wrote:
>
>> I am not sure if you setup your SolrCloud right. Can you also provide me
>> with the version of Solr that you're running.
>> Also, if you could tell me about how did you setup your SolrCloud
>> cluster.
>> Are the times consistent? Is this the only collection on the cluster?
>>
>> Also, if I am getting it right, you have 15 ZKs running. Correct me if
>> I'm
>> wrong, but if I'm not, you don't need that kind of a zk setup.
>>
>>
>> On Fri, Apr 4, 2014 at 9:39 AM, Sathya <[hidden email]<http://user/SendEmail.jtp?type=node&node=4129190&i=0>>
>> wrote:
>>
>> > Hi shawn,
>> >
>> > I have indexed 50 million data in 5 servers. 3 servers have 8gb ram.
>> One
>> > have 24gb and another one have 64gb ram. I was allocate 4 gb ram to
>> solr in
>> > each machine. I am using solrcloud. My total index size is 50gb
>> including 5
>> > servers. Each server have 3 zookeepers. Still I didnt check about OS
>> disk
>> > cache and heap memory. I will check and let u know shawn. If anything,
>> pls
>> > let me know.
>> >
>> > Thank u shawn.
>> >
>> > On Friday, April 4, 2014, Shawn Heisey-4 [via Lucene] <
>> > [hidden email] <http://user/SendEmail.jtp?type=node&node=4129190&i=1>>
>> wrote:
>> > > On 4/4/2014 1:31 AM, Sathya wrote:
>> > >> Hi All,
>> > >>
>> > >> Hi All, I am new to Solr. And i dont know how to increase the search
>> > speed
>> > >> of solrcloud. I have indexed nearly 4 GB of data. When i am
>> searching a
>> > >> document using java with solrj, solr takes more 6 seconds to return
>> a
>> > query
>> > >> result. Any one please help me to reduce the search query time to
>> less
>> > than
>> > >> 500 ms. i have allocate the 4 GB ram for solr. Please let me know
>> for
>> > >> further details about solrcloud config.
>> > >
>> > > How much total RAM do you have on the system, and how much total
>> index
>> > > data is on that system (adding up all the Solr cores)? You've
>> already
>> > > said that you have allocated 4GB of RAM for Solr.
>> > >
>> > > Later you said you had 50 million documents, and then you showed us a
>> > > URL that looks like SolrCloud.
>> > >
>> > > I suspect that you don't have enough RAM left over to cache your
>> index
>> > > effectively -- the OS Disk Cache is too small.
>> > >
>> > > http://wiki.apache.org/solr/SolrPerformanceProblems
>> > >
>> > > Another possible problem, also discussed on that page, is that your
>> Java
>> > > heap is too small.
>> > >
>> > > Thanks,
>> > > Shawn
>> > >
>> > >
>> > >
>> > > ________________________________
>> > > If you reply to this email, your message will be added to the
>> discussion
>> > below:
>> > >
>> >
>> >
>> http://lucene.472066.n3.nabble.com/How-to-reduce-the-search-speed-of-solrcloud-tp4129067p4129150.html
>> > > To unsubscribe from How to reduce the search speed of solrcloud,
>> click
>> > here.
>> > > NAML
>> >
>> >
>> >
>> >
>> > --
>> > View this message in context:
>> >
>> http://lucene.472066.n3.nabble.com/How-to-reduce-the-search-speed-of-solrcloud-tp4129067p4129173.html
>>
>> > Sent from the Solr - User mailing list archive at Nabble.com.
>> >
>>
>>
>>
>> --
>>
>> Anshum Gupta
>> http://www.anshumgupta.net
>>
>>
>> ------------------------------
>> If you reply to this email, your message will be added to the
>> discussion below:
>>
>> http://lucene.472066.n3.nabble.com/How-to-reduce-the-search-speed-of-solrcloud-tp4129067p4129190.html
>> To unsubscribe from How to reduce the search speed of solrcloud, click
>> here<http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=4129067&code=c2F0aGlhLmJsYWNrc3RhckBnbWFpbC5jb218NDEyOTA2N3wtMjEyNDcwMTI5OA==>
>> .
>> NAML<http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>
>
>
--
View this message in context: http://lucene.472066.n3.nabble.com/How-to-reduce-the-search-speed-of-solrcloud-tp4129067p4129940.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: How to reduce the search speed of solrcloud
Posted by Anshum Gupta <an...@anshumgupta.net>.
I am not sure if you setup your SolrCloud right. Can you also provide me
with the version of Solr that you're running.
Also, if you could tell me about how did you setup your SolrCloud cluster.
Are the times consistent? Is this the only collection on the cluster?
Also, if I am getting it right, you have 15 ZKs running. Correct me if I'm
wrong, but if I'm not, you don't need that kind of a zk setup.
On Fri, Apr 4, 2014 at 9:39 AM, Sathya <sa...@gmail.com> wrote:
> Hi shawn,
>
> I have indexed 50 million data in 5 servers. 3 servers have 8gb ram. One
> have 24gb and another one have 64gb ram. I was allocate 4 gb ram to solr in
> each machine. I am using solrcloud. My total index size is 50gb including 5
> servers. Each server have 3 zookeepers. Still I didnt check about OS disk
> cache and heap memory. I will check and let u know shawn. If anything, pls
> let me know.
>
> Thank u shawn.
>
> On Friday, April 4, 2014, Shawn Heisey-4 [via Lucene] <
> ml-node+s472066n4129150h25@n3.nabble.com> wrote:
> > On 4/4/2014 1:31 AM, Sathya wrote:
> >> Hi All,
> >>
> >> Hi All, I am new to Solr. And i dont know how to increase the search
> speed
> >> of solrcloud. I have indexed nearly 4 GB of data. When i am searching a
> >> document using java with solrj, solr takes more 6 seconds to return a
> query
> >> result. Any one please help me to reduce the search query time to less
> than
> >> 500 ms. i have allocate the 4 GB ram for solr. Please let me know for
> >> further details about solrcloud config.
> >
> > How much total RAM do you have on the system, and how much total index
> > data is on that system (adding up all the Solr cores)? You've already
> > said that you have allocated 4GB of RAM for Solr.
> >
> > Later you said you had 50 million documents, and then you showed us a
> > URL that looks like SolrCloud.
> >
> > I suspect that you don't have enough RAM left over to cache your index
> > effectively -- the OS Disk Cache is too small.
> >
> > http://wiki.apache.org/solr/SolrPerformanceProblems
> >
> > Another possible problem, also discussed on that page, is that your Java
> > heap is too small.
> >
> > Thanks,
> > Shawn
> >
> >
> >
> > ________________________________
> > If you reply to this email, your message will be added to the discussion
> below:
> >
>
> http://lucene.472066.n3.nabble.com/How-to-reduce-the-search-speed-of-solrcloud-tp4129067p4129150.html
> > To unsubscribe from How to reduce the search speed of solrcloud, click
> here.
> > NAML
>
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/How-to-reduce-the-search-speed-of-solrcloud-tp4129067p4129173.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
--
Anshum Gupta
http://www.anshumgupta.net
Re: How to reduce the search speed of solrcloud
Posted by Sathya <sa...@gmail.com>.
Hi shawn,
I have indexed 50 million data in 5 servers. 3 servers have 8gb ram. One
have 24gb and another one have 64gb ram. I was allocate 4 gb ram to solr in
each machine. I am using solrcloud. My total index size is 50gb including 5
servers. Each server have 3 zookeepers. Still I didnt check about OS disk
cache and heap memory. I will check and let u know shawn. If anything, pls
let me know.
Thank u shawn.
On Friday, April 4, 2014, Shawn Heisey-4 [via Lucene] <
ml-node+s472066n4129150h25@n3.nabble.com> wrote:
> On 4/4/2014 1:31 AM, Sathya wrote:
>> Hi All,
>>
>> Hi All, I am new to Solr. And i dont know how to increase the search
speed
>> of solrcloud. I have indexed nearly 4 GB of data. When i am searching a
>> document using java with solrj, solr takes more 6 seconds to return a
query
>> result. Any one please help me to reduce the search query time to less
than
>> 500 ms. i have allocate the 4 GB ram for solr. Please let me know for
>> further details about solrcloud config.
>
> How much total RAM do you have on the system, and how much total index
> data is on that system (adding up all the Solr cores)? You've already
> said that you have allocated 4GB of RAM for Solr.
>
> Later you said you had 50 million documents, and then you showed us a
> URL that looks like SolrCloud.
>
> I suspect that you don't have enough RAM left over to cache your index
> effectively -- the OS Disk Cache is too small.
>
> http://wiki.apache.org/solr/SolrPerformanceProblems
>
> Another possible problem, also discussed on that page, is that your Java
> heap is too small.
>
> Thanks,
> Shawn
>
>
>
> ________________________________
> If you reply to this email, your message will be added to the discussion
below:
>
http://lucene.472066.n3.nabble.com/How-to-reduce-the-search-speed-of-solrcloud-tp4129067p4129150.html
> To unsubscribe from How to reduce the search speed of solrcloud, click
here.
> NAML
--
View this message in context: http://lucene.472066.n3.nabble.com/How-to-reduce-the-search-speed-of-solrcloud-tp4129067p4129173.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: How to reduce the search speed of solrcloud
Posted by Shawn Heisey <so...@elyograg.org>.
On 4/4/2014 1:31 AM, Sathya wrote:
> Hi All,
>
> Hi All, I am new to Solr. And i dont know how to increase the search speed
> of solrcloud. I have indexed nearly 4 GB of data. When i am searching a
> document using java with solrj, solr takes more 6 seconds to return a query
> result. Any one please help me to reduce the search query time to less than
> 500 ms. i have allocate the 4 GB ram for solr. Please let me know for
> further details about solrcloud config.
How much total RAM do you have on the system, and how much total index
data is on that system (adding up all the Solr cores)? You've already
said that you have allocated 4GB of RAM for Solr.
Later you said you had 50 million documents, and then you showed us a
URL that looks like SolrCloud.
I suspect that you don't have enough RAM left over to cache your index
effectively -- the OS Disk Cache is too small.
http://wiki.apache.org/solr/SolrPerformanceProblems
Another possible problem, also discussed on that page, is that your Java
heap is too small.
Thanks,
Shawn
Re: How to reduce the search speed of solrcloud
Posted by Alexandre Rafalovitch <ar...@gmail.com>.
You said your request is 6 seconds when going through the SolrJ
client. But it is 1 second (1000 ms) when going directly to Solr
bypassing the SolrJ. So, the other 5 seconds must be added outside of
Solr. Concentrate on that.
Regarding your schema, you used example schema that has a lot of stuff
you do not need. Here is what a very small schema looks like:
https://github.com/arafalov/solr-indexing-book/blob/master/published/collection1/conf/schema.xml
, so you can compare. That's an example from my book. You may find the
book a fast way to get from your current state to early intermediate
(no cloud examples, though).
Contact me directly if you need a discount.
Regards,
Alex.
Personal website: http://www.outerthoughts.com/
Current project: http://www.solr-start.com/ - Accelerating your Solr proficiency
On Fri, Apr 4, 2014 at 4:11 PM, Sathya <sa...@gmail.com> wrote:
> Hi,
>
> Sorry, i cant get u alex. Can you please explain me(if you can). Because
> now only i entered into solr.
>
>
> On Fri, Apr 4, 2014 at 2:20 PM, Alexandre Rafalovitch [via Lucene] <
> ml-node+s472066n4129077h62@n3.nabble.com> wrote:
>
>> Well, if the direct browser query is 1000ms and your client query is
>> 6seconds, then it is not Solr itself you need to worry about first.
>> Something must be wrong at the client. Trying timing that bit. Maybe
>> it is writing from the client to your ultimate consumer that's the
>> problem.
>>
>> Regards,
>> Alex.
>> P.s. You should probably trim your schema to get rid of all the
>> example fields. Keep _version_ and _root_ but delete all the rest you
>> don't actually use. Same with dynamic fields and all fieldType
>> definitions you do not actually use. You can always reintroduce them
>> later from the example schemas if something is missing.
>>
>> Personal website: http://www.outerthoughts.com/
>> Current project: http://www.solr-start.com/ - Accelerating your Solr
>> proficiency
>>
>>
>> On Fri, Apr 4, 2014 at 3:41 PM, Sathya <[hidden email]<http://user/SendEmail.jtp?type=node&node=4129077&i=0>>
>> wrote:
>>
>> > Hi,
>> >
>> > I have attached my schema.xml file too.
>> >
>> > And you are right. I have 50 million documents. When i use solr
>> > browser to search a document, it will return within 1000 to 2000 ms.
>> >
>> > My query looks like this:
>> >
>> http://10.10.1.14:5050/solr/set_recent_shard1_replica5/select?q=subject&indent=true
>> >
>> > On 4/4/14, Alexandre Rafalovitch [via Lucene]
>> > <[hidden email] <http://user/SendEmail.jtp?type=node&node=4129077&i=1>>
>> wrote:
>> >>
>> >>
>> >> What does your Solr query looks like (check the Solr backend log if
>> >> you don't know)?
>> >>
>> >> And how many document is that? 50 million? Does not sound like much
>> >> for 3 fields. And what's the definitions (schema.xml rather than
>> >> solr.xml).
>> >>
>> >> And what happens if you issue the query directly to Solr rather than
>> >> through the client? Is the speed much different?
>> >>
>> >> Regards,
>> >> Alex.
>> >>
>> >> Personal website: http://www.outerthoughts.com/
>> >> Current project: http://www.solr-start.com/ - Accelerating your Solr
>> >> proficiency
>> >>
>> >>
>> >> On Fri, Apr 4, 2014 at 3:12 PM, Sathya <[hidden email]<http://user/SendEmail.jtp?type=node&node=4129077&i=2>>
>> wrote:
>> >>> Hi Alex,
>> >>>
>> >>> <str name="id">33026985</str> <str name="subject">Component Audio\:A
>> >>> Shopping List</str> <str name="download_date">2012-01-11
>> >>> 09:02:42.96</str>
>> >>>
>> >>> This is what i am indexed in solr. I have only 3 fields in index. And
>> >>> i am just indexing id, subject and date of the news articles. Nearly 5
>> >>> crore documents. Also i have attached my solrconfig and solr.xml file.
>> >>> If u need more information, pls let me know.
>> >>>
>> >>> On Fri, Apr 4, 2014 at 1:15 PM, Alexandre Rafalovitch [via Lucene]
>> >>> <[hidden email] <http://user/SendEmail.jtp?type=node&node=4129077&i=3>>
>> wrote:
>> >>>>
>> >>>> Show a sample query string that does that (takes 6 seconds to
>> return).
>> >>>> Including all defaults you may have put in solrconfig.xml (if any).
>> >>>> That might give us a hint which features you are using and what
>> >>>> possible direction you could go in next. For the bonus points, enable
>> >>>> debug flag and rows=1 parameter to see how big your documents
>> >>>> themselves are.
>> >>>>
>> >>>> You may have issues with a particular non-cloud-friendly feature,
>> with
>> >>>> caches, with not reusing parts of your queries as 'fq', returning too
>> >>>> many fields or a bunch of other things.
>> >>>>
>> >>>> Regards,
>> >>>> Alex.
>> >>>> Personal website: http://www.outerthoughts.com/
>> >>>> Current project: http://www.solr-start.com/ - Accelerating your Solr
>> >>>> proficiency
>> >>>>
>> >>>>
>> >>>> On Fri, Apr 4, 2014 at 2:31 PM, Sathya <[hidden email]> wrote:
>> >>>>
>> >>>> > Hi All,
>> >>>> >
>> >>>> > Hi All, I am new to Solr. And i dont know how to increase the
>> search
>> >>>> > speed
>> >>>> > of solrcloud. I have indexed nearly 4 GB of data. When i am
>> searching
>> >>>> > a
>> >>>> > document using java with solrj, solr takes more 6 seconds to return
>> a
>> >>>> > query
>> >>>> > result. Any one please help me to reduce the search query time to
>> less
>> >>>> > than
>> >>>> > 500 ms. i have allocate the 4 GB ram for solr. Please let me know
>> for
>> >>>> > further details about solrcloud config.
>> >>>> >
>> >>>> >
>> >>>> >
>> >>>> > --
>> >>>> > View this message in context:
>> >>>> >
>> http://lucene.472066.n3.nabble.com/How-to-reduce-the-search-speed-of-solrcloud-tp4129067.html
>> >>>> > Sent from the Solr - User mailing list archive at Nabble.com.
>> >>>>
>> >>>>
>> >>>> ________________________________
>> >>>> If you reply to this email, your message will be added to the
>> discussion
>> >>>> below:
>> >>>>
>> http://lucene.472066.n3.nabble.com/How-to-reduce-the-search-speed-of-solrcloud-tp4129067p4129068.html
>> >>>> To unsubscribe from How to reduce the search speed of solrcloud,
>> click
>> >>>> here.
>> >>>> NAML
>> >>>
>> >>>
>> >>> solrconfig.xml (101K)
>> >>> <
>> http://lucene.472066.n3.nabble.com/attachment/4129073/0/solrconfig.xml>
>> >>> solr.xml (1K)
>> >>> <http://lucene.472066.n3.nabble.com/attachment/4129073/1/solr.xml>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>> View this message in context:
>> >>>
>> http://lucene.472066.n3.nabble.com/How-to-reduce-the-search-speed-of-solrcloud-tp4129067p4129073.html
>> >>> Sent from the Solr - User mailing list archive at Nabble.com.
>> >>
>> >>
>> >>
>> >>
>> >> _______________________________________________
>> >> If you reply to this email, your message will be added to the
>> discussion
>> >> below:
>> >>
>> http://lucene.472066.n3.nabble.com/How-to-reduce-the-search-speed-of-solrcloud-tp4129067p4129074.html
>> >>
>> >> To unsubscribe from How to reduce the search speed of solrcloud, visit
>> >>
>>
>> >
>> > schema.xml (81K) <
>> http://lucene.472066.n3.nabble.com/attachment/4129075/0/schema.xml>
>> >
>> >
>> >
>> >
>> > --
>> > View this message in context:
>> http://lucene.472066.n3.nabble.com/How-to-reduce-the-search-speed-of-solrcloud-tp4129067p4129075.html
>>
>> > Sent from the Solr - User mailing list archive at Nabble.com.
>>
>>
>> ------------------------------
>> If you reply to this email, your message will be added to the discussion
>> below:
>>
>> http://lucene.472066.n3.nabble.com/How-to-reduce-the-search-speed-of-solrcloud-tp4129067p4129077.html
>> To unsubscribe from How to reduce the search speed of solrcloud, click
>> here<http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=4129067&code=c2F0aGlhLmJsYWNrc3RhckBnbWFpbC5jb218NDEyOTA2N3wtMjEyNDcwMTI5OA==>
>> .
>> NAML<http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>
>
>
>
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/How-to-reduce-the-search-speed-of-solrcloud-tp4129067p4129085.html
> Sent from the Solr - User mailing list archive at Nabble.com.
Re: How to reduce the search speed of solrcloud
Posted by Sathya <sa...@gmail.com>.
Hi,
Sorry, i cant get u alex. Can you please explain me(if you can). Because
now only i entered into solr.
On Fri, Apr 4, 2014 at 2:20 PM, Alexandre Rafalovitch [via Lucene] <
ml-node+s472066n4129077h62@n3.nabble.com> wrote:
> Well, if the direct browser query is 1000ms and your client query is
> 6seconds, then it is not Solr itself you need to worry about first.
> Something must be wrong at the client. Trying timing that bit. Maybe
> it is writing from the client to your ultimate consumer that's the
> problem.
>
> Regards,
> Alex.
> P.s. You should probably trim your schema to get rid of all the
> example fields. Keep _version_ and _root_ but delete all the rest you
> don't actually use. Same with dynamic fields and all fieldType
> definitions you do not actually use. You can always reintroduce them
> later from the example schemas if something is missing.
>
> Personal website: http://www.outerthoughts.com/
> Current project: http://www.solr-start.com/ - Accelerating your Solr
> proficiency
>
>
> On Fri, Apr 4, 2014 at 3:41 PM, Sathya <[hidden email]<http://user/SendEmail.jtp?type=node&node=4129077&i=0>>
> wrote:
>
> > Hi,
> >
> > I have attached my schema.xml file too.
> >
> > And you are right. I have 50 million documents. When i use solr
> > browser to search a document, it will return within 1000 to 2000 ms.
> >
> > My query looks like this:
> >
> http://10.10.1.14:5050/solr/set_recent_shard1_replica5/select?q=subject&indent=true
> >
> > On 4/4/14, Alexandre Rafalovitch [via Lucene]
> > <[hidden email] <http://user/SendEmail.jtp?type=node&node=4129077&i=1>>
> wrote:
> >>
> >>
> >> What does your Solr query looks like (check the Solr backend log if
> >> you don't know)?
> >>
> >> And how many document is that? 50 million? Does not sound like much
> >> for 3 fields. And what's the definitions (schema.xml rather than
> >> solr.xml).
> >>
> >> And what happens if you issue the query directly to Solr rather than
> >> through the client? Is the speed much different?
> >>
> >> Regards,
> >> Alex.
> >>
> >> Personal website: http://www.outerthoughts.com/
> >> Current project: http://www.solr-start.com/ - Accelerating your Solr
> >> proficiency
> >>
> >>
> >> On Fri, Apr 4, 2014 at 3:12 PM, Sathya <[hidden email]<http://user/SendEmail.jtp?type=node&node=4129077&i=2>>
> wrote:
> >>> Hi Alex,
> >>>
> >>> <str name="id">33026985</str> <str name="subject">Component Audio\:A
> >>> Shopping List</str> <str name="download_date">2012-01-11
> >>> 09:02:42.96</str>
> >>>
> >>> This is what i am indexed in solr. I have only 3 fields in index. And
> >>> i am just indexing id, subject and date of the news articles. Nearly 5
> >>> crore documents. Also i have attached my solrconfig and solr.xml file.
> >>> If u need more information, pls let me know.
> >>>
> >>> On Fri, Apr 4, 2014 at 1:15 PM, Alexandre Rafalovitch [via Lucene]
> >>> <[hidden email] <http://user/SendEmail.jtp?type=node&node=4129077&i=3>>
> wrote:
> >>>>
> >>>> Show a sample query string that does that (takes 6 seconds to
> return).
> >>>> Including all defaults you may have put in solrconfig.xml (if any).
> >>>> That might give us a hint which features you are using and what
> >>>> possible direction you could go in next. For the bonus points, enable
> >>>> debug flag and rows=1 parameter to see how big your documents
> >>>> themselves are.
> >>>>
> >>>> You may have issues with a particular non-cloud-friendly feature,
> with
> >>>> caches, with not reusing parts of your queries as 'fq', returning too
> >>>> many fields or a bunch of other things.
> >>>>
> >>>> Regards,
> >>>> Alex.
> >>>> Personal website: http://www.outerthoughts.com/
> >>>> Current project: http://www.solr-start.com/ - Accelerating your Solr
> >>>> proficiency
> >>>>
> >>>>
> >>>> On Fri, Apr 4, 2014 at 2:31 PM, Sathya <[hidden email]> wrote:
> >>>>
> >>>> > Hi All,
> >>>> >
> >>>> > Hi All, I am new to Solr. And i dont know how to increase the
> search
> >>>> > speed
> >>>> > of solrcloud. I have indexed nearly 4 GB of data. When i am
> searching
> >>>> > a
> >>>> > document using java with solrj, solr takes more 6 seconds to return
> a
> >>>> > query
> >>>> > result. Any one please help me to reduce the search query time to
> less
> >>>> > than
> >>>> > 500 ms. i have allocate the 4 GB ram for solr. Please let me know
> for
> >>>> > further details about solrcloud config.
> >>>> >
> >>>> >
> >>>> >
> >>>> > --
> >>>> > View this message in context:
> >>>> >
> http://lucene.472066.n3.nabble.com/How-to-reduce-the-search-speed-of-solrcloud-tp4129067.html
> >>>> > Sent from the Solr - User mailing list archive at Nabble.com.
> >>>>
> >>>>
> >>>> ________________________________
> >>>> If you reply to this email, your message will be added to the
> discussion
> >>>> below:
> >>>>
> http://lucene.472066.n3.nabble.com/How-to-reduce-the-search-speed-of-solrcloud-tp4129067p4129068.html
> >>>> To unsubscribe from How to reduce the search speed of solrcloud,
> click
> >>>> here.
> >>>> NAML
> >>>
> >>>
> >>> solrconfig.xml (101K)
> >>> <
> http://lucene.472066.n3.nabble.com/attachment/4129073/0/solrconfig.xml>
> >>> solr.xml (1K)
> >>> <http://lucene.472066.n3.nabble.com/attachment/4129073/1/solr.xml>
> >>>
> >>>
> >>>
> >>>
> >>> --
> >>> View this message in context:
> >>>
> http://lucene.472066.n3.nabble.com/How-to-reduce-the-search-speed-of-solrcloud-tp4129067p4129073.html
> >>> Sent from the Solr - User mailing list archive at Nabble.com.
> >>
> >>
> >>
> >>
> >> _______________________________________________
> >> If you reply to this email, your message will be added to the
> discussion
> >> below:
> >>
> http://lucene.472066.n3.nabble.com/How-to-reduce-the-search-speed-of-solrcloud-tp4129067p4129074.html
> >>
> >> To unsubscribe from How to reduce the search speed of solrcloud, visit
> >>
>
> >
> > schema.xml (81K) <
> http://lucene.472066.n3.nabble.com/attachment/4129075/0/schema.xml>
> >
> >
> >
> >
> > --
> > View this message in context:
> http://lucene.472066.n3.nabble.com/How-to-reduce-the-search-speed-of-solrcloud-tp4129067p4129075.html
>
> > Sent from the Solr - User mailing list archive at Nabble.com.
>
>
> ------------------------------
> If you reply to this email, your message will be added to the discussion
> below:
>
> http://lucene.472066.n3.nabble.com/How-to-reduce-the-search-speed-of-solrcloud-tp4129067p4129077.html
> To unsubscribe from How to reduce the search speed of solrcloud, click
> here<http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=4129067&code=c2F0aGlhLmJsYWNrc3RhckBnbWFpbC5jb218NDEyOTA2N3wtMjEyNDcwMTI5OA==>
> .
> NAML<http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>
--
View this message in context: http://lucene.472066.n3.nabble.com/How-to-reduce-the-search-speed-of-solrcloud-tp4129067p4129085.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: How to reduce the search speed of solrcloud
Posted by Alexandre Rafalovitch <ar...@gmail.com>.
Well, if the direct browser query is 1000ms and your client query is
6seconds, then it is not Solr itself you need to worry about first.
Something must be wrong at the client. Trying timing that bit. Maybe
it is writing from the client to your ultimate consumer that's the
problem.
Regards,
Alex.
P.s. You should probably trim your schema to get rid of all the
example fields. Keep _version_ and _root_ but delete all the rest you
don't actually use. Same with dynamic fields and all fieldType
definitions you do not actually use. You can always reintroduce them
later from the example schemas if something is missing.
Personal website: http://www.outerthoughts.com/
Current project: http://www.solr-start.com/ - Accelerating your Solr proficiency
On Fri, Apr 4, 2014 at 3:41 PM, Sathya <sa...@gmail.com> wrote:
> Hi,
>
> I have attached my schema.xml file too.
>
> And you are right. I have 50 million documents. When i use solr
> browser to search a document, it will return within 1000 to 2000 ms.
>
> My query looks like this:
> http://10.10.1.14:5050/solr/set_recent_shard1_replica5/select?q=subject&indent=true
>
> On 4/4/14, Alexandre Rafalovitch [via Lucene]
> <ml...@n3.nabble.com> wrote:
>>
>>
>> What does your Solr query looks like (check the Solr backend log if
>> you don't know)?
>>
>> And how many document is that? 50 million? Does not sound like much
>> for 3 fields. And what's the definitions (schema.xml rather than
>> solr.xml).
>>
>> And what happens if you issue the query directly to Solr rather than
>> through the client? Is the speed much different?
>>
>> Regards,
>> Alex.
>>
>> Personal website: http://www.outerthoughts.com/
>> Current project: http://www.solr-start.com/ - Accelerating your Solr
>> proficiency
>>
>>
>> On Fri, Apr 4, 2014 at 3:12 PM, Sathya <sa...@gmail.com> wrote:
>>> Hi Alex,
>>>
>>> <str name="id">33026985</str> <str name="subject">Component Audio\:A
>>> Shopping List</str> <str name="download_date">2012-01-11
>>> 09:02:42.96</str>
>>>
>>> This is what i am indexed in solr. I have only 3 fields in index. And
>>> i am just indexing id, subject and date of the news articles. Nearly 5
>>> crore documents. Also i have attached my solrconfig and solr.xml file.
>>> If u need more information, pls let me know.
>>>
>>> On Fri, Apr 4, 2014 at 1:15 PM, Alexandre Rafalovitch [via Lucene]
>>> <ml...@n3.nabble.com> wrote:
>>>>
>>>> Show a sample query string that does that (takes 6 seconds to return).
>>>> Including all defaults you may have put in solrconfig.xml (if any).
>>>> That might give us a hint which features you are using and what
>>>> possible direction you could go in next. For the bonus points, enable
>>>> debug flag and rows=1 parameter to see how big your documents
>>>> themselves are.
>>>>
>>>> You may have issues with a particular non-cloud-friendly feature, with
>>>> caches, with not reusing parts of your queries as 'fq', returning too
>>>> many fields or a bunch of other things.
>>>>
>>>> Regards,
>>>> Alex.
>>>> Personal website: http://www.outerthoughts.com/
>>>> Current project: http://www.solr-start.com/ - Accelerating your Solr
>>>> proficiency
>>>>
>>>>
>>>> On Fri, Apr 4, 2014 at 2:31 PM, Sathya <[hidden email]> wrote:
>>>>
>>>> > Hi All,
>>>> >
>>>> > Hi All, I am new to Solr. And i dont know how to increase the search
>>>> > speed
>>>> > of solrcloud. I have indexed nearly 4 GB of data. When i am searching
>>>> > a
>>>> > document using java with solrj, solr takes more 6 seconds to return a
>>>> > query
>>>> > result. Any one please help me to reduce the search query time to less
>>>> > than
>>>> > 500 ms. i have allocate the 4 GB ram for solr. Please let me know for
>>>> > further details about solrcloud config.
>>>> >
>>>> >
>>>> >
>>>> > --
>>>> > View this message in context:
>>>> > http://lucene.472066.n3.nabble.com/How-to-reduce-the-search-speed-of-solrcloud-tp4129067.html
>>>> > Sent from the Solr - User mailing list archive at Nabble.com.
>>>>
>>>>
>>>> ________________________________
>>>> If you reply to this email, your message will be added to the discussion
>>>> below:
>>>> http://lucene.472066.n3.nabble.com/How-to-reduce-the-search-speed-of-solrcloud-tp4129067p4129068.html
>>>> To unsubscribe from How to reduce the search speed of solrcloud, click
>>>> here.
>>>> NAML
>>>
>>>
>>> solrconfig.xml (101K)
>>> <http://lucene.472066.n3.nabble.com/attachment/4129073/0/solrconfig.xml>
>>> solr.xml (1K)
>>> <http://lucene.472066.n3.nabble.com/attachment/4129073/1/solr.xml>
>>>
>>>
>>>
>>>
>>> --
>>> View this message in context:
>>> http://lucene.472066.n3.nabble.com/How-to-reduce-the-search-speed-of-solrcloud-tp4129067p4129073.html
>>> Sent from the Solr - User mailing list archive at Nabble.com.
>>
>>
>>
>>
>> _______________________________________________
>> If you reply to this email, your message will be added to the discussion
>> below:
>> http://lucene.472066.n3.nabble.com/How-to-reduce-the-search-speed-of-solrcloud-tp4129067p4129074.html
>>
>> To unsubscribe from How to reduce the search speed of solrcloud, visit
>> http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=4129067&code=c2F0aGlhLmJsYWNrc3RhckBnbWFpbC5jb218NDEyOTA2N3wtMjEyNDcwMTI5OA==
>
>
> schema.xml (81K) <http://lucene.472066.n3.nabble.com/attachment/4129075/0/schema.xml>
>
>
>
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/How-to-reduce-the-search-speed-of-solrcloud-tp4129067p4129075.html
> Sent from the Solr - User mailing list archive at Nabble.com.
Re: How to reduce the search speed of solrcloud
Posted by Sathya <sa...@gmail.com>.
Hi,
I have attached my schema.xml file too.
And you are right. I have 50 million documents. When i use solr
browser to search a document, it will return within 1000 to 2000 ms.
My query looks like this:
http://10.10.1.14:5050/solr/set_recent_shard1_replica5/select?q=subject&indent=true
On 4/4/14, Alexandre Rafalovitch [via Lucene]
<ml...@n3.nabble.com> wrote:
>
>
> What does your Solr query looks like (check the Solr backend log if
> you don't know)?
>
> And how many document is that? 50 million? Does not sound like much
> for 3 fields. And what's the definitions (schema.xml rather than
> solr.xml).
>
> And what happens if you issue the query directly to Solr rather than
> through the client? Is the speed much different?
>
> Regards,
> Alex.
>
> Personal website: http://www.outerthoughts.com/
> Current project: http://www.solr-start.com/ - Accelerating your Solr
> proficiency
>
>
> On Fri, Apr 4, 2014 at 3:12 PM, Sathya <sa...@gmail.com> wrote:
>> Hi Alex,
>>
>> <str name="id">33026985</str> <str name="subject">Component Audio\:A
>> Shopping List</str> <str name="download_date">2012-01-11
>> 09:02:42.96</str>
>>
>> This is what i am indexed in solr. I have only 3 fields in index. And
>> i am just indexing id, subject and date of the news articles. Nearly 5
>> crore documents. Also i have attached my solrconfig and solr.xml file.
>> If u need more information, pls let me know.
>>
>> On Fri, Apr 4, 2014 at 1:15 PM, Alexandre Rafalovitch [via Lucene]
>> <ml...@n3.nabble.com> wrote:
>>>
>>> Show a sample query string that does that (takes 6 seconds to return).
>>> Including all defaults you may have put in solrconfig.xml (if any).
>>> That might give us a hint which features you are using and what
>>> possible direction you could go in next. For the bonus points, enable
>>> debug flag and rows=1 parameter to see how big your documents
>>> themselves are.
>>>
>>> You may have issues with a particular non-cloud-friendly feature, with
>>> caches, with not reusing parts of your queries as 'fq', returning too
>>> many fields or a bunch of other things.
>>>
>>> Regards,
>>> Alex.
>>> Personal website: http://www.outerthoughts.com/
>>> Current project: http://www.solr-start.com/ - Accelerating your Solr
>>> proficiency
>>>
>>>
>>> On Fri, Apr 4, 2014 at 2:31 PM, Sathya <[hidden email]> wrote:
>>>
>>> > Hi All,
>>> >
>>> > Hi All, I am new to Solr. And i dont know how to increase the search
>>> > speed
>>> > of solrcloud. I have indexed nearly 4 GB of data. When i am searching
>>> > a
>>> > document using java with solrj, solr takes more 6 seconds to return a
>>> > query
>>> > result. Any one please help me to reduce the search query time to less
>>> > than
>>> > 500 ms. i have allocate the 4 GB ram for solr. Please let me know for
>>> > further details about solrcloud config.
>>> >
>>> >
>>> >
>>> > --
>>> > View this message in context:
>>> > http://lucene.472066.n3.nabble.com/How-to-reduce-the-search-speed-of-solrcloud-tp4129067.html
>>> > Sent from the Solr - User mailing list archive at Nabble.com.
>>>
>>>
>>> ________________________________
>>> If you reply to this email, your message will be added to the discussion
>>> below:
>>> http://lucene.472066.n3.nabble.com/How-to-reduce-the-search-speed-of-solrcloud-tp4129067p4129068.html
>>> To unsubscribe from How to reduce the search speed of solrcloud, click
>>> here.
>>> NAML
>>
>>
>> solrconfig.xml (101K)
>> <http://lucene.472066.n3.nabble.com/attachment/4129073/0/solrconfig.xml>
>> solr.xml (1K)
>> <http://lucene.472066.n3.nabble.com/attachment/4129073/1/solr.xml>
>>
>>
>>
>>
>> --
>> View this message in context:
>> http://lucene.472066.n3.nabble.com/How-to-reduce-the-search-speed-of-solrcloud-tp4129067p4129073.html
>> Sent from the Solr - User mailing list archive at Nabble.com.
>
>
>
>
> _______________________________________________
> If you reply to this email, your message will be added to the discussion
> below:
> http://lucene.472066.n3.nabble.com/How-to-reduce-the-search-speed-of-solrcloud-tp4129067p4129074.html
>
> To unsubscribe from How to reduce the search speed of solrcloud, visit
> http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=4129067&code=c2F0aGlhLmJsYWNrc3RhckBnbWFpbC5jb218NDEyOTA2N3wtMjEyNDcwMTI5OA==
schema.xml (81K) <http://lucene.472066.n3.nabble.com/attachment/4129075/0/schema.xml>
--
View this message in context: http://lucene.472066.n3.nabble.com/How-to-reduce-the-search-speed-of-solrcloud-tp4129067p4129075.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: How to reduce the search speed of solrcloud
Posted by Alexandre Rafalovitch <ar...@gmail.com>.
What does your Solr query looks like (check the Solr backend log if
you don't know)?
And how many document is that? 50 million? Does not sound like much
for 3 fields. And what's the definitions (schema.xml rather than
solr.xml).
And what happens if you issue the query directly to Solr rather than
through the client? Is the speed much different?
Regards,
Alex.
Personal website: http://www.outerthoughts.com/
Current project: http://www.solr-start.com/ - Accelerating your Solr proficiency
On Fri, Apr 4, 2014 at 3:12 PM, Sathya <sa...@gmail.com> wrote:
> Hi Alex,
>
> <str name="id">33026985</str> <str name="subject">Component Audio\:A
> Shopping List</str> <str name="download_date">2012-01-11
> 09:02:42.96</str>
>
> This is what i am indexed in solr. I have only 3 fields in index. And
> i am just indexing id, subject and date of the news articles. Nearly 5
> crore documents. Also i have attached my solrconfig and solr.xml file.
> If u need more information, pls let me know.
>
> On Fri, Apr 4, 2014 at 1:15 PM, Alexandre Rafalovitch [via Lucene]
> <ml...@n3.nabble.com> wrote:
>>
>> Show a sample query string that does that (takes 6 seconds to return).
>> Including all defaults you may have put in solrconfig.xml (if any).
>> That might give us a hint which features you are using and what
>> possible direction you could go in next. For the bonus points, enable
>> debug flag and rows=1 parameter to see how big your documents
>> themselves are.
>>
>> You may have issues with a particular non-cloud-friendly feature, with
>> caches, with not reusing parts of your queries as 'fq', returning too
>> many fields or a bunch of other things.
>>
>> Regards,
>> Alex.
>> Personal website: http://www.outerthoughts.com/
>> Current project: http://www.solr-start.com/ - Accelerating your Solr proficiency
>>
>>
>> On Fri, Apr 4, 2014 at 2:31 PM, Sathya <[hidden email]> wrote:
>>
>> > Hi All,
>> >
>> > Hi All, I am new to Solr. And i dont know how to increase the search speed
>> > of solrcloud. I have indexed nearly 4 GB of data. When i am searching a
>> > document using java with solrj, solr takes more 6 seconds to return a query
>> > result. Any one please help me to reduce the search query time to less than
>> > 500 ms. i have allocate the 4 GB ram for solr. Please let me know for
>> > further details about solrcloud config.
>> >
>> >
>> >
>> > --
>> > View this message in context: http://lucene.472066.n3.nabble.com/How-to-reduce-the-search-speed-of-solrcloud-tp4129067.html
>> > Sent from the Solr - User mailing list archive at Nabble.com.
>>
>>
>> ________________________________
>> If you reply to this email, your message will be added to the discussion below:
>> http://lucene.472066.n3.nabble.com/How-to-reduce-the-search-speed-of-solrcloud-tp4129067p4129068.html
>> To unsubscribe from How to reduce the search speed of solrcloud, click here.
>> NAML
>
>
> solrconfig.xml (101K) <http://lucene.472066.n3.nabble.com/attachment/4129073/0/solrconfig.xml>
> solr.xml (1K) <http://lucene.472066.n3.nabble.com/attachment/4129073/1/solr.xml>
>
>
>
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/How-to-reduce-the-search-speed-of-solrcloud-tp4129067p4129073.html
> Sent from the Solr - User mailing list archive at Nabble.com.
Re: How to reduce the search speed of solrcloud
Posted by Sathya <sa...@gmail.com>.
Hi Alex,
<str name="id">33026985</str> <str name="subject">Component Audio\:A
Shopping List</str> <str name="download_date">2012-01-11
09:02:42.96</str>
This is what i am indexed in solr. I have only 3 fields in index. And
i am just indexing id, subject and date of the news articles. Nearly 5
crore documents. Also i have attached my solrconfig and solr.xml file.
If u need more information, pls let me know.
On Fri, Apr 4, 2014 at 1:15 PM, Alexandre Rafalovitch [via Lucene]
<ml...@n3.nabble.com> wrote:
>
> Show a sample query string that does that (takes 6 seconds to return).
> Including all defaults you may have put in solrconfig.xml (if any).
> That might give us a hint which features you are using and what
> possible direction you could go in next. For the bonus points, enable
> debug flag and rows=1 parameter to see how big your documents
> themselves are.
>
> You may have issues with a particular non-cloud-friendly feature, with
> caches, with not reusing parts of your queries as 'fq', returning too
> many fields or a bunch of other things.
>
> Regards,
> Alex.
> Personal website: http://www.outerthoughts.com/
> Current project: http://www.solr-start.com/ - Accelerating your Solr proficiency
>
>
> On Fri, Apr 4, 2014 at 2:31 PM, Sathya <[hidden email]> wrote:
>
> > Hi All,
> >
> > Hi All, I am new to Solr. And i dont know how to increase the search speed
> > of solrcloud. I have indexed nearly 4 GB of data. When i am searching a
> > document using java with solrj, solr takes more 6 seconds to return a query
> > result. Any one please help me to reduce the search query time to less than
> > 500 ms. i have allocate the 4 GB ram for solr. Please let me know for
> > further details about solrcloud config.
> >
> >
> >
> > --
> > View this message in context: http://lucene.472066.n3.nabble.com/How-to-reduce-the-search-speed-of-solrcloud-tp4129067.html
> > Sent from the Solr - User mailing list archive at Nabble.com.
>
>
> ________________________________
> If you reply to this email, your message will be added to the discussion below:
> http://lucene.472066.n3.nabble.com/How-to-reduce-the-search-speed-of-solrcloud-tp4129067p4129068.html
> To unsubscribe from How to reduce the search speed of solrcloud, click here.
> NAML
solrconfig.xml (101K) <http://lucene.472066.n3.nabble.com/attachment/4129073/0/solrconfig.xml>
solr.xml (1K) <http://lucene.472066.n3.nabble.com/attachment/4129073/1/solr.xml>
--
View this message in context: http://lucene.472066.n3.nabble.com/How-to-reduce-the-search-speed-of-solrcloud-tp4129067p4129073.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: How to reduce the search speed of solrcloud
Posted by Alexandre Rafalovitch <ar...@gmail.com>.
Show a sample query string that does that (takes 6 seconds to return).
Including all defaults you may have put in solrconfig.xml (if any).
That might give us a hint which features you are using and what
possible direction you could go in next. For the bonus points, enable
debug flag and rows=1 parameter to see how big your documents
themselves are.
You may have issues with a particular non-cloud-friendly feature, with
caches, with not reusing parts of your queries as 'fq', returning too
many fields or a bunch of other things.
Regards,
Alex.
Personal website: http://www.outerthoughts.com/
Current project: http://www.solr-start.com/ - Accelerating your Solr proficiency
On Fri, Apr 4, 2014 at 2:31 PM, Sathya <sa...@gmail.com> wrote:
> Hi All,
>
> Hi All, I am new to Solr. And i dont know how to increase the search speed
> of solrcloud. I have indexed nearly 4 GB of data. When i am searching a
> document using java with solrj, solr takes more 6 seconds to return a query
> result. Any one please help me to reduce the search query time to less than
> 500 ms. i have allocate the 4 GB ram for solr. Please let me know for
> further details about solrcloud config.
>
>
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/How-to-reduce-the-search-speed-of-solrcloud-tp4129067.html
> Sent from the Solr - User mailing list archive at Nabble.com.