Re: Solr cloud production set up

[Paras Lehana <pa...@indiamart.com>]

Hi Rajdeep,


   1. I assume you had enabled docValues for the facet fields, right?
   2. What does your GC logs tell? Do you get freezes and CPU spikes during
   intervals?
   3. Caching will help in querying. I'll need to see a sample query of
   yours to recommend what you can tweak.


On Tue, 28 Jan 2020 at 19:09, Jason Gerlowski <ge...@gmail.com> wrote:

> Hi Rajdeep,
>
> Unfortunately it's near impossible for anyone here to tell you what
> parameters to tweak.  People might take guesses based on their
> individual past experience, but ultimately those are just guesses.
>
> There are just too many variables affecting Solr performance for
> anyone to have a good guess without access to the cluster itself and
> the time and will to dig into it.
>
> Are there GC params that need tweaking?  Very possible, but you'll
> have to look into your gc logs to see how much time is being spent in
> gc.  Are there query params you could be changing?  Very possible, but
> you'll have to identify the types of queries you're submitting and see
> whether the ref-guide offers any information on how to tweak
> performance for those particular qparsers, facets, etc.  Is the number
> of facets the reason for slow queries?  Very possible, but you'll have
> to turn faceting off or run debug=timing and see how what that tells
> you about the QTime's.
>
> Tuning Solr performance is a tough, time consuming process.  I wish
> there was an easier answer for you, but there's not.
>
> Best,
>
> Jason
>
> On Mon, Jan 20, 2020 at 12:06 PM Rajdeep Sahoo
> <ra...@gmail.com> wrote:
> >
> > Please suggest anyone
> >
> > On Sun, 19 Jan, 2020, 9:43 AM Rajdeep Sahoo, <rajdeepsahoo2012@gmail.com
> >
> > wrote:
> >
> > > Apart from reducing no of facets in the query, is there any other query
> > > params or gc params or heap space or anything else that we need to
> tweak
> > > for improving search response time.
> > >
> > > On Sun, 19 Jan, 2020, 3:15 AM Erick Erickson, <erickerickson@gmail.com
> >
> > > wrote:
> > >
> > >> Add &debug=timing to the query and it’ll show you the time each
> component
> > >> takes.
> > >>
> > >> > On Jan 18, 2020, at 1:50 PM, Rajdeep Sahoo <
> rajdeepsahoo2012@gmail.com>
> > >> wrote:
> > >> >
> > >> > Thanks for the suggestion,
> > >> >
> > >> > Is there any way to get the info which operation or which query
> params
> > >> are
> > >> > increasing the response time.
> > >> >
> > >> >
> > >> > On Sat, 18 Jan, 2020, 11:59 PM Dave, <ha...@gmail.com>
> > >> wrote:
> > >> >
> > >> >> If you’re not getting values, don’t ask for the facet. Facets are
> > >> >> expensive as hell, maybe you should think more about your query’s
> than
> > >> your
> > >> >> infrastructure, solr cloud won’t help you at all especially if your
> > >> asking
> > >> >> for things you don’t need
> > >> >>
> > >> >>> On Jan 18, 2020, at 1:25 PM, Rajdeep Sahoo <
> > >> rajdeepsahoo2012@gmail.com>
> > >> >> wrote:
> > >> >>>
> > >> >>> We have assigned 16 gb out of 24gb for heap .
> > >> >>> No other process is running on that node.
> > >> >>>
> > >> >>> 200 facets fields are there in the query but we will not be
> getting
> > >> the
> > >> >>> values for each facets for every search.
> > >> >>> There can be max of 50-60 facets for which we will be getting
> values.
> > >> >>>
> > >> >>> We are using caching,is it not going to help.
> > >> >>>
> > >> >>>
> > >> >>>
> > >> >>>> On Sat, 18 Jan, 2020, 11:36 PM Shawn Heisey, <
> apache@elyograg.org>
> > >> >> wrote:
> > >> >>>>
> > >> >>>>> On 1/18/2020 10:09 AM, Rajdeep Sahoo wrote:
> > >> >>>>> We are having 2.3 million documents and size is 2.5 gb.
> > >> >>>>>  10 core cpu and 24 gb ram . 16 slave nodes.
> > >> >>>>>
> > >> >>>>>  Still some of the queries are taking 50 sec at solr end.
> > >> >>>>> As we are using solr 4.6 .
> > >> >>>>>  Other thing is we are having 200 (avg) facet fields  in a
> query.
> > >> >>>>> And 30 searchable fields.
> > >> >>>>> Is there any way to identify why it is taking 50 sec for a
> query.
> > >> >>>>>    Multiple concurrent requests are there.
> > >> >>>>
> > >> >>>> Searching 30 fields and computing 200 facets is never going to be
> > >> super
> > >> >>>> fast.  Switching to cloud will not help, and might make it
> slower.
> > >> >>>>
> > >> >>>> Your index is pretty small to a lot of us.  There are people
> running
> > >> >>>> indexes with billions of documents that take terabytes of disk
> space.
> > >> >>>>
> > >> >>>> As Walter mentioned, computing 200 facets is going to require a
> fair
> > >> >>>> amount of heap memory.  One *possible* problem here is that the
> Solr
> > >> >>>> heap size is too small, so a lot of GC is required.  How much of
> the
> > >> >>>> 24GB have you assigned to the heap?  Is there any software other
> than
> > >> >>>> Solr running on these nodes?
> > >> >>>>
> > >> >>>> Thanks,
> > >> >>>> Shawn
> > >> >>>>
> > >> >>
> > >>
> > >>
>


-- 
-- 
Regards,

*Paras Lehana* [65871]
Development Engineer, *Auto-Suggest*,
IndiaMART InterMESH Ltd,

11th Floor, Tower 2, Assotech Business Cresterra,
Plot No. 22, Sector 135, Noida, Uttar Pradesh, India 201305

Mob.: +91-9560911996
Work: 0120-4056700 | Extn:
*11096*

-- 
*
*

 <https://www.facebook.com/IndiaMART/videos/578196442936091/>