You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Arturas Mazeika <ma...@gmail.com> on 2020/12/03 08:22:21 UTC
increasing number of threads for faceting in JSON format
Hi Solr-Users,
I am trying to better understand the solr capabilities, how one can
formulate queries in JSON format as well as tweak parameters. Currently I
have a logs collection (ca 6GB large) with a dozen of attributes running in
single server mode (F:\solr_deployment\solr-8.7.0\bin\solr.cmd start -h
localhost -p 5555 -m 4g)
I am playing with faceting functionality in solr and query a couple of
attributes there. My typical query is:
GET http://localhost:5555/solr/db/query
<http://arteika:5555/solr/logan/query> HTTP/1.1
content-type: application/json
{
"query" : "*:*",
"limit" : 0,
"facet": {
"t" : {
"type": "terms",
"field": "fcomp",
"sort": "index",
"facet": {
"t_buckets": {
"type": "range",
"field": "t",
"sort": { "t": "asc" },
"start": "2018-05-02T17:00:00.000Z",
"end": "2020-11-16T21:00:00.000Z",
"gap": "+1HOUR"
}
}
},
}
}
not surprisingly, it takes a bit to compute the result, so I tried to
increase the number of threads. How do I do it in JSON format? I tried
adding
{
"params": {
"facet.threads": 8
},
"query" : "*:*",
...
}
and checked the jstack <pid> of the solr java process, but I still see only
one thread working. Can I configure params through the params section?
I also tried
{
"query" : "*:*",
"limit" : 0,
"facet": {
"t" : {
"type": "terms",
"field": "fcomp",
"sort": "index",
"facet": {
"t_buckets": {
"type": "range",
"field": "t",
"sort": { "t": "asc" },
"start": "2018-05-02T17:00:00.000Z",
"end": "2020-11-16T21:00:00.000Z",
"gap": "+1HOUR"
}
},
"threads":8
},
}
}
but this ran in one thread as well. Can I influence the number of threads
in the "facet" section of JSON?
Cheers,
Arturas
Re: increasing number of threads for faceting in JSON format
Posted by Arturas Mazeika <ma...@gmail.com>.
Hi Munedra,
This is great that I can get things faster by reducing the gap and by
increasing the number of threads. How to reduce gaps I know: one can
replace "gap": "+1HOUR" with "gap": "+1MONTH" What should I change
in the text below to increase the number of threads from one to 20?
Cheers,
Arturas
On Thu, Dec 3, 2020 at 1:54 PM Munendra S N <sn...@gmail.com> wrote:
> Hi,
>
> Currently, JSON facets have support for specifying the number of threads.
> In the above request, the range facet is computed over 2 years with a gap
> of 1 hour. By reducing the number of buckets, computation should become
> much faster
>
> Regards,
> Munendra S N
>
>
>
> On Thu, Dec 3, 2020 at 1:52 PM Arturas Mazeika <ma...@gmail.com> wrote:
>
> > Hi Solr-Users,
> >
> > I am trying to better understand the solr capabilities, how one can
> > formulate queries in JSON format as well as tweak parameters. Currently I
> > have a logs collection (ca 6GB large) with a dozen of attributes running
> in
> > single server mode (F:\solr_deployment\solr-8.7.0\bin\solr.cmd start -h
> > localhost -p 5555 -m 4g)
> >
> > I am playing with faceting functionality in solr and query a couple of
> > attributes there. My typical query is:
> >
> > GET http://localhost:5555/solr/db/query
> > <http://arteika:5555/solr/logan/query> HTTP/1.1
> > content-type: application/json
> >
> > {
> > "query" : "*:*",
> > "limit" : 0,
> > "facet": {
> > "t" : {
> > "type": "terms",
> > "field": "fcomp",
> > "sort": "index",
> >
> > "facet": {
> > "t_buckets": {
> > "type": "range",
> > "field": "t",
> > "sort": { "t": "asc" },
> > "start": "2018-05-02T17:00:00.000Z",
> > "end": "2020-11-16T21:00:00.000Z",
> > "gap": "+1HOUR"
> > }
> > }
> > },
> > }
> > }
> >
> > not surprisingly, it takes a bit to compute the result, so I tried to
> > increase the number of threads. How do I do it in JSON format? I tried
> > adding
> >
> > {
> > "params": {
> > "facet.threads": 8
> > },
> > "query" : "*:*",
> > ...
> > }
> >
> > and checked the jstack <pid> of the solr java process, but I still see
> only
> > one thread working. Can I configure params through the params section?
> >
> > I also tried
> >
> > {
> > "query" : "*:*",
> > "limit" : 0,
> > "facet": {
> > "t" : {
> > "type": "terms",
> > "field": "fcomp",
> > "sort": "index",
> >
> > "facet": {
> > "t_buckets": {
> > "type": "range",
> > "field": "t",
> > "sort": { "t": "asc" },
> > "start": "2018-05-02T17:00:00.000Z",
> > "end": "2020-11-16T21:00:00.000Z",
> > "gap": "+1HOUR"
> > }
> > },
> > "threads":8
> > },
> > }
> > }
> >
> > but this ran in one thread as well. Can I influence the number of threads
> > in the "facet" section of JSON?
> >
> > Cheers,
> > Arturas
> >
>
Re: increasing number of threads for faceting in JSON format
Posted by Munendra S N <sn...@gmail.com>.
Hi,
Currently, JSON facets have support for specifying the number of threads.
In the above request, the range facet is computed over 2 years with a gap
of 1 hour. By reducing the number of buckets, computation should become
much faster
Regards,
Munendra S N
On Thu, Dec 3, 2020 at 1:52 PM Arturas Mazeika <ma...@gmail.com> wrote:
> Hi Solr-Users,
>
> I am trying to better understand the solr capabilities, how one can
> formulate queries in JSON format as well as tweak parameters. Currently I
> have a logs collection (ca 6GB large) with a dozen of attributes running in
> single server mode (F:\solr_deployment\solr-8.7.0\bin\solr.cmd start -h
> localhost -p 5555 -m 4g)
>
> I am playing with faceting functionality in solr and query a couple of
> attributes there. My typical query is:
>
> GET http://localhost:5555/solr/db/query
> <http://arteika:5555/solr/logan/query> HTTP/1.1
> content-type: application/json
>
> {
> "query" : "*:*",
> "limit" : 0,
> "facet": {
> "t" : {
> "type": "terms",
> "field": "fcomp",
> "sort": "index",
>
> "facet": {
> "t_buckets": {
> "type": "range",
> "field": "t",
> "sort": { "t": "asc" },
> "start": "2018-05-02T17:00:00.000Z",
> "end": "2020-11-16T21:00:00.000Z",
> "gap": "+1HOUR"
> }
> }
> },
> }
> }
>
> not surprisingly, it takes a bit to compute the result, so I tried to
> increase the number of threads. How do I do it in JSON format? I tried
> adding
>
> {
> "params": {
> "facet.threads": 8
> },
> "query" : "*:*",
> ...
> }
>
> and checked the jstack <pid> of the solr java process, but I still see only
> one thread working. Can I configure params through the params section?
>
> I also tried
>
> {
> "query" : "*:*",
> "limit" : 0,
> "facet": {
> "t" : {
> "type": "terms",
> "field": "fcomp",
> "sort": "index",
>
> "facet": {
> "t_buckets": {
> "type": "range",
> "field": "t",
> "sort": { "t": "asc" },
> "start": "2018-05-02T17:00:00.000Z",
> "end": "2020-11-16T21:00:00.000Z",
> "gap": "+1HOUR"
> }
> },
> "threads":8
> },
> }
> }
>
> but this ran in one thread as well. Can I influence the number of threads
> in the "facet" section of JSON?
>
> Cheers,
> Arturas
>