You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Rahul Goswami <ra...@gmail.com> on 2020/04/20 18:35:16 UTC
Solr filter cache hits not reflecting
Hello,
I was trying to analyze the filter cache performance and noticed a strange
thing. Upon searching with fq, the entry gets added to the cache the first
time. Observing from the "Stats/Plugins" tab on Solr admin UI, the 'lookup'
and 'inserts' count gets incremented.
However, if I search with the same fq again, I expect the lookup and hits
count to increase, but it doesn't. This ultimately results in an incorrect
hitratio.
I tried this scenario on Solr 7.2.1, 7.7.2 and 8.5 and observe the same
behavior on all three versions.
Is this a bug or am I missing something here?
Thanks,
Rahul
Re: Solr filter cache hits not reflecting
Posted by Rahul Goswami <ra...@gmail.com>.
Hoss,
Thank you for such a succinct explanation! I was not aware of the order of
lookups (queryResultCache followed by filterCache). Makes sense now. Sorry
for the false alarm!
Rahul
On Mon, Apr 20, 2020 at 4:04 PM Chris Hostetter <ho...@fucit.org>
wrote:
> : 4) A query with different fq.
> :
> http://localhost:8984/solr/techproducts/select?q=popularity:[5%20TO%2012]&fq=manu:samsung
> ...
> : 5) A query with the same fq again (fq=manu:samsung OR manu:apple)....the
> : numbers don't get update for this fq hereafter for subsequent searches
> :
> :
> http://localhost:8984/solr/techproducts/select?q=popularity:[5%20TO%2012]&fq=manu:samsung%20OR%20manu:apple
>
> that's not just *A* query with the same fq, it's the *exact* same request
> (q + sort + pagination + all filters)
>
> Whch means that everything solr needs to reply to this request is
> available in the *queryResultCache* -- no filterCache needed at all (if
> you had faceting enabled that would be a different issue: then the
> filterCache would still be needed in order to compute facet counts over
> the entire DocSet matching the query, not just the current page window)...
>
>
> $ bin/solr -e techproducts
> ...
>
> # mostly empty caches (techproudct has a single static warming query)
>
> $ curl -sS '
> http://localhost:8983/solr/techproducts/admin/mbeans?wt=json&indent=true&category=CACHE&stats=true'
> | grep -E
> 'CACHE.searcher.(queryResultCache|filterCache).(inserts|hits|lookups)'
> "CACHE.searcher.queryResultCache.lookups":0,
> "CACHE.searcher.queryResultCache.inserts":1,
> "CACHE.searcher.queryResultCache.hits":0}},
> "CACHE.searcher.filterCache.hits":0,
> "CACHE.searcher.filterCache.lookups":0,
> "CACHE.searcher.filterCache.inserts":0,
>
> # new q and fq: lookup & insert into both caches...
>
> $ curl -sS '
> http://localhost:8983/solr/techproducts/select?q=popularity:[5%20TO%2012]&fq=manu:samsung%20OR%20manu:apple'
> > /dev/null
> $ curl -sS '
> http://localhost:8983/solr/techproducts/admin/mbeans?wt=json&indent=true&category=CACHE&stats=true'
> | grep -E
> 'CACHE.searcher.(queryResultCache|filterCache).(inserts|hits|lookups)'
> "CACHE.searcher.queryResultCache.lookups":1,
> "CACHE.searcher.queryResultCache.inserts":2,
> "CACHE.searcher.queryResultCache.hits":0}},
> "CACHE.searcher.filterCache.hits":0,
> "CACHE.searcher.filterCache.lookups":1,
> "CACHE.searcher.filterCache.inserts":1,
>
> # new q, same fq:
> # lookup on both caches, hit on filter, insert on queryResultCache
>
> $ curl -sS '
> http://localhost:8983/solr/techproducts/select?q=*:*&fq=manu:samsung%20OR%20manu:apple'
> > /dev/null
> $ curl -sS '
> http://localhost:8983/solr/techproducts/admin/mbeans?wt=json&indent=true&category=CACHE&stats=true'
> | grep -E
> 'CACHE.searcher.(queryResultCache|filterCache).(inserts|hits|lookups)'
> "CACHE.searcher.queryResultCache.lookups":2,
> "CACHE.searcher.queryResultCache.inserts":3,
> "CACHE.searcher.queryResultCache.hits":0}},
> "CACHE.searcher.filterCache.hits":1,
> "CACHE.searcher.filterCache.lookups":2,
> "CACHE.searcher.filterCache.inserts":1,
>
> # same q & fq as before:
> # hit on queryresultCache means no filterCache needed...
>
> $ curl -sS '
> http://localhost:8983/solr/techproducts/select?q=popularity:[5%20TO%2012]&fq=manu:samsung%20OR%20manu:apple'
> > /dev/null
> $ curl -sS '
> http://localhost:8983/solr/techproducts/admin/mbeans?wt=json&indent=true&category=CACHE&stats=true'
> | grep -E
> 'CACHE.searcher.(queryResultCache|filterCache).(inserts|hits|lookups)'
> "CACHE.searcher.queryResultCache.lookups":3,
> "CACHE.searcher.queryResultCache.inserts":3,
> "CACHE.searcher.queryResultCache.hits":1}},
> "CACHE.searcher.filterCache.hits":1,
> "CACHE.searcher.filterCache.lookups":2,
> "CACHE.searcher.filterCache.inserts":1,
>
>
>
> -Hoss
> http://www.lucidworks.com/
>
Re: Solr filter cache hits not reflecting
Posted by Chris Hostetter <ho...@fucit.org>.
: 4) A query with different fq.
: http://localhost:8984/solr/techproducts/select?q=popularity:[5%20TO%2012]&fq=manu:samsung
...
: 5) A query with the same fq again (fq=manu:samsung OR manu:apple)....the
: numbers don't get update for this fq hereafter for subsequent searches
:
: http://localhost:8984/solr/techproducts/select?q=popularity:[5%20TO%2012]&fq=manu:samsung%20OR%20manu:apple
that's not just *A* query with the same fq, it's the *exact* same request
(q + sort + pagination + all filters)
Whch means that everything solr needs to reply to this request is
available in the *queryResultCache* -- no filterCache needed at all (if
you had faceting enabled that would be a different issue: then the
filterCache would still be needed in order to compute facet counts over
the entire DocSet matching the query, not just the current page window)...
$ bin/solr -e techproducts
...
# mostly empty caches (techproudct has a single static warming query)
$ curl -sS 'http://localhost:8983/solr/techproducts/admin/mbeans?wt=json&indent=true&category=CACHE&stats=true' | grep -E 'CACHE.searcher.(queryResultCache|filterCache).(inserts|hits|lookups)'
"CACHE.searcher.queryResultCache.lookups":0,
"CACHE.searcher.queryResultCache.inserts":1,
"CACHE.searcher.queryResultCache.hits":0}},
"CACHE.searcher.filterCache.hits":0,
"CACHE.searcher.filterCache.lookups":0,
"CACHE.searcher.filterCache.inserts":0,
# new q and fq: lookup & insert into both caches...
$ curl -sS 'http://localhost:8983/solr/techproducts/select?q=popularity:[5%20TO%2012]&fq=manu:samsung%20OR%20manu:apple' > /dev/null
$ curl -sS 'http://localhost:8983/solr/techproducts/admin/mbeans?wt=json&indent=true&category=CACHE&stats=true' | grep -E
'CACHE.searcher.(queryResultCache|filterCache).(inserts|hits|lookups)'
"CACHE.searcher.queryResultCache.lookups":1,
"CACHE.searcher.queryResultCache.inserts":2,
"CACHE.searcher.queryResultCache.hits":0}},
"CACHE.searcher.filterCache.hits":0,
"CACHE.searcher.filterCache.lookups":1,
"CACHE.searcher.filterCache.inserts":1,
# new q, same fq:
# lookup on both caches, hit on filter, insert on queryResultCache
$ curl -sS 'http://localhost:8983/solr/techproducts/select?q=*:*&fq=manu:samsung%20OR%20manu:apple' > /dev/null
$ curl -sS 'http://localhost:8983/solr/techproducts/admin/mbeans?wt=json&indent=true&category=CACHE&stats=true' | grep -E
'CACHE.searcher.(queryResultCache|filterCache).(inserts|hits|lookups)'
"CACHE.searcher.queryResultCache.lookups":2,
"CACHE.searcher.queryResultCache.inserts":3,
"CACHE.searcher.queryResultCache.hits":0}},
"CACHE.searcher.filterCache.hits":1,
"CACHE.searcher.filterCache.lookups":2,
"CACHE.searcher.filterCache.inserts":1,
# same q & fq as before:
# hit on queryresultCache means no filterCache needed...
$ curl -sS 'http://localhost:8983/solr/techproducts/select?q=popularity:[5%20TO%2012]&fq=manu:samsung%20OR%20manu:apple' > /dev/null
$ curl -sS 'http://localhost:8983/solr/techproducts/admin/mbeans?wt=json&indent=true&category=CACHE&stats=true' | grep -E
'CACHE.searcher.(queryResultCache|filterCache).(inserts|hits|lookups)'
"CACHE.searcher.queryResultCache.lookups":3,
"CACHE.searcher.queryResultCache.inserts":3,
"CACHE.searcher.queryResultCache.hits":1}},
"CACHE.searcher.filterCache.hits":1,
"CACHE.searcher.filterCache.lookups":2,
"CACHE.searcher.filterCache.inserts":1,
-Hoss
http://www.lucidworks.com/
Re: Solr filter cache hits not reflecting
Posted by Rahul Goswami <ra...@gmail.com>.
Hi Hoss,
Thanks for your detailed response. In your steps if you go a step
further and search again with the same fq, you should be able to
uncover the problem. Here are the step-by-step observations on Solr
8.5 (7.2.1 and 7.7.2 have the same issue)
1) Before any queries:
http://localhost:8984/solr/admin/metrics?group=core&prefix=CACHE.searcher.filterCache
"solr.core.techproducts":{
"CACHE.searcher.filterCache":{
"lookups":0,
"idleEvictions":0,
"evictions":0,
"cumulative_inserts":0,
"ramBytesUsed":1328,
"cumulative_hits":0,
"cumulative_idleEvictions":0,
"hits":0,
"cumulative_evictions":0,
"cleanupThread":false,
"size":0,
"hitratio":0.0,
"cumulative_lookups":0,
"cumulative_hitratio":0.0,
"warmupTime":0,
"maxRamMB":-1,
"inserts":0}},
2) With fq:manu:samsung OR manu:apple
http://localhost:8984/solr/techproducts/select?q=*:*&fq=manu:samsung%20OR%20manu:apple
"solr.core.techproducts":{
"CACHE.searcher.filterCache":{
"lookups":1,
"idleEvictions":0,
"evictions":0,
"cumulative_inserts":1,
"ramBytesUsed":4800,
"cumulative_hits":0,
"cumulative_idleEvictions":0,
"hits":0,
"cumulative_evictions":0,
"cleanupThread":false,
"size":1,
"hitratio":0.0,
"cumulative_lookups":1,
"cumulative_hitratio":0.0,
"item_manu:samsung
manu:apple":"SortedIntDocSet{size=2,ramUsed=40 bytes}",
"warmupTime":0,
"maxRamMB":-1,
"inserts":1}},
3) q changed but same fq... the hits and lookups are updated as expected:
http://localhost:8984/solr/techproducts/select?q=popularity:[5%20TO%2012]&fq=manu:samsung%20OR%20manu:apple
"solr.core.techproducts":{
"CACHE.searcher.filterCache":{
"lookups":2,
"idleEvictions":0,
"evictions":0,
"cumulative_inserts":1,
"ramBytesUsed":4800,
"cumulative_hits":1,
"cumulative_idleEvictions":0,
"hits":1,
"cumulative_evictions":0,
"cleanupThread":false,
"size":1,
"hitratio":0.5,
"cumulative_lookups":2,
"cumulative_hitratio":0.5,
"item_manu:samsung
manu:apple":"SortedIntDocSet{size=2,ramUsed=40 bytes}",
"warmupTime":0,
"maxRamMB":-1,
"inserts":1}},
4) A query with different fq.
http://localhost:8984/solr/techproducts/select?q=popularity:[5%20TO%2012]&fq=manu:samsung
"solr.core.techproducts":{
"CACHE.searcher.filterCache":{
"lookups":3,
"idleEvictions":0,
"evictions":0,
"cumulative_inserts":2,
"ramBytesUsed":6076,
"cumulative_hits":1,
"cumulative_idleEvictions":0,
"hits":1,
"cumulative_evictions":0,
"cleanupThread":false,
"size":2,
"item_manu:samsung":"SortedIntDocSet{size=1,ramUsed=36 bytes}",
"hitratio":0.33,
"cumulative_lookups":3,
"cumulative_hitratio":0.33,
"item_manu:samsung
manu:apple":"SortedIntDocSet{size=2,ramUsed=40 bytes}",
"warmupTime":0,
"maxRamMB":-1,
5) A query with the same fq again (fq=manu:samsung OR manu:apple)....the
numbers don't get update for this fq hereafter for subsequent searches
http://localhost:8984/solr/techproducts/select?q=popularity:[5%20TO%2012]&fq=manu:samsung%20OR%20manu:apple
"solr.core.techproducts":{
"CACHE.searcher.filterCache":{
"lookups":3,
"idleEvictions":0,
"evictions":0,
"cumulative_inserts":2,
"ramBytesUsed":6076,
"cumulative_hits":1,
"cumulative_idleEvictions":0,
"hits":1,
"cumulative_evictions":0,
"cleanupThread":false,
"size":2,
"item_manu:samsung":"SortedIntDocSet{size=1,ramUsed=36 bytes}",
"hitratio":0.33,
"cumulative_lookups":3,
"cumulative_hitratio":0.33,
"item_manu:samsung
manu:apple":"SortedIntDocSet{size=2,ramUsed=40 bytes}",
"warmupTime":0,
"maxRamMB":-1,
"inserts":2}},
Thanks,
Rahul
On Mon, Apr 20, 2020 at 2:48 PM Chris Hostetter <ho...@fucit.org>
wrote:
>
> : I was trying to analyze the filter cache performance and noticed a
> strange
> : thing. Upon searching with fq, the entry gets added to the cache the
> first
> : time. Observing from the "Stats/Plugins" tab on Solr admin UI, the
> 'lookup'
> : and 'inserts' count gets incremented.
> : However, if I search with the same fq again, I expect the lookup and hits
> : count to increase, but it doesn't. This ultimately results in an
> incorrect
> : hitratio.
>
> We'll need to see the actual specifics of the requests you're executing &
> stats you're seeing in order to make any guesses as to why you're not
> seeing the expected outcome.
>
> Wild guesses:
> - Are you use Date math based fq params that don't round?
> - Are you using SolrCloud and some of your requests are getting routed to
> different replicas?
> - Are you using some complex/custom filter impl that may have a bug in
> it's equals/hashCode impl that prevents it from being a cache hit?
>
>
> Here's an example showing that the basics of filterCache work find with
> 8.5 for trivial examples...
>
> $ bin/solr -e techproducts
> ...
> $ curl -sS '
> http://localhost:8983/solr/techproducts/admin/mbeans?wt=json&indent=true&category=CACHE&stats=true&key=filterCache'
> | grep 'CACHE.searcher.filterCache'
> "CACHE.searcher.filterCache.hits":0,
> "CACHE.searcher.filterCache.cumulative_evictions":0,
> "CACHE.searcher.filterCache.cleanupThread":false,
> "CACHE.searcher.filterCache.size":0,
> "CACHE.searcher.filterCache.maxRamMB":-1,
> "CACHE.searcher.filterCache.hitratio":0.0,
> "CACHE.searcher.filterCache.warmupTime":0,
> "CACHE.searcher.filterCache.idleEvictions":0,
> "CACHE.searcher.filterCache.evictions":0,
> "CACHE.searcher.filterCache.cumulative_hitratio":0.0,
> "CACHE.searcher.filterCache.lookups":0,
> "CACHE.searcher.filterCache.cumulative_hits":0,
> "CACHE.searcher.filterCache.cumulative_inserts":0,
> "CACHE.searcher.filterCache.ramBytesUsed":1328,
> "CACHE.searcher.filterCache.cumulative_idleEvictions":0,
> "CACHE.searcher.filterCache.inserts":0,
> "CACHE.searcher.filterCache.cumulative_lookups":0}}},
> $ curl -sS '
> http://localhost:8983/solr/techproducts/query?q=*:*&fq=inStock=true' >
> /dev/null
> $ curl -sS '
> http://localhost:8983/solr/techproducts/admin/mbeans?wt=json&indent=true&category=CACHE&stats=true&key=filterCache'
> | grep 'CACHE.searcher.filterCache'
> "CACHE.searcher.filterCache.hits":0,
> "CACHE.searcher.filterCache.cumulative_evictions":0,
> "CACHE.searcher.filterCache.cleanupThread":false,
> "CACHE.searcher.filterCache.size":1,
> "CACHE.searcher.filterCache.maxRamMB":-1,
> "CACHE.searcher.filterCache.hitratio":0.0,
> "CACHE.searcher.filterCache.warmupTime":0,
> "CACHE.searcher.filterCache.idleEvictions":0,
> "CACHE.searcher.filterCache.evictions":0,
> "CACHE.searcher.filterCache.cumulative_hitratio":0.0,
> "CACHE.searcher.filterCache.lookups":1,
> "CACHE.searcher.filterCache.cumulative_hits":0,
> "CACHE.searcher.filterCache.cumulative_inserts":1,
> "CACHE.searcher.filterCache.ramBytesUsed":4808,
> "CACHE.searcher.filterCache.cumulative_idleEvictions":0,
> "CACHE.searcher.filterCache.inserts":1,
> "CACHE.searcher.filterCache.cumulative_lookups":1}}},
> $ curl -sS '
> http://localhost:8983/solr/techproducts/query?q=name:solr&fq=inStock=true'
> > /dev/null
> $ curl -sS '
> http://localhost:8983/solr/techproducts/admin/mbeans?wt=json&indent=true&category=CACHE&stats=true&key=filterCache'
> | grep 'CACHE.searcher.filterCache'
> "CACHE.searcher.filterCache.hits":1,
> "CACHE.searcher.filterCache.cumulative_evictions":0,
> "CACHE.searcher.filterCache.cleanupThread":false,
> "CACHE.searcher.filterCache.size":1,
> "CACHE.searcher.filterCache.maxRamMB":-1,
> "CACHE.searcher.filterCache.hitratio":0.5,
> "CACHE.searcher.filterCache.warmupTime":0,
> "CACHE.searcher.filterCache.idleEvictions":0,
> "CACHE.searcher.filterCache.evictions":0,
> "CACHE.searcher.filterCache.cumulative_hitratio":0.5,
> "CACHE.searcher.filterCache.lookups":2,
> "CACHE.searcher.filterCache.cumulative_hits":1,
> "CACHE.searcher.filterCache.cumulative_inserts":1,
> "CACHE.searcher.filterCache.ramBytesUsed":4808,
> "CACHE.searcher.filterCache.cumulative_idleEvictions":0,
> "CACHE.searcher.filterCache.inserts":1,
> "CACHE.searcher.filterCache.cumulative_lookups":2}}},
>
> ...so the first time we use 'fq=inStock:true' we get a single lookup and a
> single insert. he second time we use it (even with a different 'q' param)
> we get our 2nd lookup and our 1st hit -- no new inserts -- and now we have
> a 50% hitratio.
>
> how does that compare with what you see? what do similar commands show
> you with your fq?
>
>
>
>
> -Hoss
> http://www.lucidworks.com/
>
Re: Solr filter cache hits not reflecting
Posted by Chris Hostetter <ho...@fucit.org>.
: I was trying to analyze the filter cache performance and noticed a strange
: thing. Upon searching with fq, the entry gets added to the cache the first
: time. Observing from the "Stats/Plugins" tab on Solr admin UI, the 'lookup'
: and 'inserts' count gets incremented.
: However, if I search with the same fq again, I expect the lookup and hits
: count to increase, but it doesn't. This ultimately results in an incorrect
: hitratio.
We'll need to see the actual specifics of the requests you're executing &
stats you're seeing in order to make any guesses as to why you're not
seeing the expected outcome.
Wild guesses:
- Are you use Date math based fq params that don't round?
- Are you using SolrCloud and some of your requests are getting routed to
different replicas?
- Are you using some complex/custom filter impl that may have a bug in
it's equals/hashCode impl that prevents it from being a cache hit?
Here's an example showing that the basics of filterCache work find with
8.5 for trivial examples...
$ bin/solr -e techproducts
...
$ curl -sS 'http://localhost:8983/solr/techproducts/admin/mbeans?wt=json&indent=true&category=CACHE&stats=true&key=filterCache' | grep 'CACHE.searcher.filterCache'
"CACHE.searcher.filterCache.hits":0,
"CACHE.searcher.filterCache.cumulative_evictions":0,
"CACHE.searcher.filterCache.cleanupThread":false,
"CACHE.searcher.filterCache.size":0,
"CACHE.searcher.filterCache.maxRamMB":-1,
"CACHE.searcher.filterCache.hitratio":0.0,
"CACHE.searcher.filterCache.warmupTime":0,
"CACHE.searcher.filterCache.idleEvictions":0,
"CACHE.searcher.filterCache.evictions":0,
"CACHE.searcher.filterCache.cumulative_hitratio":0.0,
"CACHE.searcher.filterCache.lookups":0,
"CACHE.searcher.filterCache.cumulative_hits":0,
"CACHE.searcher.filterCache.cumulative_inserts":0,
"CACHE.searcher.filterCache.ramBytesUsed":1328,
"CACHE.searcher.filterCache.cumulative_idleEvictions":0,
"CACHE.searcher.filterCache.inserts":0,
"CACHE.searcher.filterCache.cumulative_lookups":0}}},
$ curl -sS 'http://localhost:8983/solr/techproducts/query?q=*:*&fq=inStock=true' > /dev/null
$ curl -sS 'http://localhost:8983/solr/techproducts/admin/mbeans?wt=json&indent=true&category=CACHE&stats=true&key=filterCache' | grep 'CACHE.searcher.filterCache'
"CACHE.searcher.filterCache.hits":0,
"CACHE.searcher.filterCache.cumulative_evictions":0,
"CACHE.searcher.filterCache.cleanupThread":false,
"CACHE.searcher.filterCache.size":1,
"CACHE.searcher.filterCache.maxRamMB":-1,
"CACHE.searcher.filterCache.hitratio":0.0,
"CACHE.searcher.filterCache.warmupTime":0,
"CACHE.searcher.filterCache.idleEvictions":0,
"CACHE.searcher.filterCache.evictions":0,
"CACHE.searcher.filterCache.cumulative_hitratio":0.0,
"CACHE.searcher.filterCache.lookups":1,
"CACHE.searcher.filterCache.cumulative_hits":0,
"CACHE.searcher.filterCache.cumulative_inserts":1,
"CACHE.searcher.filterCache.ramBytesUsed":4808,
"CACHE.searcher.filterCache.cumulative_idleEvictions":0,
"CACHE.searcher.filterCache.inserts":1,
"CACHE.searcher.filterCache.cumulative_lookups":1}}},
$ curl -sS 'http://localhost:8983/solr/techproducts/query?q=name:solr&fq=inStock=true' > /dev/null
$ curl -sS 'http://localhost:8983/solr/techproducts/admin/mbeans?wt=json&indent=true&category=CACHE&stats=true&key=filterCache' | grep 'CACHE.searcher.filterCache'
"CACHE.searcher.filterCache.hits":1,
"CACHE.searcher.filterCache.cumulative_evictions":0,
"CACHE.searcher.filterCache.cleanupThread":false,
"CACHE.searcher.filterCache.size":1,
"CACHE.searcher.filterCache.maxRamMB":-1,
"CACHE.searcher.filterCache.hitratio":0.5,
"CACHE.searcher.filterCache.warmupTime":0,
"CACHE.searcher.filterCache.idleEvictions":0,
"CACHE.searcher.filterCache.evictions":0,
"CACHE.searcher.filterCache.cumulative_hitratio":0.5,
"CACHE.searcher.filterCache.lookups":2,
"CACHE.searcher.filterCache.cumulative_hits":1,
"CACHE.searcher.filterCache.cumulative_inserts":1,
"CACHE.searcher.filterCache.ramBytesUsed":4808,
"CACHE.searcher.filterCache.cumulative_idleEvictions":0,
"CACHE.searcher.filterCache.inserts":1,
"CACHE.searcher.filterCache.cumulative_lookups":2}}},
...so the first time we use 'fq=inStock:true' we get a single lookup and a
single insert. he second time we use it (even with a different 'q' param)
we get our 2nd lookup and our 1st hit -- no new inserts -- and now we have
a 50% hitratio.
how does that compare with what you see? what do similar commands show
you with your fq?
-Hoss
http://www.lucidworks.com/