Re: Content search and applying ACL

[k-jingyang <k....@protonmail.com.INVALID>]

After upgrading our Solr to 8.8 from 7.2, I've tested out the *score=none*,
and *method=toplevelDV* with two collections. The first few queries took
even longer, the first one took up to 20s as compared to 8s from
*method=index*. 

I guess this is inline with what the docs mentioned about topLevelDV, "These
data structures outperform other methods as the number of values matched in
the from field grows high. But they are also expensive to build and need to
be lazily populated after each commit, causing a sometimes-noticeable
slowdown on the first query to use them after each commit." 

What I'm now curious is what these "data structures that are expensive to
build" are?

I dived into /TopLevelJoinQuery.java/,  and saw that the implementation
retrieves the DocValues from the respective indexes and use them for the
join operation. Am I right to assume that the "data structures that are
expensive to build" is referring to DocValues?  

Would that also mean that other operations (e.g. faceting) that use
DocValues will take some time after a commit?

Anyway, to solve the problem detailed in the first post, we denormalised our
indexes (i.e. embed the list of teams that can read the information into our
content documents), and lookups are now almost instantaneous. One
consequence of this is that whenever the ACL changes (i.e. which teams can
read this content), we'll have to re-index all relevant content data. 

We have judged this to be acceptable, because
1) We can run this asynchronously
2) ACL changes need not be instantaneous
3) Due to the nature of our app, the amount time required to re-index the
relevant content data is still within acceptable range and will be difficult
to grow out of unacceptable range.

Hope this helps any individual who comes across this! :)    




--
Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html