[jira] [Commented] (LUCENENET-610) Reduce locking in FieldCacheImpl::Cache::Get

["Shad Storhaug (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/LUCENENET-610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16960423#comment-16960423 ] 

Shad Storhaug commented on LUCENENET-610:
-----------------------------------------

While having a thread safe collection would be ideal, that is not how it was implemented in Lucene. However, I took a look at the implementation of {{WeakDictionary}} and it is dissimilar to that of the {{java.util.WeakHashMap}} that was used in the OpenJDK. In particular the {{CleanIfNeeded()}} method moves the entries from one dictionary instance to another every time it is called, which is less than ideal.

According to [this StackOverflow answer|https://stackoverflow.com/a/2081664], it may be possible to use a [ConditionalWeakTable<TKey,TValue>|https://docs.microsoft.com/en-us/dotnet/api/system.runtime.compilerservices.conditionalweaktable-2?redirectedfrom=MSDN&view=netframework-4.8] as a replacement.

However, before it is attempted, it would be nice to have a reproduction case so it is possible to see whether we are indeed solving the issue or making it worse. Could you please provide one?

> Reduce locking in FieldCacheImpl::Cache::Get
> --------------------------------------------
>
>                 Key: LUCENENET-610
>                 URL: https://issues.apache.org/jira/browse/LUCENENET-610
>             Project: Lucene.Net
>          Issue Type: Improvement
>          Components: Lucene.Net Core
>    Affects Versions: Lucene.Net 4.8.0
>            Reporter: Thomas Mathew
>            Priority: Minor
>              Labels: up-for-grabs
>
> We noticed a lot of contention in FieldCacheImpl::Cache::Get (our queries use a lot of query time joins + sorting, so we hit the field cache a lot).
> We use a SearcherManager with warm-up queries to populate the field cache so we would expect it to be initialized in most cases before we hit it for actual requests.
> The implementation seems to lock even for the happy path (when everything's already initialized). This seems like a by-product of the choice of data structures (the underlying WeakDictionary, WeakHashMap etc are not threadsafe) and so the locking is required in case the dictionary gets resized.
> Ideally we could be using thread-safe data structures and only lock when initializing the data.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)