You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@lucene.apache.org by "Zach Chen (Jira)" <ji...@apache.org> on 2021/06/12 18:47:00 UTC
[jira] [Commented] (LUCENE-9959) Can we remove threadlocals of
stored fields and term vectors
[ https://issues.apache.org/jira/browse/LUCENE-9959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17362392#comment-17362392 ]
Zach Chen commented on LUCENE-9959:
-----------------------------------
I took a look at this issue and the idea suggested by Robert (and https://issues.apache.org/jira/browse/LUCENE-1195 that seems to introduce thread local originally), and gave it a try with this WIP PR [https://github.com/apache/lucene/pull/180] (with commit [https://github.com/apache/lucene/commit/5062e4d69938f104b461004022e19c10a65960a5] that has the most meaningful changes). Is the implementation what you are expecting? I feel since `IndexReader` already has APIs _getTermVectors_ and _getTermVector_, it might not be too bad to add a new API to go alongside with them, and gradually phased out the use of the existing two (at least for term vector)?
In addition, I'm a bit wondering why other readers from SegmentReader don't need to use the same thread local approach for concurrency / caching (namely, the PointsReader, NormsProducer, DocValuesProducer, VectorReader, FieldsProducer in SegmentReader). I'm guessing these readers' operations might be much less costly compared with term vector reader and stored field reader, so their operations are made thread-safe internally? I'll dig around to understand more about the context there...
> Can we remove threadlocals of stored fields and term vectors
> ------------------------------------------------------------
>
> Key: LUCENE-9959
> URL: https://issues.apache.org/jira/browse/LUCENE-9959
> Project: Lucene - Core
> Issue Type: Improvement
> Reporter: Adrien Grand
> Priority: Minor
>
> [~rmuir] suggested removing these threadlocals at https://github.com/apache/lucene/pull/137#issuecomment-840111367.
> These threadlocals are trappy if you manage many segments and threads within the same JVM, or worse: non-fixed threadpools. The challenge is to keep the API easy to use.
> We could take advantage of 9.0 to change the stored fields API?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org