You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Michael McCandless (Created) (JIRA)" <ji...@apache.org> on 2012/02/27 20:16:46 UTC
[jira] [Created] (LUCENE-3829) Lucene40 codec's DocValues
DirectSource impls aren't thread-safe
Lucene40 codec's DocValues DirectSource impls aren't thread-safe
----------------------------------------------------------------
Key: LUCENE-3829
URL: https://issues.apache.org/jira/browse/LUCENE-3829
Project: Lucene - Java
Issue Type: Bug
Reporter: Michael McCandless
Fix For: 4.0
Our DirectSource impls hold IndexInput(s) open against the dat/idx
files, which we then seek + read when loading a specific document's
value. But this is in no way protected against multiple threads
I think...?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (LUCENE-3829) Lucene40 codec's DocValues
DirectSource impls aren't thread-safe
Posted by "Michael McCandless (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13217428#comment-13217428 ]
Michael McCandless commented on LUCENE-3829:
--------------------------------------------
bq. maybe we need per-thread cloning like for stored fields?
Yeah I think we need something just like that... not sure where we should insert it into the API though.
bq. At least all this is more funny than stupid FC insanity by reader wrapping... DOH!
:)
> Lucene40 codec's DocValues DirectSource impls aren't thread-safe
> ----------------------------------------------------------------
>
> Key: LUCENE-3829
> URL: https://issues.apache.org/jira/browse/LUCENE-3829
> Project: Lucene - Java
> Issue Type: Bug
> Reporter: Michael McCandless
> Fix For: 4.0
>
> Attachments: LUCENE-3829.patch
>
>
> Our DirectSource impls hold IndexInput(s) open against the dat/idx
> files, which we then seek + read when loading a specific document's
> value. But this is in no way protected against multiple threads
> I think...?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (LUCENE-3829) Lucene40 codec's DocValues
DirectSource impls aren't thread-safe
Posted by "Simon Willnauer (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13218542#comment-13218542 ]
Simon Willnauer commented on LUCENE-3829:
-----------------------------------------
wait, I think your test is wrong. DV consumer should pull a source directly from DV per thread. An instance is not threadsafe per-se. if you pull an "in-memory" source you can simply share it but the interface doesn't guarantee this. you can simply pull the "in-memory" source in each thread and you get the same instance since it is cached. The same is true for the on-disk source since it is not cached. I clone the IndexInput in getDirectSource() to prevent this problem.
> Lucene40 codec's DocValues DirectSource impls aren't thread-safe
> ----------------------------------------------------------------
>
> Key: LUCENE-3829
> URL: https://issues.apache.org/jira/browse/LUCENE-3829
> Project: Lucene - Java
> Issue Type: Bug
> Reporter: Michael McCandless
> Fix For: 4.0
>
> Attachments: LUCENE-3829.patch
>
>
> Our DirectSource impls hold IndexInput(s) open against the dat/idx
> files, which we then seek + read when loading a specific document's
> value. But this is in no way protected against multiple threads
> I think...?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Resolved] (LUCENE-3829) Lucene40 codec's DocValues
DirectSource impls aren't thread-safe
Posted by "Michael McCandless (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael McCandless resolved LUCENE-3829.
----------------------------------------
Resolution: Invalid
Duh, thanks Simon ;)
Once I fixed the test to use the API correctly, it passes!
> Lucene40 codec's DocValues DirectSource impls aren't thread-safe
> ----------------------------------------------------------------
>
> Key: LUCENE-3829
> URL: https://issues.apache.org/jira/browse/LUCENE-3829
> Project: Lucene - Java
> Issue Type: Bug
> Reporter: Michael McCandless
> Fix For: 4.0
>
> Attachments: LUCENE-3829.patch
>
>
> Our DirectSource impls hold IndexInput(s) open against the dat/idx
> files, which we then seek + read when loading a specific document's
> value. But this is in no way protected against multiple threads
> I think...?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (LUCENE-3829) Lucene40 codec's DocValues
DirectSource impls aren't thread-safe
Posted by "Uwe Schindler (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13217417#comment-13217417 ]
Uwe Schindler commented on LUCENE-3829:
---------------------------------------
Good catch! maybe we need per-thread cloning like for stored fields?
At least all this is more funny than stupid FC insanity by reader wrapping... DOH!
> Lucene40 codec's DocValues DirectSource impls aren't thread-safe
> ----------------------------------------------------------------
>
> Key: LUCENE-3829
> URL: https://issues.apache.org/jira/browse/LUCENE-3829
> Project: Lucene - Java
> Issue Type: Bug
> Reporter: Michael McCandless
> Fix For: 4.0
>
> Attachments: LUCENE-3829.patch
>
>
> Our DirectSource impls hold IndexInput(s) open against the dat/idx
> files, which we then seek + read when loading a specific document's
> value. But this is in no way protected against multiple threads
> I think...?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (LUCENE-3829) Lucene40 codec's DocValues
DirectSource impls aren't thread-safe
Posted by "Michael McCandless (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael McCandless updated LUCENE-3829:
---------------------------------------
Attachment: LUCENE-3829.patch
Patch showing the failure (you have to remove the true || in
TestDocValuesIndexing.testWithThreads), and also (separately) adding
DV coverage to LineFileDocs for better test coverage.
I'll commit the patch shortly (it's tests only, and passing because I
disabled testing DirectSource with threads until we can fix this
issue; we still test Source with threads).
> Lucene40 codec's DocValues DirectSource impls aren't thread-safe
> ----------------------------------------------------------------
>
> Key: LUCENE-3829
> URL: https://issues.apache.org/jira/browse/LUCENE-3829
> Project: Lucene - Java
> Issue Type: Bug
> Reporter: Michael McCandless
> Fix For: 4.0
>
> Attachments: LUCENE-3829.patch
>
>
> Our DirectSource impls hold IndexInput(s) open against the dat/idx
> files, which we then seek + read when loading a specific document's
> value. But this is in no way protected against multiple threads
> I think...?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org