You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@jena.apache.org by "Brian McBride (JIRA)" <ji...@apache.org> on 2010/12/09 17:44:00 UTC
[jira] Created: (JENA-5) IndexLARQ is not thread safe
IndexLARQ is not thread safe
----------------------------
Key: JENA-5
URL: https://issues.apache.org/jira/browse/JENA-5
Project: Jena
Issue Type: Bug
Reporter: Brian McBride
Copying a previously sent email:
I ran into a concurrency problem in LARQ. When I try to run concurrent queries the Lucene query parser barfs.
After a bit of digging, I think I understand what is going on. I was creating an IndexLARQ and setting that as the global index, then running my queries. The IndexLARQ constructor constructs a Lucene QueryParser object which it uses for all queries. Lucene QueryParser objects are not thread safe, so multiple concurrent queries cause the parser the barf.
I have had a go at patching the HEAD from SVN. Basically I make the QueryParser object in IndexLARQ ThreadLocal.
The problem with this is that there is a constructor to IndexLARQ which takes an externally provided QueryParser object and QueryParser is not clonable. I could replicate the query parser for each thread provided I know its a QueryParser and not a subclass of QueryParser. If someone passed in a subclass of QueryParser I'd go creating the wrong class of objects as a replica.
Whilst this scenario is not very likely, since I think the constructor should be deprecated anyway, what I've done is esentially left that constructor alone. Creating an IndexLARQ with that contructor will continue to construct one that is not thread safe. The other constructors will construct threadsafe instances.
Attached are:
a modified IndexLARQ.java that fixes the problem as I've described.
TestLARQConcurrent.java - a test case
larqConcurrentPatch - a patch file that applies the changes
Oops - Just noticed the license notice on the test case file - you will want to change that if you use this.
hmmm - there doesn't seem to be a button here for attaching files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Closed: (JENA-5) IndexLARQ is not thread safe
Posted by "Andy Seaborne (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JENA-5?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andy Seaborne closed JENA-5.
----------------------------
Resolution: Fixed
(At the moment, the Jena and ARQ source have not been brought into SVN in Apache)
Fixed in ARQ in pre-Apache codebase by creating new parser each time (the patch file was not used due to copyright comment).
This will give the necessary concurrent use - we will need to check that no performance issues arise but there is a new release being made of Jena and of ARQ so fix now is better than release unfixed.
> IndexLARQ is not thread safe
> ----------------------------
>
> Key: JENA-5
> URL: https://issues.apache.org/jira/browse/JENA-5
> Project: Jena
> Issue Type: Bug
> Components: ARQ
> Reporter: Brian McBride
> Assignee: Andy Seaborne
> Attachments: IndexLARQ.java, larqConcurrentPatch, TestLARQConcurrent.java
>
>
> Copying a previously sent email:
> I ran into a concurrency problem in LARQ. When I try to run concurrent queries the Lucene query parser barfs.
> After a bit of digging, I think I understand what is going on. I was creating an IndexLARQ and setting that as the global index, then running my queries. The IndexLARQ constructor constructs a Lucene QueryParser object which it uses for all queries. Lucene QueryParser objects are not thread safe, so multiple concurrent queries cause the parser the barf.
> I have had a go at patching the HEAD from SVN. Basically I make the QueryParser object in IndexLARQ ThreadLocal.
> The problem with this is that there is a constructor to IndexLARQ which takes an externally provided QueryParser object and QueryParser is not clonable. I could replicate the query parser for each thread provided I know its a QueryParser and not a subclass of QueryParser. If someone passed in a subclass of QueryParser I'd go creating the wrong class of objects as a replica.
> Whilst this scenario is not very likely, since I think the constructor should be deprecated anyway, what I've done is esentially left that constructor alone. Creating an IndexLARQ with that contructor will continue to construct one that is not thread safe. The other constructors will construct threadsafe instances.
> Attached are:
> a modified IndexLARQ.java that fixes the problem as I've described.
> TestLARQConcurrent.java - a test case
> larqConcurrentPatch - a patch file that applies the changes
> Oops - Just noticed the license notice on the test case file - you will want to change that if you use this.
> hmmm - there doesn't seem to be a button here for attaching files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (JENA-5) IndexLARQ is not thread safe
Posted by "Brian McBride (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JENA-5?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brian McBride updated JENA-5:
-----------------------------
Attachment: larqConcurrentPatch
A patch file that applies my fix to the code and creates a junit test to test it.
> IndexLARQ is not thread safe
> ----------------------------
>
> Key: JENA-5
> URL: https://issues.apache.org/jira/browse/JENA-5
> Project: Jena
> Issue Type: Bug
> Reporter: Brian McBride
> Attachments: IndexLARQ.java, larqConcurrentPatch, TestLARQConcurrent.java
>
>
> Copying a previously sent email:
> I ran into a concurrency problem in LARQ. When I try to run concurrent queries the Lucene query parser barfs.
> After a bit of digging, I think I understand what is going on. I was creating an IndexLARQ and setting that as the global index, then running my queries. The IndexLARQ constructor constructs a Lucene QueryParser object which it uses for all queries. Lucene QueryParser objects are not thread safe, so multiple concurrent queries cause the parser the barf.
> I have had a go at patching the HEAD from SVN. Basically I make the QueryParser object in IndexLARQ ThreadLocal.
> The problem with this is that there is a constructor to IndexLARQ which takes an externally provided QueryParser object and QueryParser is not clonable. I could replicate the query parser for each thread provided I know its a QueryParser and not a subclass of QueryParser. If someone passed in a subclass of QueryParser I'd go creating the wrong class of objects as a replica.
> Whilst this scenario is not very likely, since I think the constructor should be deprecated anyway, what I've done is esentially left that constructor alone. Creating an IndexLARQ with that contructor will continue to construct one that is not thread safe. The other constructors will construct threadsafe instances.
> Attached are:
> a modified IndexLARQ.java that fixes the problem as I've described.
> TestLARQConcurrent.java - a test case
> larqConcurrentPatch - a patch file that applies the changes
> Oops - Just noticed the license notice on the test case file - you will want to change that if you use this.
> hmmm - there doesn't seem to be a button here for attaching files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (JENA-5) IndexLARQ is not thread safe
Posted by "Andy Seaborne (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JENA-5?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12969804#action_12969804 ]
Andy Seaborne commented on JENA-5:
----------------------------------
Brian has also said in previous email:
"""
Thinking about this a bit more, I'm not sure that the ThreadLocal approach is the best one. It means keeping one parser per thread around for the lifetime of the index. In my case that would around 150 for the lifetime of the application. I presume there is not a lot of overhead in setting up a new parser when one is needed. So maybe better to do that.
And my test case just tests for the specific problem I had. It needs more work to be a generic test of concurrent queries in LARQ.
"""
> IndexLARQ is not thread safe
> ----------------------------
>
> Key: JENA-5
> URL: https://issues.apache.org/jira/browse/JENA-5
> Project: Jena
> Issue Type: Bug
> Reporter: Brian McBride
> Attachments: IndexLARQ.java, larqConcurrentPatch, TestLARQConcurrent.java
>
>
> Copying a previously sent email:
> I ran into a concurrency problem in LARQ. When I try to run concurrent queries the Lucene query parser barfs.
> After a bit of digging, I think I understand what is going on. I was creating an IndexLARQ and setting that as the global index, then running my queries. The IndexLARQ constructor constructs a Lucene QueryParser object which it uses for all queries. Lucene QueryParser objects are not thread safe, so multiple concurrent queries cause the parser the barf.
> I have had a go at patching the HEAD from SVN. Basically I make the QueryParser object in IndexLARQ ThreadLocal.
> The problem with this is that there is a constructor to IndexLARQ which takes an externally provided QueryParser object and QueryParser is not clonable. I could replicate the query parser for each thread provided I know its a QueryParser and not a subclass of QueryParser. If someone passed in a subclass of QueryParser I'd go creating the wrong class of objects as a replica.
> Whilst this scenario is not very likely, since I think the constructor should be deprecated anyway, what I've done is esentially left that constructor alone. Creating an IndexLARQ with that contructor will continue to construct one that is not thread safe. The other constructors will construct threadsafe instances.
> Attached are:
> a modified IndexLARQ.java that fixes the problem as I've described.
> TestLARQConcurrent.java - a test case
> larqConcurrentPatch - a patch file that applies the changes
> Oops - Just noticed the license notice on the test case file - you will want to change that if you use this.
> hmmm - there doesn't seem to be a button here for attaching files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (JENA-5) IndexLARQ is not thread safe
Posted by "Andy Seaborne (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JENA-5?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andy Seaborne updated JENA-5:
-----------------------------
Component/s: ARQ
Assignee: Andy Seaborne
> IndexLARQ is not thread safe
> ----------------------------
>
> Key: JENA-5
> URL: https://issues.apache.org/jira/browse/JENA-5
> Project: Jena
> Issue Type: Bug
> Components: ARQ
> Reporter: Brian McBride
> Assignee: Andy Seaborne
> Attachments: IndexLARQ.java, larqConcurrentPatch, TestLARQConcurrent.java
>
>
> Copying a previously sent email:
> I ran into a concurrency problem in LARQ. When I try to run concurrent queries the Lucene query parser barfs.
> After a bit of digging, I think I understand what is going on. I was creating an IndexLARQ and setting that as the global index, then running my queries. The IndexLARQ constructor constructs a Lucene QueryParser object which it uses for all queries. Lucene QueryParser objects are not thread safe, so multiple concurrent queries cause the parser the barf.
> I have had a go at patching the HEAD from SVN. Basically I make the QueryParser object in IndexLARQ ThreadLocal.
> The problem with this is that there is a constructor to IndexLARQ which takes an externally provided QueryParser object and QueryParser is not clonable. I could replicate the query parser for each thread provided I know its a QueryParser and not a subclass of QueryParser. If someone passed in a subclass of QueryParser I'd go creating the wrong class of objects as a replica.
> Whilst this scenario is not very likely, since I think the constructor should be deprecated anyway, what I've done is esentially left that constructor alone. Creating an IndexLARQ with that contructor will continue to construct one that is not thread safe. The other constructors will construct threadsafe instances.
> Attached are:
> a modified IndexLARQ.java that fixes the problem as I've described.
> TestLARQConcurrent.java - a test case
> larqConcurrentPatch - a patch file that applies the changes
> Oops - Just noticed the license notice on the test case file - you will want to change that if you use this.
> hmmm - there doesn't seem to be a button here for attaching files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (JENA-5) IndexLARQ is not thread safe
Posted by "Brian McBride (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JENA-5?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brian McBride updated JENA-5:
-----------------------------
Attachment: TestLARQConcurrent.java
A test case for problem.
> IndexLARQ is not thread safe
> ----------------------------
>
> Key: JENA-5
> URL: https://issues.apache.org/jira/browse/JENA-5
> Project: Jena
> Issue Type: Bug
> Reporter: Brian McBride
> Attachments: TestLARQConcurrent.java
>
>
> Copying a previously sent email:
> I ran into a concurrency problem in LARQ. When I try to run concurrent queries the Lucene query parser barfs.
> After a bit of digging, I think I understand what is going on. I was creating an IndexLARQ and setting that as the global index, then running my queries. The IndexLARQ constructor constructs a Lucene QueryParser object which it uses for all queries. Lucene QueryParser objects are not thread safe, so multiple concurrent queries cause the parser the barf.
> I have had a go at patching the HEAD from SVN. Basically I make the QueryParser object in IndexLARQ ThreadLocal.
> The problem with this is that there is a constructor to IndexLARQ which takes an externally provided QueryParser object and QueryParser is not clonable. I could replicate the query parser for each thread provided I know its a QueryParser and not a subclass of QueryParser. If someone passed in a subclass of QueryParser I'd go creating the wrong class of objects as a replica.
> Whilst this scenario is not very likely, since I think the constructor should be deprecated anyway, what I've done is esentially left that constructor alone. Creating an IndexLARQ with that contructor will continue to construct one that is not thread safe. The other constructors will construct threadsafe instances.
> Attached are:
> a modified IndexLARQ.java that fixes the problem as I've described.
> TestLARQConcurrent.java - a test case
> larqConcurrentPatch - a patch file that applies the changes
> Oops - Just noticed the license notice on the test case file - you will want to change that if you use this.
> hmmm - there doesn't seem to be a button here for attaching files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (JENA-5) IndexLARQ is not thread safe
Posted by "Brian McBride (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JENA-5?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brian McBride updated JENA-5:
-----------------------------
Attachment: IndexLARQ.java
This is a modified IndexLARQ that gets around the problem by making the query parser object thread local. This was a fix that makes a minimal change to the structure of the original code but is probably not the best way to fix things.
Note there are issues with the interface to IndexLARQ as its public methods and constructurs seem to expect there to be a single lucene query parser object.
> IndexLARQ is not thread safe
> ----------------------------
>
> Key: JENA-5
> URL: https://issues.apache.org/jira/browse/JENA-5
> Project: Jena
> Issue Type: Bug
> Reporter: Brian McBride
> Attachments: IndexLARQ.java, TestLARQConcurrent.java
>
>
> Copying a previously sent email:
> I ran into a concurrency problem in LARQ. When I try to run concurrent queries the Lucene query parser barfs.
> After a bit of digging, I think I understand what is going on. I was creating an IndexLARQ and setting that as the global index, then running my queries. The IndexLARQ constructor constructs a Lucene QueryParser object which it uses for all queries. Lucene QueryParser objects are not thread safe, so multiple concurrent queries cause the parser the barf.
> I have had a go at patching the HEAD from SVN. Basically I make the QueryParser object in IndexLARQ ThreadLocal.
> The problem with this is that there is a constructor to IndexLARQ which takes an externally provided QueryParser object and QueryParser is not clonable. I could replicate the query parser for each thread provided I know its a QueryParser and not a subclass of QueryParser. If someone passed in a subclass of QueryParser I'd go creating the wrong class of objects as a replica.
> Whilst this scenario is not very likely, since I think the constructor should be deprecated anyway, what I've done is esentially left that constructor alone. Creating an IndexLARQ with that contructor will continue to construct one that is not thread safe. The other constructors will construct threadsafe instances.
> Attached are:
> a modified IndexLARQ.java that fixes the problem as I've described.
> TestLARQConcurrent.java - a test case
> larqConcurrentPatch - a patch file that applies the changes
> Oops - Just noticed the license notice on the test case file - you will want to change that if you use this.
> hmmm - there doesn't seem to be a button here for attaching files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.