You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@jackrabbit.apache.org by Sean Callan <se...@gmail.com> on 2008/01/23 19:35:45 UTC
Lucene AlreadyClosedException
Hi guys,
I just upgraded to 1.4 and I'm suddenly getting an exception that
hadn't previously occurred. It's being thrown when the following
Xpath is executed : "//webfmis:role[@webfmis:general='true']"
org.apache.lucene.store.AlreadyClosedException: this IndexReader is closed
Any idea what is causing this? I tried searching for related issue
but have not found any.
Thanks,
Sean
RE: Lucene AlreadyClosedException
Posted by julien herfurth <jh...@gmail.com>.
Ard Schrijvers wrote:
>
> Also, Jackrabbit 1.4 is released with a dependency on Lucene 2.2.0.
> Perhaps the closing of an IndexReader in lucene has changed. Anyway, can
> you confirm that you have the same problem with lucene 2.2.0? Lucene
> 2.3.0 will probably be used in the the next jr version, so if it would
> be a problem, it will certainly be fixed then
>
> -Ard
>
>
I confirm, it works well with lucene-2.2.0 and fails with lucene-2.3.0. I'll
stick to lucene-2.2.0 :-)
Thanks,
Julien
--
View this message in context: http://www.nabble.com/Lucene-AlreadyClosedException-tp15049132p15166032.html
Sent from the Jackrabbit - Users mailing list archive at Nabble.com.
RE: Lucene AlreadyClosedException
Posted by Ard Schrijvers <a....@hippo.nl>.
> I have not tried 2.2.0 and will continue to utilize 2.3.0 as
Ofcourse you are totally free to do so, but I guess you can update lots
of dependencies on jars which might have newer releases. I just don't
understand why you are convinced that everything works with 2.3.0.
Ideally, it would, but there is a reason why a pom.xml contains a
version to some used jar.
> it is the latest version of Lucene and seems to function
> properly with this change. For testing purposes I will try
> and look through 2.2.0 to see if there is still a conflict
> with the refCount variable.
>
> Why wait to fix an issue that has been identified early on?
Again, I saw how you tried to fix the issue, and it simply isn't a fix.
As far as I can see you just removed the refCount. So, when a index
reader in your case gets closed, it is closed instantly, while another
request might still be using it (multithreaded environment, hence ref
counting, etc etc).
Anyway, I now explained it twice, your fix is not a fix, and the current
released version of jr 1.4 which depends on lucene 2.2.0 does not have
this problem. If we start to use lucene 2.3.0 we will fix this problem
properly,
-Ard
>
> On Jan 29, 2008 11:13 AM, Ard Schrijvers
> <a....@hippo.nl> wrote:
> > Hello,
> >
> > >
> > > Hi,
> > >
> > > I was able to track this issue down in my scenario and
> I'll provide
> > > a diff later today. In my case I am using Websphere
> > > 6.1 and Lucene 2.3.0. It would appear that there is a variable
> > > conflict that is only present when utilizing Lucene
> 2.3.0, refCount,
> > > the reference counter is used in JackRabbit's
> > > CachingMultiIndexSearch and in Lucene's IndexReader.
> >
> > Think you mean CachingMultiIndexReader
> >
> > >
> > > If you make the following changes to CachingMultiIndexSearch the
> > > issue appears to be resolved:
> > > * Comment out line 64 `private int refCount = 1`
> > > * Replace line 159 with `incRef()`
> > > * Replace lines 168-170 with `super.doClose()`
> > >
> > > IndexReader 2.3.0 has the variable refCount and this is
> the variable
> > > that is checked and results in the thrown exception.
> > > CachingMultiIndexSearch has the same variable and increments it
> > > accordingly, failing to increment the parent class's variable.
> >
> > The same variable? It is a private member in
> CachingMultiIndexReader.
> > Anyway, your solution seems to me like just removing
> refCount at all,
> > making it very well possible that your indexreader needs to
> be opened
> > over and over again, or that it is never actually garbage
> collected (I
> > am guessing because i am not sure about the exact
> behavior). I haven't
> > tested your solution, but I am quite confident it is not correct.
> >
> > Also, Jackrabbit 1.4 is released with a dependency on Lucene 2.2.0.
> > Perhaps the closing of an IndexReader in lucene has
> changed. Anyway,
> > can you confirm that you have the same problem with lucene 2.2.0?
> > Lucene 2.3.0 will probably be used in the the next jr
> version, so if
> > it would be a problem, it will certainly be fixed then
> >
> > -Ard
> >
> >
> > >
> > > Hope that helps,
> > > Sean
> > >
> > >
> > >
> > > On Jan 29, 2008 3:23 AM, Ard Schrijvers
> <a....@hippo.nl> wrote:
> > > > Hello,
> > > >
> > > > pls do send the test case (do you have a unit test??)
> > > >
> > > > Regards Ard
> > > >
> > > >
> > > > > Hi,
> > > > >
> > > > > I have the same issue. It seems that the only XPATH
> query that
> > > > > is working is "//*". Anything else will fail with the same
> > > exception. I
> > > > > can pack a test case (libraries included) and send it.
> > > > >
> > > > > Julien
> > > > > --
> > > > > View this message in context:
> > > > > http://www.nabble.com/Lucene-AlreadyClosedException-tp15049132
> > > > > p15152752.html
> > > > > Sent from the Jackrabbit - Users mailing list archive at
> > > Nabble.com.
> > > > >
> > > > >
> > > >
> > >
> >
>
Re: Lucene AlreadyClosedException
Posted by Jukka Zitting <ju...@gmail.com>.
Hi,
On Thu, Oct 9, 2008 at 8:01 PM, nicholas_dipiazza <ni...@xerceo.com> wrote:
> I have been tasked with fixing Jackrabbit 1.4 to work with Lucene 2.3.2. Our
> application is too dependent upon it now to revert.
>
> Has anyone taken this on at this point? I would hate to redo work already
> done by a developer who works on the Jackrabbit codebase all day.
See https://issues.apache.org/jira/browse/JCR-1363.
BR,
Jukka Zitting
RE: Lucene AlreadyClosedException
Posted by nicholas_dipiazza <ni...@xerceo.com>.
I have been tasked with fixing Jackrabbit 1.4 to work with Lucene 2.3.2. Our
application is too dependent upon it now to revert.
Has anyone taken this on at this point? I would hate to redo work already
done by a developer who works on the Jackrabbit codebase all day.
Ard Schrijvers wrote:
>
> Hello,
>
> I have looked a little further, this Lucene issue is related to it:
>
> https://issues.apache.org/jira/browse/LUCENE-743
>
> They did change quite a bit regarding ref counting in lucene from 2.2.0
> to 2.3.0. In the lucene multiReader the doClose() changed from:
>
> protected synchronized void doClose() throws IOException {
> for (int i = 0; i < subReaders.length; i++)
> subReaders[i].close();
> }
>
> to
>
> protected synchronized void doClose() throws IOException {
> for (int i = 0; i < subReaders.length; i++) {
> if (decrefOnClose[i]) {
> subReaders[i].decRef();
> } else {
> subReaders[i].close();
> }
> }
> }
>
> Also Lucene IndexReader mechanism regarding when pending changes are
> added to the index have changed (only when refCount = 0 ). Without
> investigating too much, it think it will require quite some changes in
> jackrabbit to account for these changes. The refCount in jackrabbit is
> used for different purposes then lucene seem to be using it now
>
> -Ard
>
>>
>> Why wait to fix an issue that has been identified early on?
>>
>> On Jan 29, 2008 11:13 AM, Ard Schrijvers
>> <a....@hippo.nl> wrote:
>> > Hello,
>> >
>> > >
>> > > Hi,
>> > >
>> > > I was able to track this issue down in my scenario and
>> I'll provide
>> > > a diff later today. In my case I am using Websphere
>> > > 6.1 and Lucene 2.3.0. It would appear that there is a variable
>> > > conflict that is only present when utilizing Lucene
>> 2.3.0, refCount,
>> > > the reference counter is used in JackRabbit's
>> > > CachingMultiIndexSearch and in Lucene's IndexReader.
>> >
>> > Think you mean CachingMultiIndexReader
>> >
>> > >
>> > > If you make the following changes to CachingMultiIndexSearch the
>> > > issue appears to be resolved:
>> > > * Comment out line 64 `private int refCount = 1`
>> > > * Replace line 159 with `incRef()`
>> > > * Replace lines 168-170 with `super.doClose()`
>> > >
>> > > IndexReader 2.3.0 has the variable refCount and this is
>> the variable
>> > > that is checked and results in the thrown exception.
>> > > CachingMultiIndexSearch has the same variable and increments it
>> > > accordingly, failing to increment the parent class's variable.
>> >
>> > The same variable? It is a private member in
>> CachingMultiIndexReader.
>> > Anyway, your solution seems to me like just removing
>> refCount at all,
>> > making it very well possible that your indexreader needs to
>> be opened
>> > over and over again, or that it is never actually garbage
>> collected (I
>> > am guessing because i am not sure about the exact
>> behavior). I haven't
>> > tested your solution, but I am quite confident it is not correct.
>> >
>> > Also, Jackrabbit 1.4 is released with a dependency on Lucene 2.2.0.
>> > Perhaps the closing of an IndexReader in lucene has
>> changed. Anyway,
>> > can you confirm that you have the same problem with lucene 2.2.0?
>> > Lucene 2.3.0 will probably be used in the the next jr
>> version, so if
>> > it would be a problem, it will certainly be fixed then
>> >
>> > -Ard
>> >
>> >
>> > >
>> > > Hope that helps,
>> > > Sean
>> > >
>> > >
>> > >
>> > > On Jan 29, 2008 3:23 AM, Ard Schrijvers
>> <a....@hippo.nl> wrote:
>> > > > Hello,
>> > > >
>> > > > pls do send the test case (do you have a unit test??)
>> > > >
>> > > > Regards Ard
>> > > >
>> > > >
>> > > > > Hi,
>> > > > >
>> > > > > I have the same issue. It seems that the only XPATH
>> query that
>> > > > > is working is "//*". Anything else will fail with the same
>> > > exception. I
>> > > > > can pack a test case (libraries included) and send it.
>> > > > >
>> > > > > Julien
>> > > > > --
>> > > > > View this message in context:
>> > > > > http://www.nabble.com/Lucene-AlreadyClosedException-tp15049132
>> > > > > p15152752.html
>> > > > > Sent from the Jackrabbit - Users mailing list archive at
>> > > Nabble.com.
>> > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>
>
--
View this message in context: http://www.nabble.com/Lucene-AlreadyClosedException-tp15049132p19904146.html
Sent from the Jackrabbit - Users mailing list archive at Nabble.com.
RE: Lucene AlreadyClosedException
Posted by Ard Schrijvers <a....@hippo.nl>.
Hello,
I have looked a little further, this Lucene issue is related to it:
https://issues.apache.org/jira/browse/LUCENE-743
They did change quite a bit regarding ref counting in lucene from 2.2.0
to 2.3.0. In the lucene multiReader the doClose() changed from:
protected synchronized void doClose() throws IOException {
for (int i = 0; i < subReaders.length; i++)
subReaders[i].close();
}
to
protected synchronized void doClose() throws IOException {
for (int i = 0; i < subReaders.length; i++) {
if (decrefOnClose[i]) {
subReaders[i].decRef();
} else {
subReaders[i].close();
}
}
}
Also Lucene IndexReader mechanism regarding when pending changes are
added to the index have changed (only when refCount = 0 ). Without
investigating too much, it think it will require quite some changes in
jackrabbit to account for these changes. The refCount in jackrabbit is
used for different purposes then lucene seem to be using it now
-Ard
>
> Why wait to fix an issue that has been identified early on?
>
> On Jan 29, 2008 11:13 AM, Ard Schrijvers
> <a....@hippo.nl> wrote:
> > Hello,
> >
> > >
> > > Hi,
> > >
> > > I was able to track this issue down in my scenario and
> I'll provide
> > > a diff later today. In my case I am using Websphere
> > > 6.1 and Lucene 2.3.0. It would appear that there is a variable
> > > conflict that is only present when utilizing Lucene
> 2.3.0, refCount,
> > > the reference counter is used in JackRabbit's
> > > CachingMultiIndexSearch and in Lucene's IndexReader.
> >
> > Think you mean CachingMultiIndexReader
> >
> > >
> > > If you make the following changes to CachingMultiIndexSearch the
> > > issue appears to be resolved:
> > > * Comment out line 64 `private int refCount = 1`
> > > * Replace line 159 with `incRef()`
> > > * Replace lines 168-170 with `super.doClose()`
> > >
> > > IndexReader 2.3.0 has the variable refCount and this is
> the variable
> > > that is checked and results in the thrown exception.
> > > CachingMultiIndexSearch has the same variable and increments it
> > > accordingly, failing to increment the parent class's variable.
> >
> > The same variable? It is a private member in
> CachingMultiIndexReader.
> > Anyway, your solution seems to me like just removing
> refCount at all,
> > making it very well possible that your indexreader needs to
> be opened
> > over and over again, or that it is never actually garbage
> collected (I
> > am guessing because i am not sure about the exact
> behavior). I haven't
> > tested your solution, but I am quite confident it is not correct.
> >
> > Also, Jackrabbit 1.4 is released with a dependency on Lucene 2.2.0.
> > Perhaps the closing of an IndexReader in lucene has
> changed. Anyway,
> > can you confirm that you have the same problem with lucene 2.2.0?
> > Lucene 2.3.0 will probably be used in the the next jr
> version, so if
> > it would be a problem, it will certainly be fixed then
> >
> > -Ard
> >
> >
> > >
> > > Hope that helps,
> > > Sean
> > >
> > >
> > >
> > > On Jan 29, 2008 3:23 AM, Ard Schrijvers
> <a....@hippo.nl> wrote:
> > > > Hello,
> > > >
> > > > pls do send the test case (do you have a unit test??)
> > > >
> > > > Regards Ard
> > > >
> > > >
> > > > > Hi,
> > > > >
> > > > > I have the same issue. It seems that the only XPATH
> query that
> > > > > is working is "//*". Anything else will fail with the same
> > > exception. I
> > > > > can pack a test case (libraries included) and send it.
> > > > >
> > > > > Julien
> > > > > --
> > > > > View this message in context:
> > > > > http://www.nabble.com/Lucene-AlreadyClosedException-tp15049132
> > > > > p15152752.html
> > > > > Sent from the Jackrabbit - Users mailing list archive at
> > > Nabble.com.
> > > > >
> > > > >
> > > >
> > >
> >
>
Re: Lucene AlreadyClosedException
Posted by Sean Callan <se...@gmail.com>.
I apologize, I did in fact mean CachingMultiIndexReader.
CachingMultiIndexReader is a child class of IndexReader and therefore
I use the refCount that can be found in IndexReader. It is this
variable that is being checked in the "ensureOpen" function which is
responsible for this exception being thrown. If you step through the
flow you'll notice that the value is never incremented in IndexReader
but rather the second refCount created in CachingMultiIndexReader is
incremented. So yes, my solution does result in the removal of
refCount being created as a private variable in the child of
IndexReader. There seems to be no reason for these duplicates and
with them Lucene 2.3.0 does not seem to function.
I have not tried 2.2.0 and will continue to utilize 2.3.0 as it is the
latest version of Lucene and seems to function properly with this
change. For testing purposes I will try and look through 2.2.0 to see
if there is still a conflict with the refCount variable.
Why wait to fix an issue that has been identified early on?
On Jan 29, 2008 11:13 AM, Ard Schrijvers <a....@hippo.nl> wrote:
> Hello,
>
> >
> > Hi,
> >
> > I was able to track this issue down in my scenario and I'll
> > provide a diff later today. In my case I am using Websphere
> > 6.1 and Lucene 2.3.0. It would appear that there is a
> > variable conflict that is only present when utilizing Lucene
> > 2.3.0, refCount, the reference counter is used in
> > JackRabbit's CachingMultiIndexSearch and in Lucene's IndexReader.
>
> Think you mean CachingMultiIndexReader
>
> >
> > If you make the following changes to CachingMultiIndexSearch
> > the issue appears to be resolved:
> > * Comment out line 64 `private int refCount = 1`
> > * Replace line 159 with `incRef()`
> > * Replace lines 168-170 with `super.doClose()`
> >
> > IndexReader 2.3.0 has the variable refCount and this is the
> > variable that is checked and results in the thrown exception.
> > CachingMultiIndexSearch has the same variable and increments
> > it accordingly, failing to increment the parent class's variable.
>
> The same variable? It is a private member in CachingMultiIndexReader.
> Anyway, your solution seems to me like just removing refCount at all,
> making it very well possible that your indexreader needs to be opened
> over and over again, or that it is never actually garbage collected (I
> am guessing because i am not sure about the exact behavior). I haven't
> tested your solution, but I am quite confident it is not correct.
>
> Also, Jackrabbit 1.4 is released with a dependency on Lucene 2.2.0.
> Perhaps the closing of an IndexReader in lucene has changed. Anyway, can
> you confirm that you have the same problem with lucene 2.2.0? Lucene
> 2.3.0 will probably be used in the the next jr version, so if it would
> be a problem, it will certainly be fixed then
>
> -Ard
>
>
> >
> > Hope that helps,
> > Sean
> >
> >
> >
> > On Jan 29, 2008 3:23 AM, Ard Schrijvers <a....@hippo.nl> wrote:
> > > Hello,
> > >
> > > pls do send the test case (do you have a unit test??)
> > >
> > > Regards Ard
> > >
> > >
> > > > Hi,
> > > >
> > > > I have the same issue. It seems that the only XPATH query that is
> > > > working is "//*". Anything else will fail with the same
> > exception. I
> > > > can pack a test case (libraries included) and send it.
> > > >
> > > > Julien
> > > > --
> > > > View this message in context:
> > > > http://www.nabble.com/Lucene-AlreadyClosedException-tp15049132
> > > > p15152752.html
> > > > Sent from the Jackrabbit - Users mailing list archive at
> > Nabble.com.
> > > >
> > > >
> > >
> >
>
RE: Lucene AlreadyClosedException
Posted by Ard Schrijvers <a....@hippo.nl>.
Hello,
>
> Hi,
>
> I was able to track this issue down in my scenario and I'll
> provide a diff later today. In my case I am using Websphere
> 6.1 and Lucene 2.3.0. It would appear that there is a
> variable conflict that is only present when utilizing Lucene
> 2.3.0, refCount, the reference counter is used in
> JackRabbit's CachingMultiIndexSearch and in Lucene's IndexReader.
Think you mean CachingMultiIndexReader
>
> If you make the following changes to CachingMultiIndexSearch
> the issue appears to be resolved:
> * Comment out line 64 `private int refCount = 1`
> * Replace line 159 with `incRef()`
> * Replace lines 168-170 with `super.doClose()`
>
> IndexReader 2.3.0 has the variable refCount and this is the
> variable that is checked and results in the thrown exception.
> CachingMultiIndexSearch has the same variable and increments
> it accordingly, failing to increment the parent class's variable.
The same variable? It is a private member in CachingMultiIndexReader.
Anyway, your solution seems to me like just removing refCount at all,
making it very well possible that your indexreader needs to be opened
over and over again, or that it is never actually garbage collected (I
am guessing because i am not sure about the exact behavior). I haven't
tested your solution, but I am quite confident it is not correct.
Also, Jackrabbit 1.4 is released with a dependency on Lucene 2.2.0.
Perhaps the closing of an IndexReader in lucene has changed. Anyway, can
you confirm that you have the same problem with lucene 2.2.0? Lucene
2.3.0 will probably be used in the the next jr version, so if it would
be a problem, it will certainly be fixed then
-Ard
>
> Hope that helps,
> Sean
>
>
>
> On Jan 29, 2008 3:23 AM, Ard Schrijvers <a....@hippo.nl> wrote:
> > Hello,
> >
> > pls do send the test case (do you have a unit test??)
> >
> > Regards Ard
> >
> >
> > > Hi,
> > >
> > > I have the same issue. It seems that the only XPATH query that is
> > > working is "//*". Anything else will fail with the same
> exception. I
> > > can pack a test case (libraries included) and send it.
> > >
> > > Julien
> > > --
> > > View this message in context:
> > > http://www.nabble.com/Lucene-AlreadyClosedException-tp15049132
> > > p15152752.html
> > > Sent from the Jackrabbit - Users mailing list archive at
> Nabble.com.
> > >
> > >
> >
>
Re: Lucene AlreadyClosedException
Posted by Sean Callan <se...@gmail.com>.
Hi,
I was able to track this issue down in my scenario and I'll provide a
diff later today. In my case I am using Websphere 6.1 and Lucene
2.3.0. It would appear that there is a variable conflict that is only
present when utilizing Lucene 2.3.0, refCount, the reference counter
is used in JackRabbit's CachingMultiIndexSearch and in Lucene's
IndexReader.
If you make the following changes to CachingMultiIndexSearch the issue
appears to be resolved:
* Comment out line 64 `private int refCount = 1`
* Replace line 159 with `incRef()`
* Replace lines 168-170 with `super.doClose()`
IndexReader 2.3.0 has the variable refCount and this is the variable
that is checked and results in the thrown exception.
CachingMultiIndexSearch has the same variable and increments it
accordingly, failing to increment the parent class's variable.
Hope that helps,
Sean
On Jan 29, 2008 3:23 AM, Ard Schrijvers <a....@hippo.nl> wrote:
> Hello,
>
> pls do send the test case (do you have a unit test??)
>
> Regards Ard
>
>
> > Hi,
> >
> > I have the same issue. It seems that the only XPATH query
> > that is working is "//*". Anything else will fail with the
> > same exception. I can pack a test case (libraries included)
> > and send it.
> >
> > Julien
> > --
> > View this message in context:
> > http://www.nabble.com/Lucene-AlreadyClosedException-tp15049132
> > p15152752.html
> > Sent from the Jackrabbit - Users mailing list archive at Nabble.com.
> >
> >
>
RE: Lucene AlreadyClosedException
Posted by Ard Schrijvers <a....@hippo.nl>.
Hello,
pls do send the test case (do you have a unit test??)
Regards Ard
> Hi,
>
> I have the same issue. It seems that the only XPATH query
> that is working is "//*". Anything else will fail with the
> same exception. I can pack a test case (libraries included)
> and send it.
>
> Julien
> --
> View this message in context:
> http://www.nabble.com/Lucene-AlreadyClosedException-tp15049132
> p15152752.html
> Sent from the Jackrabbit - Users mailing list archive at Nabble.com.
>
>
RE: Lucene AlreadyClosedException
Posted by julien herfurth <jh...@gmail.com>.
Hi,
I have the same issue. It seems that the only XPATH query that is working is
"//*". Anything else will fail with the same exception. I can pack a test
case (libraries included) and send it.
Julien
--
View this message in context: http://www.nabble.com/Lucene-AlreadyClosedException-tp15049132p15152752.html
Sent from the Jackrabbit - Users mailing list archive at Nabble.com.
RE: Lucene AlreadyClosedException
Posted by Ard Schrijvers <a....@hippo.nl>.
Hello Sean,
> Hi guys,
>
> I just upgraded to 1.4 and I'm suddenly getting an exception
> that hadn't previously occurred. It's being thrown when the
> following Xpath is executed :
> "//webfmis:role[@webfmis:general='true']"
>
> org.apache.lucene.store.AlreadyClosedException: this
> IndexReader is closed
Can you try to simulate your situation in a unit test and see if you can
reproduce it there. Would make it easier for us to find the issue if we
have this test, because I haven't seen it before
-Ard
>
> Any idea what is causing this? I tried searching for related
> issue but have not found any.
>
> Thanks,
> Sean
>