You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@jena.apache.org by Simon Helsen <sh...@ca.ibm.com> on 2012/06/02 00:35:32 UTC
StoreConnection.release() does not allow to connect again
Hi everyone,
before filing a bug, I want to make sure I am not doing something wrong. I
am running in the following problem.
Until I call StoreConnection.release(location) the TDB code holds a lock
on the file system of the index. I think this is the journal file mostly
because I can separately close a dataset.
However, when I call this and then later try to "reconnect", e.g. by
calling StoreConnection.make(location) or just creating a dataset, it
seems that the persisted data is not accessible anymore. I.e. it seems
that StoreConnection.release is doing more than just releasing the
location. It seems to persist something that makes the data go away.
Note that I am seeing this independent on whether I use transactions or
not (I have 2 versions around to compare the transactional and
non-transactional behavior)
This is troublesome. I need some way to "disconnect" from a TDB store (so
all file handles are released), but such that I can reconnect at a later
time. Am I doing something wrong? Or is this a bug?
thanks
Simon
PS: if it is a bug, it would be a showstopper for our 2.7.1 adoption
Re: StoreConnection.release() does not allow to connect again
Posted by Simon Helsen <sh...@ca.ibm.com>.
Andy
I managed to produce a test case. It took me a bit because at first,
things appeared to be working, so I kept looking back at what our code
does. The problem exists when the last action before
StoreConnection.release is a write activity (which is the case in our
production code)
I opened https://issues.apache.org/jira/browse/JENA-250
and attached my test case (just bind the latest libraries)
This is probably not hard to fix. I suspect I can work around the issue
given my initial failed attempt to write the test case. I'll have to look
at that, but of course, best would be if this can be fixed in 2.7.1
thanks
Simon
From:
Andy Seaborne <an...@apache.org>
To:
Simon Helsen/Toronto/IBM@IBMCA
Cc:
dev@jena.apache.org
Date:
06/05/2012 04:33 PM
Subject:
Re: StoreConnection.release() does not allow to connect again
Simon - do you use remote disk, or local disk?
Andy
On 04/06/12 17:30, Andy Seaborne wrote:
> On 04/06/12 14:53, Simon Helsen wrote:
>> Andy,
>>
>> Although I know someone tied it to Windows, I don't find it intuitive
>> here that this is a windows problem. I will try to verify the behavior
>> on linux, but it may take some time. As for the build, it is quite
>> recent. This was the tdb snapshot: jena-tdb-0.9.1-20120514.040618-2
>>
>> Just to clarify: are you saying that if you run
>> StoreConnection.release()on linux, that you can still connect
afterwards
>> and query the same store? Do you have junits which do this?
>
> I was asking to find out before I investigate (and I was away from my
> dev machines at the time).
>
> Your report did not include a clear example, only that release and
> reconnect, not, say, how you had used the dataset or whether there may
> be in-progress actions. As the mantra goes: "complete, minimal example"
> please.
>
> I sense you have one as your description hints at a sequence of actions.
> If you know the issue is windows specific, it's going to save me a lot
> of time investigating the wrong thing.
>
> So if you could provide the sequence of actions that cause the problem
> it would be helpful - ideally a test case and is it Windows specific?
>
> (Windows is different to Linux - you can't delete open files on Windows)
>
> Andy
>
>>
>> thanks
>>
>> Simon
>>
>>
>>
>>
>> From: Andy Seaborne <an...@apache.org>
>> To: dev@jena.apache.org
>> Date: 06/03/2012 06:01 AM
>> Subject: Re: StoreConnection.release() does not allow to connect again
>>
>>
>>
------------------------------------------------------------------------
>>
>>
>>
>> Simon,
>>
>> Which build of the snapshot are you running and are you saying it does
>> not happen on Linux?
>>
>> Andy
>> (not near a Windows machine for a few days)
>>
>> On 02/06/12 00:58, Simon Helsen wrote:
>> > thanks for the quick response Rob,
>> >
>> > yes, this is on windows, but I use direct I/O (deliberately) because
of
>> > the known problems with mapped I/O. I have not yet had a chance to
>> > follow-up on the suggestions earlier this month to work around the
>> problem
>> > with mapped I/O on windows.
>> >
>> > I think this is an entirely different matter though. It seems to me
>> that
>> > the Journal is open when I don't call StoreConnection.release(), but
>> when
>> > I do this, as I mentioned below, the persisted store is lost, i.e
>> even a
>> > new Java process cannot connect to it again (or rather, it can do
this,
>> > but thinks the store is empty). It seems to me that that is the
>> problem,
>> > not that the journal object holds on to its channel without an
explicit
>> > close. I actually tried to just close the journal, but then it
>> cannot be
>> > re-opened again within the same Java process.
>> >
>> > Simon
>> >
>> >
>> >
>> >
>> >
>> > From:
>> > Rob Vesse<rv...@yarcdata.com>
>> > To:
>> > "dev@jena.apache.org"<de...@jena.apache.org>
>> > Date:
>> > 06/01/2012 07:06 PM
>> > Subject:
>> > Re: StoreConnection.release() does not allow to connect again
>> >
>> >
>> >
>> > Hi Simon
>> >
>> > Are you running on Windows per chance?
>> >
>> > TDB uses memory mapped files for its data and there are some issues
>> around
>> > Windows releasing memory mapped files. I thought this was mostly
>> confined
>> > to the case where you were trying to delete such files but maybe it
>> also
>> > applies in this case?
>> >
>> > Someone like Andy/Paolo who is more knowledgeable about the
>> internals of
>> > TDB would be better placed to answer your question.
>> >
>> > The details of your environment (like OS) will likely be helpful in
>> > tracking down whether this is a bug or not
>> >
>> > Rob
>> >
>> >
>> > On 6/1/12 3:35 PM, "Simon Helsen"<sh...@ca.ibm.com> wrote:
>> >
>> >> Hi everyone,
>> >>
>> >> before filing a bug, I want to make sure I am not doing something
>> wrong.
>> >> I
>> >> am running in the following problem.
>> >>
>> >> Until I call StoreConnection.release(location) the TDB code holds a
>> lock
>> >> on the file system of the index. I think this is the journal file
>> mostly
>> >> because I can separately close a dataset.
>> >>
>> >> However, when I call this and then later try to "reconnect", e.g. by
>> >> calling StoreConnection.make(location) or just creating a dataset,
it
>> >> seems that the persisted data is not accessible anymore. I.e. it
seems
>> >> that StoreConnection.release is doing more than just releasing the
>> >> location. It seems to persist something that makes the data go away.
>> >>
>> >> Note that I am seeing this independent on whether I use
>> transactions or
>> >> not (I have 2 versions around to compare the transactional and
>> >> non-transactional behavior)
>> >>
>> >> This is troublesome. I need some way to "disconnect" from a TDB
>> store (so
>> >> all file handles are released), but such that I can reconnect at a
>> later
>> >> time. Am I doing something wrong? Or is this a bug?
>> >>
>> >> thanks
>> >>
>> >> Simon
>> >>
>> >> PS: if it is a bug, it would be a showstopper for our 2.7.1 adoption
>> >
>> >
>> >
>> >
>>
>>
>>
>
Re: StoreConnection.release() does not allow to connect again
Posted by Simon Helsen <sh...@ca.ibm.com>.
local
I am about to start writing a simple test program to illustrate the point.
FYI, we first reproduced the problem on a large installation (because I
didn't suspect a problem here and we didn't have a junit to cover this).
What was interesting was that after we had released the store, the index
appeared empty (and I checked this by running tdbDump on it), so we
reindexed everything again. The size on disk had doubled after this
operation suggesting that the data was all there and persisted, just not
"reachable"
Simon
From:
Andy Seaborne <an...@apache.org>
To:
Simon Helsen/Toronto/IBM@IBMCA
Cc:
dev@jena.apache.org
Date:
06/05/2012 04:33 PM
Subject:
Re: StoreConnection.release() does not allow to connect again
Simon - do you use remote disk, or local disk?
Andy
On 04/06/12 17:30, Andy Seaborne wrote:
> On 04/06/12 14:53, Simon Helsen wrote:
>> Andy,
>>
>> Although I know someone tied it to Windows, I don't find it intuitive
>> here that this is a windows problem. I will try to verify the behavior
>> on linux, but it may take some time. As for the build, it is quite
>> recent. This was the tdb snapshot: jena-tdb-0.9.1-20120514.040618-2
>>
>> Just to clarify: are you saying that if you run
>> StoreConnection.release()on linux, that you can still connect
afterwards
>> and query the same store? Do you have junits which do this?
>
> I was asking to find out before I investigate (and I was away from my
> dev machines at the time).
>
> Your report did not include a clear example, only that release and
> reconnect, not, say, how you had used the dataset or whether there may
> be in-progress actions. As the mantra goes: "complete, minimal example"
> please.
>
> I sense you have one as your description hints at a sequence of actions.
> If you know the issue is windows specific, it's going to save me a lot
> of time investigating the wrong thing.
>
> So if you could provide the sequence of actions that cause the problem
> it would be helpful - ideally a test case and is it Windows specific?
>
> (Windows is different to Linux - you can't delete open files on Windows)
>
> Andy
>
>>
>> thanks
>>
>> Simon
>>
>>
>>
>>
>> From: Andy Seaborne <an...@apache.org>
>> To: dev@jena.apache.org
>> Date: 06/03/2012 06:01 AM
>> Subject: Re: StoreConnection.release() does not allow to connect again
>>
>>
>>
------------------------------------------------------------------------
>>
>>
>>
>> Simon,
>>
>> Which build of the snapshot are you running and are you saying it does
>> not happen on Linux?
>>
>> Andy
>> (not near a Windows machine for a few days)
>>
>> On 02/06/12 00:58, Simon Helsen wrote:
>> > thanks for the quick response Rob,
>> >
>> > yes, this is on windows, but I use direct I/O (deliberately) because
of
>> > the known problems with mapped I/O. I have not yet had a chance to
>> > follow-up on the suggestions earlier this month to work around the
>> problem
>> > with mapped I/O on windows.
>> >
>> > I think this is an entirely different matter though. It seems to me
>> that
>> > the Journal is open when I don't call StoreConnection.release(), but
>> when
>> > I do this, as I mentioned below, the persisted store is lost, i.e
>> even a
>> > new Java process cannot connect to it again (or rather, it can do
this,
>> > but thinks the store is empty). It seems to me that that is the
>> problem,
>> > not that the journal object holds on to its channel without an
explicit
>> > close. I actually tried to just close the journal, but then it
>> cannot be
>> > re-opened again within the same Java process.
>> >
>> > Simon
>> >
>> >
>> >
>> >
>> >
>> > From:
>> > Rob Vesse<rv...@yarcdata.com>
>> > To:
>> > "dev@jena.apache.org"<de...@jena.apache.org>
>> > Date:
>> > 06/01/2012 07:06 PM
>> > Subject:
>> > Re: StoreConnection.release() does not allow to connect again
>> >
>> >
>> >
>> > Hi Simon
>> >
>> > Are you running on Windows per chance?
>> >
>> > TDB uses memory mapped files for its data and there are some issues
>> around
>> > Windows releasing memory mapped files. I thought this was mostly
>> confined
>> > to the case where you were trying to delete such files but maybe it
>> also
>> > applies in this case?
>> >
>> > Someone like Andy/Paolo who is more knowledgeable about the
>> internals of
>> > TDB would be better placed to answer your question.
>> >
>> > The details of your environment (like OS) will likely be helpful in
>> > tracking down whether this is a bug or not
>> >
>> > Rob
>> >
>> >
>> > On 6/1/12 3:35 PM, "Simon Helsen"<sh...@ca.ibm.com> wrote:
>> >
>> >> Hi everyone,
>> >>
>> >> before filing a bug, I want to make sure I am not doing something
>> wrong.
>> >> I
>> >> am running in the following problem.
>> >>
>> >> Until I call StoreConnection.release(location) the TDB code holds a
>> lock
>> >> on the file system of the index. I think this is the journal file
>> mostly
>> >> because I can separately close a dataset.
>> >>
>> >> However, when I call this and then later try to "reconnect", e.g. by
>> >> calling StoreConnection.make(location) or just creating a dataset,
it
>> >> seems that the persisted data is not accessible anymore. I.e. it
seems
>> >> that StoreConnection.release is doing more than just releasing the
>> >> location. It seems to persist something that makes the data go away.
>> >>
>> >> Note that I am seeing this independent on whether I use
>> transactions or
>> >> not (I have 2 versions around to compare the transactional and
>> >> non-transactional behavior)
>> >>
>> >> This is troublesome. I need some way to "disconnect" from a TDB
>> store (so
>> >> all file handles are released), but such that I can reconnect at a
>> later
>> >> time. Am I doing something wrong? Or is this a bug?
>> >>
>> >> thanks
>> >>
>> >> Simon
>> >>
>> >> PS: if it is a bug, it would be a showstopper for our 2.7.1 adoption
>> >
>> >
>> >
>> >
>>
>>
>>
>
Re: StoreConnection.release() does not allow to connect again
Posted by Andy Seaborne <an...@apache.org>.
Simon - do you use remote disk, or local disk?
Andy
On 04/06/12 17:30, Andy Seaborne wrote:
> On 04/06/12 14:53, Simon Helsen wrote:
>> Andy,
>>
>> Although I know someone tied it to Windows, I don't find it intuitive
>> here that this is a windows problem. I will try to verify the behavior
>> on linux, but it may take some time. As for the build, it is quite
>> recent. This was the tdb snapshot: jena-tdb-0.9.1-20120514.040618-2
>>
>> Just to clarify: are you saying that if you run
>> StoreConnection.release()on linux, that you can still connect afterwards
>> and query the same store? Do you have junits which do this?
>
> I was asking to find out before I investigate (and I was away from my
> dev machines at the time).
>
> Your report did not include a clear example, only that release and
> reconnect, not, say, how you had used the dataset or whether there may
> be in-progress actions. As the mantra goes: "complete, minimal example"
> please.
>
> I sense you have one as your description hints at a sequence of actions.
> If you know the issue is windows specific, it's going to save me a lot
> of time investigating the wrong thing.
>
> So if you could provide the sequence of actions that cause the problem
> it would be helpful - ideally a test case and is it Windows specific?
>
> (Windows is different to Linux - you can't delete open files on Windows)
>
> Andy
>
>>
>> thanks
>>
>> Simon
>>
>>
>>
>>
>> From: Andy Seaborne <an...@apache.org>
>> To: dev@jena.apache.org
>> Date: 06/03/2012 06:01 AM
>> Subject: Re: StoreConnection.release() does not allow to connect again
>>
>>
>> ------------------------------------------------------------------------
>>
>>
>>
>> Simon,
>>
>> Which build of the snapshot are you running and are you saying it does
>> not happen on Linux?
>>
>> Andy
>> (not near a Windows machine for a few days)
>>
>> On 02/06/12 00:58, Simon Helsen wrote:
>> > thanks for the quick response Rob,
>> >
>> > yes, this is on windows, but I use direct I/O (deliberately) because of
>> > the known problems with mapped I/O. I have not yet had a chance to
>> > follow-up on the suggestions earlier this month to work around the
>> problem
>> > with mapped I/O on windows.
>> >
>> > I think this is an entirely different matter though. It seems to me
>> that
>> > the Journal is open when I don't call StoreConnection.release(), but
>> when
>> > I do this, as I mentioned below, the persisted store is lost, i.e
>> even a
>> > new Java process cannot connect to it again (or rather, it can do this,
>> > but thinks the store is empty). It seems to me that that is the
>> problem,
>> > not that the journal object holds on to its channel without an explicit
>> > close. I actually tried to just close the journal, but then it
>> cannot be
>> > re-opened again within the same Java process.
>> >
>> > Simon
>> >
>> >
>> >
>> >
>> >
>> > From:
>> > Rob Vesse<rv...@yarcdata.com>
>> > To:
>> > "dev@jena.apache.org"<de...@jena.apache.org>
>> > Date:
>> > 06/01/2012 07:06 PM
>> > Subject:
>> > Re: StoreConnection.release() does not allow to connect again
>> >
>> >
>> >
>> > Hi Simon
>> >
>> > Are you running on Windows per chance?
>> >
>> > TDB uses memory mapped files for its data and there are some issues
>> around
>> > Windows releasing memory mapped files. I thought this was mostly
>> confined
>> > to the case where you were trying to delete such files but maybe it
>> also
>> > applies in this case?
>> >
>> > Someone like Andy/Paolo who is more knowledgeable about the
>> internals of
>> > TDB would be better placed to answer your question.
>> >
>> > The details of your environment (like OS) will likely be helpful in
>> > tracking down whether this is a bug or not
>> >
>> > Rob
>> >
>> >
>> > On 6/1/12 3:35 PM, "Simon Helsen"<sh...@ca.ibm.com> wrote:
>> >
>> >> Hi everyone,
>> >>
>> >> before filing a bug, I want to make sure I am not doing something
>> wrong.
>> >> I
>> >> am running in the following problem.
>> >>
>> >> Until I call StoreConnection.release(location) the TDB code holds a
>> lock
>> >> on the file system of the index. I think this is the journal file
>> mostly
>> >> because I can separately close a dataset.
>> >>
>> >> However, when I call this and then later try to "reconnect", e.g. by
>> >> calling StoreConnection.make(location) or just creating a dataset, it
>> >> seems that the persisted data is not accessible anymore. I.e. it seems
>> >> that StoreConnection.release is doing more than just releasing the
>> >> location. It seems to persist something that makes the data go away.
>> >>
>> >> Note that I am seeing this independent on whether I use
>> transactions or
>> >> not (I have 2 versions around to compare the transactional and
>> >> non-transactional behavior)
>> >>
>> >> This is troublesome. I need some way to "disconnect" from a TDB
>> store (so
>> >> all file handles are released), but such that I can reconnect at a
>> later
>> >> time. Am I doing something wrong? Or is this a bug?
>> >>
>> >> thanks
>> >>
>> >> Simon
>> >>
>> >> PS: if it is a bug, it would be a showstopper for our 2.7.1 adoption
>> >
>> >
>> >
>> >
>>
>>
>>
>
Re: StoreConnection.release() does not allow to connect again
Posted by Andy Seaborne <an...@apache.org>.
On 04/06/12 14:53, Simon Helsen wrote:
> Andy,
>
> Although I know someone tied it to Windows, I don't find it intuitive
> here that this is a windows problem. I will try to verify the behavior
> on linux, but it may take some time. As for the build, it is quite
> recent. This was the tdb snapshot: jena-tdb-0.9.1-20120514.040618-2
>
> Just to clarify: are you saying that if you run
> StoreConnection.release()on linux, that you can still connect afterwards
> and query the same store? Do you have junits which do this?
I was asking to find out before I investigate (and I was away from my
dev machines at the time).
Your report did not include a clear example, only that release and
reconnect, not, say, how you had used the dataset or whether there may
be in-progress actions. As the mantra goes: "complete, minimal example"
please.
I sense you have one as your description hints at a sequence of actions.
If you know the issue is windows specific, it's going to save me a lot
of time investigating the wrong thing.
So if you could provide the sequence of actions that cause the problem
it would be helpful - ideally a test case and is it Windows specific?
(Windows is different to Linux - you can't delete open files on Windows)
Andy
>
> thanks
>
> Simon
>
>
>
>
> From: Andy Seaborne <an...@apache.org>
> To: dev@jena.apache.org
> Date: 06/03/2012 06:01 AM
> Subject: Re: StoreConnection.release() does not allow to connect again
>
>
> ------------------------------------------------------------------------
>
>
>
> Simon,
>
> Which build of the snapshot are you running and are you saying it does
> not happen on Linux?
>
> Andy
> (not near a Windows machine for a few days)
>
> On 02/06/12 00:58, Simon Helsen wrote:
> > thanks for the quick response Rob,
> >
> > yes, this is on windows, but I use direct I/O (deliberately) because of
> > the known problems with mapped I/O. I have not yet had a chance to
> > follow-up on the suggestions earlier this month to work around the
> problem
> > with mapped I/O on windows.
> >
> > I think this is an entirely different matter though. It seems to me that
> > the Journal is open when I don't call StoreConnection.release(), but when
> > I do this, as I mentioned below, the persisted store is lost, i.e even a
> > new Java process cannot connect to it again (or rather, it can do this,
> > but thinks the store is empty). It seems to me that that is the problem,
> > not that the journal object holds on to its channel without an explicit
> > close. I actually tried to just close the journal, but then it cannot be
> > re-opened again within the same Java process.
> >
> > Simon
> >
> >
> >
> >
> >
> > From:
> > Rob Vesse<rv...@yarcdata.com>
> > To:
> > "dev@jena.apache.org"<de...@jena.apache.org>
> > Date:
> > 06/01/2012 07:06 PM
> > Subject:
> > Re: StoreConnection.release() does not allow to connect again
> >
> >
> >
> > Hi Simon
> >
> > Are you running on Windows per chance?
> >
> > TDB uses memory mapped files for its data and there are some issues
> around
> > Windows releasing memory mapped files. I thought this was mostly confined
> > to the case where you were trying to delete such files but maybe it also
> > applies in this case?
> >
> > Someone like Andy/Paolo who is more knowledgeable about the internals of
> > TDB would be better placed to answer your question.
> >
> > The details of your environment (like OS) will likely be helpful in
> > tracking down whether this is a bug or not
> >
> > Rob
> >
> >
> > On 6/1/12 3:35 PM, "Simon Helsen"<sh...@ca.ibm.com> wrote:
> >
> >> Hi everyone,
> >>
> >> before filing a bug, I want to make sure I am not doing something wrong.
> >> I
> >> am running in the following problem.
> >>
> >> Until I call StoreConnection.release(location) the TDB code holds a lock
> >> on the file system of the index. I think this is the journal file mostly
> >> because I can separately close a dataset.
> >>
> >> However, when I call this and then later try to "reconnect", e.g. by
> >> calling StoreConnection.make(location) or just creating a dataset, it
> >> seems that the persisted data is not accessible anymore. I.e. it seems
> >> that StoreConnection.release is doing more than just releasing the
> >> location. It seems to persist something that makes the data go away.
> >>
> >> Note that I am seeing this independent on whether I use transactions or
> >> not (I have 2 versions around to compare the transactional and
> >> non-transactional behavior)
> >>
> >> This is troublesome. I need some way to "disconnect" from a TDB
> store (so
> >> all file handles are released), but such that I can reconnect at a later
> >> time. Am I doing something wrong? Or is this a bug?
> >>
> >> thanks
> >>
> >> Simon
> >>
> >> PS: if it is a bug, it would be a showstopper for our 2.7.1 adoption
> >
> >
> >
> >
>
>
>
Re: StoreConnection.release() does not allow to connect again
Posted by Simon Helsen <sh...@ca.ibm.com>.
Andy,
Although I know someone tied it to Windows, I don't find it intuitive here
that this is a windows problem. I will try to verify the behavior on
linux, but it may take some time. As for the build, it is quite recent.
This was the tdb snapshot: jena-tdb-0.9.1-20120514.040618-2
Just to clarify: are you saying that if you run StoreConnection.release()
on linux, that you can still connect afterwards and query the same store?
Do you have junits which do this?
thanks
Simon
From:
Andy Seaborne <an...@apache.org>
To:
dev@jena.apache.org
Date:
06/03/2012 06:01 AM
Subject:
Re: StoreConnection.release() does not allow to connect again
Simon,
Which build of the snapshot are you running and are you saying it does
not happen on Linux?
Andy
(not near a Windows machine for a few days)
On 02/06/12 00:58, Simon Helsen wrote:
> thanks for the quick response Rob,
>
> yes, this is on windows, but I use direct I/O (deliberately) because of
> the known problems with mapped I/O. I have not yet had a chance to
> follow-up on the suggestions earlier this month to work around the
problem
> with mapped I/O on windows.
>
> I think this is an entirely different matter though. It seems to me that
> the Journal is open when I don't call StoreConnection.release(), but
when
> I do this, as I mentioned below, the persisted store is lost, i.e even a
> new Java process cannot connect to it again (or rather, it can do this,
> but thinks the store is empty). It seems to me that that is the problem,
> not that the journal object holds on to its channel without an explicit
> close. I actually tried to just close the journal, but then it cannot be
> re-opened again within the same Java process.
>
> Simon
>
>
>
>
>
> From:
> Rob Vesse<rv...@yarcdata.com>
> To:
> "dev@jena.apache.org"<de...@jena.apache.org>
> Date:
> 06/01/2012 07:06 PM
> Subject:
> Re: StoreConnection.release() does not allow to connect again
>
>
>
> Hi Simon
>
> Are you running on Windows per chance?
>
> TDB uses memory mapped files for its data and there are some issues
around
> Windows releasing memory mapped files. I thought this was mostly
confined
> to the case where you were trying to delete such files but maybe it also
> applies in this case?
>
> Someone like Andy/Paolo who is more knowledgeable about the internals of
> TDB would be better placed to answer your question.
>
> The details of your environment (like OS) will likely be helpful in
> tracking down whether this is a bug or not
>
> Rob
>
>
> On 6/1/12 3:35 PM, "Simon Helsen"<sh...@ca.ibm.com> wrote:
>
>> Hi everyone,
>>
>> before filing a bug, I want to make sure I am not doing something
wrong.
>> I
>> am running in the following problem.
>>
>> Until I call StoreConnection.release(location) the TDB code holds a
lock
>> on the file system of the index. I think this is the journal file
mostly
>> because I can separately close a dataset.
>>
>> However, when I call this and then later try to "reconnect", e.g. by
>> calling StoreConnection.make(location) or just creating a dataset, it
>> seems that the persisted data is not accessible anymore. I.e. it seems
>> that StoreConnection.release is doing more than just releasing the
>> location. It seems to persist something that makes the data go away.
>>
>> Note that I am seeing this independent on whether I use transactions or
>> not (I have 2 versions around to compare the transactional and
>> non-transactional behavior)
>>
>> This is troublesome. I need some way to "disconnect" from a TDB store
(so
>> all file handles are released), but such that I can reconnect at a
later
>> time. Am I doing something wrong? Or is this a bug?
>>
>> thanks
>>
>> Simon
>>
>> PS: if it is a bug, it would be a showstopper for our 2.7.1 adoption
>
>
>
>
Re: StoreConnection.release() does not allow to connect again
Posted by Andy Seaborne <an...@apache.org>.
Simon,
Which build of the snapshot are you running and are you saying it does
not happen on Linux?
Andy
(not near a Windows machine for a few days)
On 02/06/12 00:58, Simon Helsen wrote:
> thanks for the quick response Rob,
>
> yes, this is on windows, but I use direct I/O (deliberately) because of
> the known problems with mapped I/O. I have not yet had a chance to
> follow-up on the suggestions earlier this month to work around the problem
> with mapped I/O on windows.
>
> I think this is an entirely different matter though. It seems to me that
> the Journal is open when I don't call StoreConnection.release(), but when
> I do this, as I mentioned below, the persisted store is lost, i.e even a
> new Java process cannot connect to it again (or rather, it can do this,
> but thinks the store is empty). It seems to me that that is the problem,
> not that the journal object holds on to its channel without an explicit
> close. I actually tried to just close the journal, but then it cannot be
> re-opened again within the same Java process.
>
> Simon
>
>
>
>
>
> From:
> Rob Vesse<rv...@yarcdata.com>
> To:
> "dev@jena.apache.org"<de...@jena.apache.org>
> Date:
> 06/01/2012 07:06 PM
> Subject:
> Re: StoreConnection.release() does not allow to connect again
>
>
>
> Hi Simon
>
> Are you running on Windows per chance?
>
> TDB uses memory mapped files for its data and there are some issues around
> Windows releasing memory mapped files. I thought this was mostly confined
> to the case where you were trying to delete such files but maybe it also
> applies in this case?
>
> Someone like Andy/Paolo who is more knowledgeable about the internals of
> TDB would be better placed to answer your question.
>
> The details of your environment (like OS) will likely be helpful in
> tracking down whether this is a bug or not
>
> Rob
>
>
> On 6/1/12 3:35 PM, "Simon Helsen"<sh...@ca.ibm.com> wrote:
>
>> Hi everyone,
>>
>> before filing a bug, I want to make sure I am not doing something wrong.
>> I
>> am running in the following problem.
>>
>> Until I call StoreConnection.release(location) the TDB code holds a lock
>> on the file system of the index. I think this is the journal file mostly
>> because I can separately close a dataset.
>>
>> However, when I call this and then later try to "reconnect", e.g. by
>> calling StoreConnection.make(location) or just creating a dataset, it
>> seems that the persisted data is not accessible anymore. I.e. it seems
>> that StoreConnection.release is doing more than just releasing the
>> location. It seems to persist something that makes the data go away.
>>
>> Note that I am seeing this independent on whether I use transactions or
>> not (I have 2 versions around to compare the transactional and
>> non-transactional behavior)
>>
>> This is troublesome. I need some way to "disconnect" from a TDB store (so
>> all file handles are released), but such that I can reconnect at a later
>> time. Am I doing something wrong? Or is this a bug?
>>
>> thanks
>>
>> Simon
>>
>> PS: if it is a bug, it would be a showstopper for our 2.7.1 adoption
>
>
>
>
Re: StoreConnection.release() does not allow to connect again
Posted by Simon Helsen <sh...@ca.ibm.com>.
thanks for the quick response Rob,
yes, this is on windows, but I use direct I/O (deliberately) because of
the known problems with mapped I/O. I have not yet had a chance to
follow-up on the suggestions earlier this month to work around the problem
with mapped I/O on windows.
I think this is an entirely different matter though. It seems to me that
the Journal is open when I don't call StoreConnection.release(), but when
I do this, as I mentioned below, the persisted store is lost, i.e even a
new Java process cannot connect to it again (or rather, it can do this,
but thinks the store is empty). It seems to me that that is the problem,
not that the journal object holds on to its channel without an explicit
close. I actually tried to just close the journal, but then it cannot be
re-opened again within the same Java process.
Simon
From:
Rob Vesse <rv...@yarcdata.com>
To:
"dev@jena.apache.org" <de...@jena.apache.org>
Date:
06/01/2012 07:06 PM
Subject:
Re: StoreConnection.release() does not allow to connect again
Hi Simon
Are you running on Windows per chance?
TDB uses memory mapped files for its data and there are some issues around
Windows releasing memory mapped files. I thought this was mostly confined
to the case where you were trying to delete such files but maybe it also
applies in this case?
Someone like Andy/Paolo who is more knowledgeable about the internals of
TDB would be better placed to answer your question.
The details of your environment (like OS) will likely be helpful in
tracking down whether this is a bug or not
Rob
On 6/1/12 3:35 PM, "Simon Helsen" <sh...@ca.ibm.com> wrote:
>Hi everyone,
>
>before filing a bug, I want to make sure I am not doing something wrong.
>I
>am running in the following problem.
>
>Until I call StoreConnection.release(location) the TDB code holds a lock
>on the file system of the index. I think this is the journal file mostly
>because I can separately close a dataset.
>
>However, when I call this and then later try to "reconnect", e.g. by
>calling StoreConnection.make(location) or just creating a dataset, it
>seems that the persisted data is not accessible anymore. I.e. it seems
>that StoreConnection.release is doing more than just releasing the
>location. It seems to persist something that makes the data go away.
>
>Note that I am seeing this independent on whether I use transactions or
>not (I have 2 versions around to compare the transactional and
>non-transactional behavior)
>
>This is troublesome. I need some way to "disconnect" from a TDB store (so
>all file handles are released), but such that I can reconnect at a later
>time. Am I doing something wrong? Or is this a bug?
>
>thanks
>
>Simon
>
>PS: if it is a bug, it would be a showstopper for our 2.7.1 adoption
Re: StoreConnection.release() does not allow to connect again
Posted by Rob Vesse <rv...@yarcdata.com>.
Hi Simon
Are you running on Windows per chance?
TDB uses memory mapped files for its data and there are some issues around
Windows releasing memory mapped files. I thought this was mostly confined
to the case where you were trying to delete such files but maybe it also
applies in this case?
Someone like Andy/Paolo who is more knowledgeable about the internals of
TDB would be better placed to answer your question.
The details of your environment (like OS) will likely be helpful in
tracking down whether this is a bug or not
Rob
On 6/1/12 3:35 PM, "Simon Helsen" <sh...@ca.ibm.com> wrote:
>Hi everyone,
>
>before filing a bug, I want to make sure I am not doing something wrong.
>I
>am running in the following problem.
>
>Until I call StoreConnection.release(location) the TDB code holds a lock
>on the file system of the index. I think this is the journal file mostly
>because I can separately close a dataset.
>
>However, when I call this and then later try to "reconnect", e.g. by
>calling StoreConnection.make(location) or just creating a dataset, it
>seems that the persisted data is not accessible anymore. I.e. it seems
>that StoreConnection.release is doing more than just releasing the
>location. It seems to persist something that makes the data go away.
>
>Note that I am seeing this independent on whether I use transactions or
>not (I have 2 versions around to compare the transactional and
>non-transactional behavior)
>
>This is troublesome. I need some way to "disconnect" from a TDB store (so
>all file handles are released), but such that I can reconnect at a later
>time. Am I doing something wrong? Or is this a bug?
>
>thanks
>
>Simon
>
>PS: if it is a bug, it would be a showstopper for our 2.7.1 adoption