You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@jena.apache.org by Site Guo <gu...@hotmail.com> on 2013/08/08 18:41:26 UTC
Dataset cannot be deleted completely during run time.
Hi, GuysI deleted a dataset by using FileUtils.deleteDirectory to delete the directory which holds the dataset. The directory got deleted and the dataset did not exist. There was no problem. However, the problem arises when the program keeps running and creates another dataset with the same name (namely, creates the same directory). All the models and the inside triplets still exist in the dataset. Although, I've checked that the directory did get deleted during the running time.
Please shed some light here. Which Jena API Method can we use to completely delete a dataset?
Thanks,Victor
RE: Dataset cannot be deleted completely during run time.
Posted by David Jordan <Da...@sas.com>.
Thanks, this is less of a concern.
-----Original Message-----
From: Andy Seaborne [mailto:andy@apache.org]
Sent: Friday, August 09, 2013 11:27 AM
To: users@jena.apache.org
Subject: Re: Dataset cannot be deleted completely during run time.
On 09/08/13 14:48, David Jordan wrote:
> Is this the only known issue with Windows/64bit ? So this means that
> you cannot completely release a particular dataset from the in-memory
> cache and continue to run the JVM, which you may want to do if you
> have a set of separate datasets and you would want to dynamically open
> and close them in a running process? This could be a reason to decide
> that Windows/64bit is not an appropriate platform on which to run TDB
> if you have such a situation. I'd like to get this clarified.
Only issue I know of.
You an free up the in-memory in-heap resources - you can't delete the files on disk.
You can just have them all open. The OS will do it's thing and page out unused stuff. On 64 bit, not much goes in heap space, most is in the index memory mapped files (which are no part of the heap).
You can force TDB to run in direct mode as is used on 32 bit systems (memory mapped files are limited in usefulness there because only a maximum 1.5G is addressable).
SystemTDB.setFileMode
which must be called before any other TDB operation is called.
Apparently, there is no a great deal of difference in performance on Windows (I haven't tried - it has been reported). On Linux/Unix it does make a significant difference ... but then it works properly on those systems!
http://bugs.sun.com/view_bug.do?bug_id=4715154
There are various variations on this as well - this one is just one example. There are implementation tricks that are claimed to work on Sun JVMs.
Andy
>
> -----Original Message----- From: Andy Seaborne
> [mailto:andy@apache.org] Sent: Thursday, August 08, 2013 5:36 PM To:
> users@jena.apache.org Subject: Re: Dataset cannot be deleted
> completely during run time.
>
> TDBFactory.release(location) ;
>
> This does not work on Windows/64bit.
>
> Andy
>
> On 08/08/13 17:46, Kurt Landen wrote:
>> "Deleting files behind TDB's back can only be done with great care.
>> You must use StoredConnection.release() to expel it from the in-JVM
>> cache. Then delete the files, then reopen it the database.
>>
>> Otherwise, the caches are still there and some of your data is still
>> around. Eventually something will go wrong. Such data is not
>> persistent.
>>
>> This does not work on Windows/64bit. This is a well-documented issue
>> on MS Windows with memory mapped files." - Andy Seaborne
>>
>>
>> On Thu, Aug 8, 2013 at 11:41 AM, Site Guo <gu...@hotmail.com>
>> wrote:
>>
>>> Hi, GuysI deleted a dataset by using FileUtils.deleteDirectory to
>>> delete the directory which holds the dataset. The directory got
>>> deleted and the dataset did not exist. There was no problem.
>>> However, the problem arises when the program keeps running and
>>> creates another dataset with the same name (namely, creates the same
>>> directory). All the models and the inside triplets still exist in
>>> the dataset. Although, I've checked that the directory did get
>>> deleted during the running time. Please shed some light here. Which
>>> Jena API Method can we use to completely delete a dataset?
>>> Thanks,Victor
>>
>
>
>
Re: Dataset cannot be deleted completely during run time.
Posted by Andy Seaborne <an...@apache.org>.
On 09/08/13 14:48, David Jordan wrote:
> Is this the only known issue with Windows/64bit ? So this means that
> you cannot completely release a particular dataset from the in-memory
> cache and continue to run the JVM, which you may want to do if you
> have a set of separate datasets and you would want to dynamically
> open and close them in a running process? This could be a reason to
> decide that Windows/64bit is not an appropriate platform on which to
> run TDB if you have such a situation. I'd like to get this
> clarified.
Only issue I know of.
You an free up the in-memory in-heap resources - you can't delete the
files on disk.
You can just have them all open. The OS will do it's thing and page out
unused stuff. On 64 bit, not much goes in heap space, most is in the
index memory mapped files (which are no part of the heap).
You can force TDB to run in direct mode as is used on 32 bit systems
(memory mapped files are limited in usefulness there because only a
maximum 1.5G is addressable).
SystemTDB.setFileMode
which must be called before any other TDB operation is called.
Apparently, there is no a great deal of difference in performance on
Windows (I haven't tried - it has been reported). On Linux/Unix it does
make a significant difference ... but then it works properly on those
systems!
http://bugs.sun.com/view_bug.do?bug_id=4715154
There are various variations on this as well - this one is just one
example. There are implementation tricks that are claimed to work on Sun
JVMs.
Andy
>
> -----Original Message----- From: Andy Seaborne
> [mailto:andy@apache.org] Sent: Thursday, August 08, 2013 5:36 PM To:
> users@jena.apache.org Subject: Re: Dataset cannot be deleted
> completely during run time.
>
> TDBFactory.release(location) ;
>
> This does not work on Windows/64bit.
>
> Andy
>
> On 08/08/13 17:46, Kurt Landen wrote:
>> "Deleting files behind TDB's back can only be done with great
>> care. You must use StoredConnection.release() to expel it from the
>> in-JVM cache. Then delete the files, then reopen it the database.
>>
>> Otherwise, the caches are still there and some of your data is
>> still around. Eventually something will go wrong. Such data is
>> not persistent.
>>
>> This does not work on Windows/64bit. This is a well-documented
>> issue on MS Windows with memory mapped files." - Andy Seaborne
>>
>>
>> On Thu, Aug 8, 2013 at 11:41 AM, Site Guo <gu...@hotmail.com>
>> wrote:
>>
>>> Hi, GuysI deleted a dataset by using FileUtils.deleteDirectory
>>> to delete the directory which holds the dataset. The directory
>>> got deleted and the dataset did not exist. There was no problem.
>>> However, the problem arises when the program keeps running and
>>> creates another dataset with the same name (namely, creates the
>>> same directory). All the models and the inside triplets still
>>> exist in the dataset. Although, I've checked that the directory
>>> did get deleted during the running time. Please shed some light
>>> here. Which Jena API Method can we use to completely delete a
>>> dataset? Thanks,Victor
>>
>
>
>
RE: Dataset cannot be deleted completely during run time.
Posted by David Jordan <Da...@sas.com>.
Is this the only known issue with Windows/64bit ? So this means that you cannot completely release a particular dataset from the in-memory cache and continue to run the JVM, which you may want to do if you have a set of separate datasets and you would want to dynamically open and close them in a running process? This could be a reason to decide that Windows/64bit is not an appropriate platform on which to run TDB if you have such a situation. I'd like to get this clarified.
-----Original Message-----
From: Andy Seaborne [mailto:andy@apache.org]
Sent: Thursday, August 08, 2013 5:36 PM
To: users@jena.apache.org
Subject: Re: Dataset cannot be deleted completely during run time.
TDBFactory.release(location) ;
This does not work on Windows/64bit.
Andy
On 08/08/13 17:46, Kurt Landen wrote:
> "Deleting files behind TDB's back can only be done with great care.
> You must use StoredConnection.release() to expel it from the in-JVM
> cache. Then delete the files, then reopen it the database.
>
> Otherwise, the caches are still there and some of your data is still
> around. Eventually something will go wrong. Such data is not persistent.
>
> This does not work on Windows/64bit. This is a well-documented issue
> on MS Windows with memory mapped files." - Andy Seaborne
>
>
> On Thu, Aug 8, 2013 at 11:41 AM, Site Guo <gu...@hotmail.com> wrote:
>
>> Hi, GuysI deleted a dataset by using FileUtils.deleteDirectory to
>> delete the directory which holds the dataset. The directory got
>> deleted and the dataset did not exist. There was no problem. However,
>> the problem arises when the program keeps running and creates another
>> dataset with the same name (namely, creates the same directory). All
>> the models and the inside triplets still exist in the dataset.
>> Although, I've checked that the directory did get deleted during the running time.
>> Please shed some light here. Which Jena API Method can we use to
>> completely delete a dataset?
>> Thanks,Victor
>
Re: Dataset cannot be deleted completely during run time.
Posted by Andy Seaborne <an...@apache.org>.
TDBFactory.release(location) ;
This does not work on Windows/64bit.
Andy
On 08/08/13 17:46, Kurt Landen wrote:
> "Deleting files behind TDB's back can only be done with great care. You
> must use StoredConnection.release() to expel it from the in-JVM cache. Then
> delete the files, then reopen it the database.
>
> Otherwise, the caches are still there and some of your data is still
> around. Eventually something will go wrong. Such data is not persistent.
>
> This does not work on Windows/64bit. This is a well-documented issue on MS
> Windows with memory mapped files." - Andy Seaborne
>
>
> On Thu, Aug 8, 2013 at 11:41 AM, Site Guo <gu...@hotmail.com> wrote:
>
>> Hi, GuysI deleted a dataset by using FileUtils.deleteDirectory to delete
>> the directory which holds the dataset. The directory got deleted and the
>> dataset did not exist. There was no problem. However, the problem arises
>> when the program keeps running and creates another dataset with the same
>> name (namely, creates the same directory). All the models and the inside
>> triplets still exist in the dataset. Although, I've checked that the
>> directory did get deleted during the running time.
>> Please shed some light here. Which Jena API Method can we use to
>> completely delete a dataset?
>> Thanks,Victor
>
Re: Dataset cannot be deleted completely during run time.
Posted by Kurt Landen <st...@gmail.com>.
"Deleting files behind TDB's back can only be done with great care. You
must use StoredConnection.release() to expel it from the in-JVM cache. Then
delete the files, then reopen it the database.
Otherwise, the caches are still there and some of your data is still
around. Eventually something will go wrong. Such data is not persistent.
This does not work on Windows/64bit. This is a well-documented issue on MS
Windows with memory mapped files." - Andy Seaborne
On Thu, Aug 8, 2013 at 11:41 AM, Site Guo <gu...@hotmail.com> wrote:
> Hi, GuysI deleted a dataset by using FileUtils.deleteDirectory to delete
> the directory which holds the dataset. The directory got deleted and the
> dataset did not exist. There was no problem. However, the problem arises
> when the program keeps running and creates another dataset with the same
> name (namely, creates the same directory). All the models and the inside
> triplets still exist in the dataset. Although, I've checked that the
> directory did get deleted during the running time.
> Please shed some light here. Which Jena API Method can we use to
> completely delete a dataset?
> Thanks,Victor