You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@jena.apache.org by Mikael Pesonen <mi...@lingsoft.fi> on 2023/10/05 11:22:07 UTC
Re: Jena hangs on deleted files
We have the same issue now, here's a truncated lsof:
https://pastebin.com/rkUvEK0f
So Data-0001 size was 86G and after compact with delete, Data-0002 was
8G. Data-0001 is not released although the folder doesn't exist anymore.
So df shows they take about 100G. Another issue is of course the
bloating of data folder to 10x. I guess that can't be prevented?
On 12/09/2023 12.12, Rob @ DNR wrote:
> Well, yes, there shouldn’t be, but that wasn’t what Andy suggested/asked.
>
> Have you verified that nothing else is holding references to those files in any way e.g.
>
> lsof | grep /path/to/your/db
>
> And checked that only a single Java process is listed in the output?
>
> We don’t know your deployment environment, it could be some mundane background process (e.g. anti-virus, search indexer) running on your system, it could be a bug in the particular JVM you are using, or something else entirely but without any more details we can only guess at possibilities
>
> Another long shot is that it could be a hardware issue, if you’re running the database on an SSD it could be driver optimisation to not actually delete files until the holding process exits to avoid unnecessary write operations and prolong the life of the drive
>
> Rob
>
> From: Mikael Pesonen <mi...@lingsoft.fi>
> Date: Monday, 11 September 2023 at 12:17
> To: users@jena.apache.org <us...@jena.apache.org>
> Subject: Re: Jena hangs on deleted files
> There should not be other processes accessing the files. When jena is
> restarted, space from deleted files is released.
>
> On 09/09/2023 18.56, Andy Seaborne wrote:
>> This situation could be related to the other issues you're reported
>> (corrupted node tables) if some other Linux process,not nece3ssarily
>> java) is accessing the files.
>>
>> A process holding them open will stop them becoming recyclable by the OS.
>>
>> Andy
>>
>> On 08/09/2023 13:09, Mikael Pesonen wrote:
>>> Just on a command line (dev system)
>>>
>>> /usr/bin/java -Xmx8G -jar fuseki-server.jar --update --port 3030
>>> --config=../jena_config/fuseki_config.ttl
>>>
>>>
>>> On 08/09/2023 11.47, Andy Seaborne wrote:
>>>> In a container? As a VM?
>>>>
>>>> On 08/09/2023 07:36, Mikael Pesonen wrote:
>>>>> We are using Ubuntu.
>>>>>
>>>>> On Thu, 7 Sept 2023 at 16:33, Andy Seaborne <an...@apache.org> wrote:
>>>>>
>>>>>> Are the database files on a MS Windows filesystem?
>>>>>>
>>>>>> There is a long-standing Java issue that memory mapped files on MS
>>>>>> Windows do not get freed until the JVM exists.
>>>>>>
>>>>>> Various bugs in the OpenJDK bug database such as:
>>>>>>
>>>>>> https://bugs.openjdk.org/browse/JDK-4715154
>>>>>>
>>>>>> Andy
>>>>>>
>>>>>> On 07/09/2023 13:06, Mikael Pesonen wrote:
>>>>>>> We used deleteOld param. The 50 gigs are ghost files that are
>>>>>>> deleted
>>>>>>> but not released, that's what I meant by hanging on deleted files.
>>>>>>> Restarting jena releases them and now for example freed 50 gigs
>>>>>>> of space.
>>>>>>>
>>>>>>> On 07/09/2023 15.02, Øyvind Gjesdal wrote:
>>>>>>>> What does the content of the tdb2 folder look like?
>>>>>>>>
>>>>>>>> I think compact by default never deletes the old data, but you have
>>>>>>>> parameters for making it delete the old content on completion.
>>>>>>>>
>>>>>>>> `--deleteOld` can be supplied to the tdb2.tdbcompact command
>>>>>>>> line tool
>>>>>>>> and
>>>>>>>> `?deleteOld=true` can be supplied to the administration api when
>>>>>>>> calling
>>>>>>>> compact
>>>>>>>>
>>>>>> https://jena.apache.org/documentation/fuseki2/fuseki-server-protocol.html#compact
>>>>>>
>>>>>>>> You can also delete the Data-NNNN that isn't the latest one in the
>>>>>>>> database folder.
>>>>>>>>
>>>>>>>> Best regards,
>>>>>>>> Øyvind
>>>>>>>>
>>>>>>>> On Thu, Sep 7, 2023 at 1:33 PM Mikael Pesonen
>>>>>>>> <mi...@lingsoft.fi>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> After a while 25 gigs of files on data folder becomes 80 gigs
>>>>>>>>> of disk
>>>>>>>>> usage because Jena (4.6.1) doen't release files. Same with
>>>>>>>>> compact. Is
>>>>>>>>> this fixed in newer versions?
>>>>>>>>>
--
Lingsoft - 30 years of Leading Language Management
www.lingsoft.fi
Speech Applications - Language Management - Translation - Reader's and Writer's Tools - Text Tools - E-books and M-books
Mikael Pesonen
Semantic Technologies
e-mail: mikael.pesonen@lingsoft.fi
Tel. +358 2 279 3300
Time zone: GMT+2
Helsinki Office
Eteläranta 10
FI-00130 Helsinki
FINLAND
Turku Office
Kauppiaskatu 5 A
FI-20100 Turku
FINLAND