You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@jena.apache.org by Gaspar Bartalus <ba...@gmail.com> on 2022/11/23 08:39:25 UTC
Compaction failing
Hi Andy,
I've changed my email address, but I've raised this topic yesterday:
Executing POST request /$/compact/database_name?deleteOld=true sometimes
creates a task which is hanging, and after which the database seems
unreachable. The copy folders (Data-000_) are created, but the old ones are
not deleted when this happens. The logs also seem to indicate that the
process is started but not finished:
- 06:49:50 INFO Admin :: [192] Compact dataset /database_name
- 23/11/2022 8:49:50
06:49:50 INFO Server :: Task : 3 : Compact
- 23/11/2022 8:49:50
06:49:50 INFO Server :: [Task 3] starts : Compact
- 23/11/2022 8:49:50
06:49:50 INFO Compact :: [192] >>>> Start compact /database_name
We are using jena-fuseki 4.4.0 btw.
Gaspar
Re: Compaction failing
Posted by Gaspar Bartalus <ba...@gmail.com>.
Hi Andy,
We are actually planning to schedule compaction as a nightly job to run
once a day or every second day maybe, so we are not planning to use it
excessively or very often.
Thanks for the support.
Gaspar
On Thu, Nov 24, 2022 at 11:23 AM Andy Seaborne <an...@apache.org> wrote:
>
>
> On 24/11/2022 08:35, Gaspar Bartalus wrote:
> > Hi Andy,
> >
> > It seems that the issue occurs when I execute multiple compaction
> requests
> > in a short time period.
> > It probably runs out of resources in this case.
>
> That would be odd because if it did (e.g. heap) you'll get something in
> the log files and also it not be able to start later requests.
>
> > If I leave a minute between each request then it seems stable.
>
> The compact request schedules a compaction, it does not actually do it.
> So, yes, a gap will help.
>
> That does sound 4.6.1 could fix that.
>
> Andy
>
> We could add a "?sync=". The compaction would still scheduled and run on
> a separate thread but the request would wait for the compaction to
> finish. If the client gets bored and drops the connection before it
> returns success, the compaction continues.
>
> >
> > Gaspar
> >
> >
> > On Thu, Nov 24, 2022 at 12:10 AM Andy Seaborne <an...@apache.org> wrote:
> >
> >> Hi Gaspar,
> >>
> >> Not being able to look at the system make this tricky.
> >>
> >> Are you able to try 4.6.1?
> >>
> >> There are some changes related to database switching and compaction.
> >>
> >> Otherwise,
> >>
> >> If you can attach a debugger, can you see what threads are locked up?
> >>
> >> Andy
> >>
> >> On 23/11/2022 08:39, Gaspar Bartalus wrote:
> >>> Hi Andy,
> >>>
> >>> I've changed my email address, but I've raised this topic yesterday:
> >>>
> >>> Executing POST request /$/compact/database_name?deleteOld=true
> sometimes
> >>> creates a task which is hanging, and after which the database seems
> >>> unreachable. The copy folders (Data-000_) are created, but the old ones
> >> are
> >>> not deleted when this happens. The logs also seem to indicate that the
> >>> process is started but not finished:
> >>>
> >>>
> >>> - 06:49:50 INFO Admin :: [192] Compact dataset /database_name
> >>> - 23/11/2022 8:49:50
> >>> 06:49:50 INFO Server :: Task : 3 : Compact
> >>> - 23/11/2022 8:49:50
> >>> 06:49:50 INFO Server :: [Task 3] starts : Compact
> >>> - 23/11/2022 8:49:50
> >>> 06:49:50 INFO Compact :: [192] >>>> Start compact /database_name
> >>>
> >>>
> >>> We are using jena-fuseki 4.4.0 btw.
> >>>
> >>> Gaspar
> >>>
> >>
> >
>
Re: Compaction failing
Posted by Andy Seaborne <an...@apache.org>.
On 24/11/2022 08:35, Gaspar Bartalus wrote:
> Hi Andy,
>
> It seems that the issue occurs when I execute multiple compaction requests
> in a short time period.
> It probably runs out of resources in this case.
That would be odd because if it did (e.g. heap) you'll get something in
the log files and also it not be able to start later requests.
> If I leave a minute between each request then it seems stable.
The compact request schedules a compaction, it does not actually do it.
So, yes, a gap will help.
That does sound 4.6.1 could fix that.
Andy
We could add a "?sync=". The compaction would still scheduled and run on
a separate thread but the request would wait for the compaction to
finish. If the client gets bored and drops the connection before it
returns success, the compaction continues.
>
> Gaspar
>
>
> On Thu, Nov 24, 2022 at 12:10 AM Andy Seaborne <an...@apache.org> wrote:
>
>> Hi Gaspar,
>>
>> Not being able to look at the system make this tricky.
>>
>> Are you able to try 4.6.1?
>>
>> There are some changes related to database switching and compaction.
>>
>> Otherwise,
>>
>> If you can attach a debugger, can you see what threads are locked up?
>>
>> Andy
>>
>> On 23/11/2022 08:39, Gaspar Bartalus wrote:
>>> Hi Andy,
>>>
>>> I've changed my email address, but I've raised this topic yesterday:
>>>
>>> Executing POST request /$/compact/database_name?deleteOld=true sometimes
>>> creates a task which is hanging, and after which the database seems
>>> unreachable. The copy folders (Data-000_) are created, but the old ones
>> are
>>> not deleted when this happens. The logs also seem to indicate that the
>>> process is started but not finished:
>>>
>>>
>>> - 06:49:50 INFO Admin :: [192] Compact dataset /database_name
>>> - 23/11/2022 8:49:50
>>> 06:49:50 INFO Server :: Task : 3 : Compact
>>> - 23/11/2022 8:49:50
>>> 06:49:50 INFO Server :: [Task 3] starts : Compact
>>> - 23/11/2022 8:49:50
>>> 06:49:50 INFO Compact :: [192] >>>> Start compact /database_name
>>>
>>>
>>> We are using jena-fuseki 4.4.0 btw.
>>>
>>> Gaspar
>>>
>>
>
Re: Compaction failing
Posted by Gaspar Bartalus <ba...@gmail.com>.
Hi Andy,
It seems that the issue occurs when I execute multiple compaction requests
in a short time period.
It probably runs out of resources in this case.
If I leave a minute between each request then it seems stable.
Gaspar
On Thu, Nov 24, 2022 at 12:10 AM Andy Seaborne <an...@apache.org> wrote:
> Hi Gaspar,
>
> Not being able to look at the system make this tricky.
>
> Are you able to try 4.6.1?
>
> There are some changes related to database switching and compaction.
>
> Otherwise,
>
> If you can attach a debugger, can you see what threads are locked up?
>
> Andy
>
> On 23/11/2022 08:39, Gaspar Bartalus wrote:
> > Hi Andy,
> >
> > I've changed my email address, but I've raised this topic yesterday:
> >
> > Executing POST request /$/compact/database_name?deleteOld=true sometimes
> > creates a task which is hanging, and after which the database seems
> > unreachable. The copy folders (Data-000_) are created, but the old ones
> are
> > not deleted when this happens. The logs also seem to indicate that the
> > process is started but not finished:
> >
> >
> > - 06:49:50 INFO Admin :: [192] Compact dataset /database_name
> > - 23/11/2022 8:49:50
> > 06:49:50 INFO Server :: Task : 3 : Compact
> > - 23/11/2022 8:49:50
> > 06:49:50 INFO Server :: [Task 3] starts : Compact
> > - 23/11/2022 8:49:50
> > 06:49:50 INFO Compact :: [192] >>>> Start compact /database_name
> >
> >
> > We are using jena-fuseki 4.4.0 btw.
> >
> > Gaspar
> >
>
Re: Compaction failing
Posted by Andy Seaborne <an...@apache.org>.
Hi Gaspar,
Not being able to look at the system make this tricky.
Are you able to try 4.6.1?
There are some changes related to database switching and compaction.
Otherwise,
If you can attach a debugger, can you see what threads are locked up?
Andy
On 23/11/2022 08:39, Gaspar Bartalus wrote:
> Hi Andy,
>
> I've changed my email address, but I've raised this topic yesterday:
>
> Executing POST request /$/compact/database_name?deleteOld=true sometimes
> creates a task which is hanging, and after which the database seems
> unreachable. The copy folders (Data-000_) are created, but the old ones are
> not deleted when this happens. The logs also seem to indicate that the
> process is started but not finished:
>
>
> - 06:49:50 INFO Admin :: [192] Compact dataset /database_name
> - 23/11/2022 8:49:50
> 06:49:50 INFO Server :: Task : 3 : Compact
> - 23/11/2022 8:49:50
> 06:49:50 INFO Server :: [Task 3] starts : Compact
> - 23/11/2022 8:49:50
> 06:49:50 INFO Compact :: [192] >>>> Start compact /database_name
>
>
> We are using jena-fuseki 4.4.0 btw.
>
> Gaspar
>