You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@marmotta.apache.org by Frans Knibbe <fr...@geodan.nl> on 2016/08/23 09:03:14 UTC
Slow responses to first time requests
Hello,
I wonder if the following can be explained... I run Marmotta 3.3.0 with a
PostgreSQL 9.5 data store. I notice that when I request a certain resource
the first time it takes a long time (more than a minute) to produce the
reply. Subsequent requests for the same resource are resolved quickly (less
than a second). Perhaps I should mention that resource requests are
rewritten to {BASE}/resource?uri=<resource URI>.
If I recall correctly, the effect was less or absent when I used the
default H2 data store.
Is some kind of caching going on? It seems that the effect is not caused by
browser caching or by PostgreSQL's cache.
Is there something I could do to remedy the effect?
Greetings,
Frans
Re: Slow responses to first time requests
Posted by Frans Knibbe <fr...@geodan.nl>.
On 29 August 2016 at 18:09, Sergio Fernández <wi...@apache.org> wrote:
> Frans, do yor resources have a non-lical URI? That would explain the
> issue, sincd Marmotta would try to find more information out there.
>
Yes, I guess my resources have non-local URIs for Marmotta, depending how
you look at it. I do not use Marmotta's standard URI scheme because I want
to have freedom in how to mint my URIs. So the Apache HTTPD server that
sits before Marmotta rewrites URIs like
http://www.example.org/mydataset/resource123
to
http://www.example.org/marmotta/resource?uri=http://www.example.org/mydataset/resource123
In my mind both are local resources, but I can imagine that Marmotta thinks
otherwise. What happens next is that probably Marmotta gets in an infinite
loop until it reaches the 60 second limit.
> Besides disabling LDCaxhe, you can customize the configuration (
> http://marmotta.apache.org/platform/ldcache-module.html) to ignore your
> fake URI.
>
Great. I think it works. I have added the public URI of my Marmotta
instance (http://www.example.org in the example above) as an LD-Cache
Endpoint with "NONE" as value for the Kind / Provider parameter. Now LD
Cache in enabled but it no longer causes problems.
By the way, on the page dedicated to the configuration of the LD cache
(marmotta/cache/admin/configuration.html) I also get the error when I try
to change the value of *ldcache.so_timeout*.
Regards,
Frans
>
> On Aug 29, 2016 17:48, "Frans Knibbe" <fr...@geodan.nl> wrote:
>
>> Hello,
>>
>> I have done some test, to see if can find out what caused the long wait
>> time for first time requests. Here are some findings:
>>
>> - The first time requests always seem to take 60 seconds and a bit
>> (e.g. 60.232 seconds). So every time it is suspiciously close to a full
>> minute.
>> - The subsequent requests take less than a second (e.g. 0.109
>> seconds).
>> - The effect still occurs when I restart Marmotta between first and
>> subsequent requests.
>> - I tried turning off versioning (*versioning.enabled*). That did not
>> seem to have an effect on response times. The response headers did still
>> include timegate and timemap links, which I don't understand.
>> - Of the settings that can be changed using the admin web UI, I saw
>> the setting *ld_cache.so_timeout *is the only thing set to 60 seconds
>> (60000 miliseconds), which could somehow have to do with the delay of a bit
>> more than 60 seconds for a first request. To test that, I tried changing
>> the value of 60000 to something else. But I was not able to save the change
>> because of an error: cannot store content: TypeError:
>> value.getValue(...).split is not a function.
>> - Disabling ldcache altogether (*ldcache.enabled*) did solve the
>> problem.
>>
>> So I was able to solve the issue by disabling caching of remote
>> resources. That was unexpected, because all resources I requested where not
>> remote but local. But perhaps the problem lies with the way in which local
>> and remote resources are discerned, I do have some URI rewriting configured
>> (I wrote about that in another thread).
>>
>> The Linked Data Caching Module looks like a useful Marmotta component, it
>> would be a pity if I can not use it.
>>
>> Would it make sense or be useful if I log my findings in the issue
>> tracker? Or can everything be easily explained?
>>
>> Greetings,
>> Frans
>>
>>
>>
>> On 23 August 2016 at 11:03, Frans Knibbe <fr...@geodan.nl> wrote:
>>
>>> Hello,
>>>
>>> I wonder if the following can be explained... I run Marmotta 3.3.0 with
>>> a PostgreSQL 9.5 data store. I notice that when I request a certain
>>> resource the first time it takes a long time (more than a minute) to
>>> produce the reply. Subsequent requests for the same resource are resolved
>>> quickly (less than a second). Perhaps I should mention that resource
>>> requests are rewritten to {BASE}/resource?uri=<resource URI>.
>>>
>>> If I recall correctly, the effect was less or absent when I used the
>>> default H2 data store.
>>>
>>> Is some kind of caching going on? It seems that the effect is not caused
>>> by browser caching or by PostgreSQL's cache.
>>>
>>> Is there something I could do to remedy the effect?
>>>
>>> Greetings,
>>> Frans
>>>
>>
>>
Re: Slow responses to first time requests
Posted by Frans Knibbe <fr...@geodan.nl>.
Thanks Sebastian,
Yes, I managed to blacklist my own server and that seems to have worked. I
can still use the LD Cache now.
Regards,
Frans
On 29 August 2016 at 20:51, Sebastian Schaffert <
sebastian.schaffert@gmail.com> wrote:
> Note that this behaviour can be turned off by disabling LDCache in the
> Marmotta configuration (or fine-tuning it so that resources matching
> certain regular expressions are black-listed).
>
> Sebastian
>
> Sergio Fernández <wi...@apache.org> schrieb am Mo., 29. Aug. 2016 um
> 18:11 Uhr:
>
>> I forgot to say Marmotta _only_ considers "local" resources under
>> http://host.to/marmotta/resource/...
>>
>> On Aug 29, 2016 18:09, "Sergio Fernández" <wi...@apache.org> wrote:
>>
>>> Frans, do yor resources have a non-lical URI? That would explain the
>>> issue, sincd Marmotta would try to find more information out there.
>>>
>>> Besides disabling LDCaxhe, you can customize the configuration (
>>> http://marmotta.apache.org/platform/ldcache-module.html) to ignore your
>>> fake URI.
>>>
>>> On Aug 29, 2016 17:48, "Frans Knibbe" <fr...@geodan.nl> wrote:
>>>
>>>> Hello,
>>>>
>>>> I have done some test, to see if can find out what caused the long wait
>>>> time for first time requests. Here are some findings:
>>>>
>>>> - The first time requests always seem to take 60 seconds and a bit
>>>> (e.g. 60.232 seconds). So every time it is suspiciously close to a full
>>>> minute.
>>>> - The subsequent requests take less than a second (e.g. 0.109
>>>> seconds).
>>>> - The effect still occurs when I restart Marmotta between first and
>>>> subsequent requests.
>>>> - I tried turning off versioning (*versioning.enabled*). That did
>>>> not seem to have an effect on response times. The response headers did
>>>> still include timegate and timemap links, which I don't understand.
>>>> - Of the settings that can be changed using the admin web UI, I saw
>>>> the setting *ld_cache.so_timeout *is the only thing set to 60
>>>> seconds (60000 miliseconds), which could somehow have to do with the delay
>>>> of a bit more than 60 seconds for a first request. To test that, I tried
>>>> changing the value of 60000 to something else. But I was not able to save
>>>> the change because of an error: cannot store content: TypeError:
>>>> value.getValue(...).split is not a function.
>>>> - Disabling ldcache altogether (*ldcache.enabled*) did solve the
>>>> problem.
>>>>
>>>> So I was able to solve the issue by disabling caching of remote
>>>> resources. That was unexpected, because all resources I requested where not
>>>> remote but local. But perhaps the problem lies with the way in which local
>>>> and remote resources are discerned, I do have some URI rewriting configured
>>>> (I wrote about that in another thread).
>>>>
>>>> The Linked Data Caching Module looks like a useful Marmotta component,
>>>> it would be a pity if I can not use it.
>>>>
>>>> Would it make sense or be useful if I log my findings in the issue
>>>> tracker? Or can everything be easily explained?
>>>>
>>>> Greetings,
>>>> Frans
>>>>
>>>>
>>>>
>>>> On 23 August 2016 at 11:03, Frans Knibbe <fr...@geodan.nl>
>>>> wrote:
>>>>
>>>>> Hello,
>>>>>
>>>>> I wonder if the following can be explained... I run Marmotta 3.3.0
>>>>> with a PostgreSQL 9.5 data store. I notice that when I request a certain
>>>>> resource the first time it takes a long time (more than a minute) to
>>>>> produce the reply. Subsequent requests for the same resource are resolved
>>>>> quickly (less than a second). Perhaps I should mention that resource
>>>>> requests are rewritten to {BASE}/resource?uri=<resource URI>.
>>>>>
>>>>> If I recall correctly, the effect was less or absent when I used the
>>>>> default H2 data store.
>>>>>
>>>>> Is some kind of caching going on? It seems that the effect is not
>>>>> caused by browser caching or by PostgreSQL's cache.
>>>>>
>>>>> Is there something I could do to remedy the effect?
>>>>>
>>>>> Greetings,
>>>>> Frans
>>>>>
>>>>
>>>>
Re: Slow responses to first time requests
Posted by Sebastian Schaffert <se...@gmail.com>.
Note that this behaviour can be turned off by disabling LDCache in the
Marmotta configuration (or fine-tuning it so that resources matching
certain regular expressions are black-listed).
Sebastian
Sergio Fernández <wi...@apache.org> schrieb am Mo., 29. Aug. 2016 um
18:11 Uhr:
> I forgot to say Marmotta _only_ considers "local" resources under
> http://host.to/marmotta/resource/...
>
> On Aug 29, 2016 18:09, "Sergio Fernández" <wi...@apache.org> wrote:
>
>> Frans, do yor resources have a non-lical URI? That would explain the
>> issue, sincd Marmotta would try to find more information out there.
>>
>> Besides disabling LDCaxhe, you can customize the configuration (
>> http://marmotta.apache.org/platform/ldcache-module.html) to ignore your
>> fake URI.
>>
>> On Aug 29, 2016 17:48, "Frans Knibbe" <fr...@geodan.nl> wrote:
>>
>>> Hello,
>>>
>>> I have done some test, to see if can find out what caused the long wait
>>> time for first time requests. Here are some findings:
>>>
>>> - The first time requests always seem to take 60 seconds and a bit
>>> (e.g. 60.232 seconds). So every time it is suspiciously close to a full
>>> minute.
>>> - The subsequent requests take less than a second (e.g. 0.109
>>> seconds).
>>> - The effect still occurs when I restart Marmotta between first and
>>> subsequent requests.
>>> - I tried turning off versioning (*versioning.enabled*). That did
>>> not seem to have an effect on response times. The response headers did
>>> still include timegate and timemap links, which I don't understand.
>>> - Of the settings that can be changed using the admin web UI, I saw
>>> the setting *ld_cache.so_timeout *is the only thing set to 60
>>> seconds (60000 miliseconds), which could somehow have to do with the delay
>>> of a bit more than 60 seconds for a first request. To test that, I tried
>>> changing the value of 60000 to something else. But I was not able to save
>>> the change because of an error: cannot store content: TypeError:
>>> value.getValue(...).split is not a function.
>>> - Disabling ldcache altogether (*ldcache.enabled*) did solve the
>>> problem.
>>>
>>> So I was able to solve the issue by disabling caching of remote
>>> resources. That was unexpected, because all resources I requested where not
>>> remote but local. But perhaps the problem lies with the way in which local
>>> and remote resources are discerned, I do have some URI rewriting configured
>>> (I wrote about that in another thread).
>>>
>>> The Linked Data Caching Module looks like a useful Marmotta component,
>>> it would be a pity if I can not use it.
>>>
>>> Would it make sense or be useful if I log my findings in the issue
>>> tracker? Or can everything be easily explained?
>>>
>>> Greetings,
>>> Frans
>>>
>>>
>>>
>>> On 23 August 2016 at 11:03, Frans Knibbe <fr...@geodan.nl> wrote:
>>>
>>>> Hello,
>>>>
>>>> I wonder if the following can be explained... I run Marmotta 3.3.0 with
>>>> a PostgreSQL 9.5 data store. I notice that when I request a certain
>>>> resource the first time it takes a long time (more than a minute) to
>>>> produce the reply. Subsequent requests for the same resource are resolved
>>>> quickly (less than a second). Perhaps I should mention that resource
>>>> requests are rewritten to {BASE}/resource?uri=<resource URI>.
>>>>
>>>> If I recall correctly, the effect was less or absent when I used the
>>>> default H2 data store.
>>>>
>>>> Is some kind of caching going on? It seems that the effect is not
>>>> caused by browser caching or by PostgreSQL's cache.
>>>>
>>>> Is there something I could do to remedy the effect?
>>>>
>>>> Greetings,
>>>> Frans
>>>>
>>>
>>>
Re: Slow responses to first time requests
Posted by Sergio Fernández <wi...@apache.org>.
I forgot to say Marmotta _only_ considers "local" resources under
http://host.to/marmotta/resource/...
On Aug 29, 2016 18:09, "Sergio Fernández" <wi...@apache.org> wrote:
> Frans, do yor resources have a non-lical URI? That would explain the
> issue, sincd Marmotta would try to find more information out there.
>
> Besides disabling LDCaxhe, you can customize the configuration (
> http://marmotta.apache.org/platform/ldcache-module.html) to ignore your
> fake URI.
>
> On Aug 29, 2016 17:48, "Frans Knibbe" <fr...@geodan.nl> wrote:
>
>> Hello,
>>
>> I have done some test, to see if can find out what caused the long wait
>> time for first time requests. Here are some findings:
>>
>> - The first time requests always seem to take 60 seconds and a bit
>> (e.g. 60.232 seconds). So every time it is suspiciously close to a full
>> minute.
>> - The subsequent requests take less than a second (e.g. 0.109
>> seconds).
>> - The effect still occurs when I restart Marmotta between first and
>> subsequent requests.
>> - I tried turning off versioning (*versioning.enabled*). That did not
>> seem to have an effect on response times. The response headers did still
>> include timegate and timemap links, which I don't understand.
>> - Of the settings that can be changed using the admin web UI, I saw
>> the setting *ld_cache.so_timeout *is the only thing set to 60 seconds
>> (60000 miliseconds), which could somehow have to do with the delay of a bit
>> more than 60 seconds for a first request. To test that, I tried changing
>> the value of 60000 to something else. But I was not able to save the change
>> because of an error: cannot store content: TypeError:
>> value.getValue(...).split is not a function.
>> - Disabling ldcache altogether (*ldcache.enabled*) did solve the
>> problem.
>>
>> So I was able to solve the issue by disabling caching of remote
>> resources. That was unexpected, because all resources I requested where not
>> remote but local. But perhaps the problem lies with the way in which local
>> and remote resources are discerned, I do have some URI rewriting configured
>> (I wrote about that in another thread).
>>
>> The Linked Data Caching Module looks like a useful Marmotta component, it
>> would be a pity if I can not use it.
>>
>> Would it make sense or be useful if I log my findings in the issue
>> tracker? Or can everything be easily explained?
>>
>> Greetings,
>> Frans
>>
>>
>>
>> On 23 August 2016 at 11:03, Frans Knibbe <fr...@geodan.nl> wrote:
>>
>>> Hello,
>>>
>>> I wonder if the following can be explained... I run Marmotta 3.3.0 with
>>> a PostgreSQL 9.5 data store. I notice that when I request a certain
>>> resource the first time it takes a long time (more than a minute) to
>>> produce the reply. Subsequent requests for the same resource are resolved
>>> quickly (less than a second). Perhaps I should mention that resource
>>> requests are rewritten to {BASE}/resource?uri=<resource URI>.
>>>
>>> If I recall correctly, the effect was less or absent when I used the
>>> default H2 data store.
>>>
>>> Is some kind of caching going on? It seems that the effect is not caused
>>> by browser caching or by PostgreSQL's cache.
>>>
>>> Is there something I could do to remedy the effect?
>>>
>>> Greetings,
>>> Frans
>>>
>>
>>
Re: Slow responses to first time requests
Posted by Sergio Fernández <wi...@apache.org>.
Frans, do yor resources have a non-lical URI? That would explain the issue,
sincd Marmotta would try to find more information out there.
Besides disabling LDCaxhe, you can customize the configuration (
http://marmotta.apache.org/platform/ldcache-module.html) to ignore your
fake URI.
On Aug 29, 2016 17:48, "Frans Knibbe" <fr...@geodan.nl> wrote:
> Hello,
>
> I have done some test, to see if can find out what caused the long wait
> time for first time requests. Here are some findings:
>
> - The first time requests always seem to take 60 seconds and a bit
> (e.g. 60.232 seconds). So every time it is suspiciously close to a full
> minute.
> - The subsequent requests take less than a second (e.g. 0.109 seconds).
> - The effect still occurs when I restart Marmotta between first and
> subsequent requests.
> - I tried turning off versioning (*versioning.enabled*). That did not
> seem to have an effect on response times. The response headers did still
> include timegate and timemap links, which I don't understand.
> - Of the settings that can be changed using the admin web UI, I saw
> the setting *ld_cache.so_timeout *is the only thing set to 60 seconds
> (60000 miliseconds), which could somehow have to do with the delay of a bit
> more than 60 seconds for a first request. To test that, I tried changing
> the value of 60000 to something else. But I was not able to save the change
> because of an error: cannot store content: TypeError:
> value.getValue(...).split is not a function.
> - Disabling ldcache altogether (*ldcache.enabled*) did solve the
> problem.
>
> So I was able to solve the issue by disabling caching of remote resources.
> That was unexpected, because all resources I requested where not remote but
> local. But perhaps the problem lies with the way in which local and remote
> resources are discerned, I do have some URI rewriting configured (I wrote
> about that in another thread).
>
> The Linked Data Caching Module looks like a useful Marmotta component, it
> would be a pity if I can not use it.
>
> Would it make sense or be useful if I log my findings in the issue
> tracker? Or can everything be easily explained?
>
> Greetings,
> Frans
>
>
>
> On 23 August 2016 at 11:03, Frans Knibbe <fr...@geodan.nl> wrote:
>
>> Hello,
>>
>> I wonder if the following can be explained... I run Marmotta 3.3.0 with a
>> PostgreSQL 9.5 data store. I notice that when I request a certain resource
>> the first time it takes a long time (more than a minute) to produce the
>> reply. Subsequent requests for the same resource are resolved quickly (less
>> than a second). Perhaps I should mention that resource requests are
>> rewritten to {BASE}/resource?uri=<resource URI>.
>>
>> If I recall correctly, the effect was less or absent when I used the
>> default H2 data store.
>>
>> Is some kind of caching going on? It seems that the effect is not caused
>> by browser caching or by PostgreSQL's cache.
>>
>> Is there something I could do to remedy the effect?
>>
>> Greetings,
>> Frans
>>
>
>
Re: Slow responses to first time requests
Posted by Frans Knibbe <fr...@geodan.nl>.
Hello,
I have done some test, to see if can find out what caused the long wait
time for first time requests. Here are some findings:
- The first time requests always seem to take 60 seconds and a bit (e.g.
60.232 seconds). So every time it is suspiciously close to a full minute.
- The subsequent requests take less than a second (e.g. 0.109 seconds).
- The effect still occurs when I restart Marmotta between first and
subsequent requests.
- I tried turning off versioning (*versioning.enabled*). That did not
seem to have an effect on response times. The response headers did still
include timegate and timemap links, which I don't understand.
- Of the settings that can be changed using the admin web UI, I saw the
setting *ld_cache.so_timeout *is the only thing set to 60 seconds (60000
miliseconds), which could somehow have to do with the delay of a bit more
than 60 seconds for a first request. To test that, I tried changing the
value of 60000 to something else. But I was not able to save the change
because of an error: cannot store content: TypeError:
value.getValue(...).split is not a function.
- Disabling ldcache altogether (*ldcache.enabled*) did solve the problem.
So I was able to solve the issue by disabling caching of remote resources.
That was unexpected, because all resources I requested where not remote but
local. But perhaps the problem lies with the way in which local and remote
resources are discerned, I do have some URI rewriting configured (I wrote
about that in another thread).
The Linked Data Caching Module looks like a useful Marmotta component, it
would be a pity if I can not use it.
Would it make sense or be useful if I log my findings in the issue tracker?
Or can everything be easily explained?
Greetings,
Frans
On 23 August 2016 at 11:03, Frans Knibbe <fr...@geodan.nl> wrote:
> Hello,
>
> I wonder if the following can be explained... I run Marmotta 3.3.0 with a
> PostgreSQL 9.5 data store. I notice that when I request a certain resource
> the first time it takes a long time (more than a minute) to produce the
> reply. Subsequent requests for the same resource are resolved quickly (less
> than a second). Perhaps I should mention that resource requests are
> rewritten to {BASE}/resource?uri=<resource URI>.
>
> If I recall correctly, the effect was less or absent when I used the
> default H2 data store.
>
> Is some kind of caching going on? It seems that the effect is not caused
> by browser caching or by PostgreSQL's cache.
>
> Is there something I could do to remedy the effect?
>
> Greetings,
> Frans
>