You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@shindig.apache.org by Marijn Speelman <ma...@hyves.nl> on 2008/07/07 19:49:40 UTC
Gadget/MessageBundle spec OR HttpResponse caching?
I'm a bit confused about the configuration of caching in Java Shindig.
From what I can see, when a gadget render request is made the spec can
come from three locations:
- In-memory gadget-spec cache.
- In-memory http reponse cache.
- The real http location.
By default the gadget and message bundle spec capacities are:
file: gadgets/conf/gadgets.properties:
gadget-spec.cache.capacity=0
message-bundle.cache.capacity=0
I've implemented an extra memcached layer on top of BasicHttpCache and I
want to be sure that I understand the caching process completely.
Is it true that this effectively turns these two caches off, because
whenever an element is added to the cache, it will be thrown out
immediately because the size exceeds the capacity?
In other words: is this the way to choose between http response caching
OR just the gadget/messagebundle caching?
Some more documentation about the different places and ways of caching
and configuration would be also be greatly appreciated. For instance, is
there any place where the use of the client-side caching parameter
version ("v=") is documented? Although it's a simple concept, I could
only find it by digging through the code.
Thanks,
Marijn
Shindig Wiki..
Posted by "Michael Ryan (Software Developer)" <Mi...@apollogrp.edu>.
I created an account on the confluence site, and would be interested in
adding to the existing wiki...
http://cwiki.apache.org/confluence/display/~tracker1
Would probably start with the existing installation instructions for
Java and PHP, and break them down a bit... I am doing this anyway for
opencampfire.net ... As I will be looking into the existing
implementations, and creating documentation anyhow, I'm more than happy
to work within the current wiki system.
--
Michael J. Ryan -- Software Developer -- Apollo Group
> -----Original Message-----
> From: Kevin Brown [mailto:etnu@google.com]
> Sent: Monday, July 07, 2008 12:26 PM
> To: shindig-dev@incubator.apache.org
> Subject: Re: Gadget/MessageBundle spec OR HttpResponse caching?
>
> On Mon, Jul 7, 2008 at 10:49 AM, Marijn Speelman <ma...@hyves.nl>
> wrote:
>
> > I'm a bit confused about the configuration of caching in Java
> Shindig.
> >
> > From what I can see, when a gadget render request is made the spec
> can come
> > from three locations:
> > - In-memory gadget-spec cache.
> > - In-memory http reponse cache.
> > - The real http location.
> >
> > By default the gadget and message bundle spec capacities are:
> >
> > file: gadgets/conf/gadgets.properties:
> > gadget-spec.cache.capacity=0
> > message-bundle.cache.capacity=0
> >
> > I've implemented an extra memcached layer on top of BasicHttpCache
> and I
> > want to be sure that I understand the caching process completely.
> >
> > Is it true that this effectively turns these two caches off, because
> > whenever an element is added to the cache, it will be thrown out
> immediately
> > because the size exceeds the capacity?
> > In other words: is this the way to choose between http response
> caching OR
> > just the gadget/messagebundle caching?
>
>
> You should use both. The manifest files (gadget spec + message
bundles)
> should be cached in memory because the cost of parsing the xml is
> fairly
> significant (in profiling, it's by far the most expensive operation in
> the
> stack, even beating out link rewriting). The HTTP cache is more
general
> purpose, and in a production environment you would always want to use
a
> distributed cache such as memcache.
>
>
> >
> > Some more documentation about the different places and ways of
> caching and
> > configuration would be also be greatly appreciated. For instance, is
> there
> > any place where the use of the client-side caching parameter version
> ("v=")
> > is documented? Although it's a simple concept, I could only find it
> by
> > digging through the code.
>
>
> There really isn't much by way of "user documentation" for Shindig
> right
> now, though it would definitely be a welcomed contribution. We mostly
> keep
> things on confluence (http://cwiki.apache.org/SHINDIGxSITE), but a
wiki
> solution with more open access would be nice.
>
>
> >
> >
> > Thanks,
> >
> > Marijn
> >
This message is private and confidential. If you have received it in error, please notify the sender and remove it from your system.
Re: Gadget/MessageBundle spec OR HttpResponse caching?
Posted by Louis Ryan <lr...@google.com>.
Hi,
I made the defaults 0 so as not to significantly alter the behavior in the
default sample-container. These values should absolutely be changed in
production systems for improved performance based on the number of distinct
gadgets served etc.
-Louis
On Mon, Jul 7, 2008 at 2:57 PM, Kevin Brown <et...@google.com> wrote:
> On Mon, Jul 7, 2008 at 2:48 PM, Marijn Speelman <ma...@hyves.nl> wrote:
>
>> On 7-7-2008 21:25, Kevin Brown wrote:
>>
>>> On Mon, Jul 7, 2008 at 10:49 AM, Marijn Speelman <ma...@hyves.nl>
>>> wrote:
>>>
>>>> Is it true that this effectively turns these two caches off, because
>>>> whenever an element is added to the cache, it will be thrown out
>>>> immediately
>>>> because the size exceeds the capacity?
>>>> In other words: is this the way to choose between http response caching
>>>> OR
>>>> just the gadget/messagebundle caching?
>>>>
>>>
>>> You should use both. The manifest files (gadget spec + message bundles)
>>> should be cached in memory because the cost of parsing the xml is fairly
>>> significant (in profiling, it's by far the most expensive operation in
>>> the
>>> stack, even beating out link rewriting). The HTTP cache is more general
>>> purpose, and in a production environment you would always want to use a
>>> distributed cache such as memcache.
>>>
>>
>> Thank you for clarifying that. So, just to be sure, by default the
>> (parsed) manifest files are NOT cached as their cache capacity is set to 0
>> in the configuration, but it's advised to do change this number so they will
>> be?
>>
>
> The defaults should probably actually be changed. Louis added this, so
> maybe he had a good reason for defaulting to zero.
>
Re: Gadget/MessageBundle spec OR HttpResponse caching?
Posted by Kevin Brown <et...@google.com>.
On Mon, Jul 7, 2008 at 2:48 PM, Marijn Speelman <ma...@hyves.nl> wrote:
> On 7-7-2008 21:25, Kevin Brown wrote:
>
>> On Mon, Jul 7, 2008 at 10:49 AM, Marijn Speelman <ma...@hyves.nl> wrote:
>>
>>> Is it true that this effectively turns these two caches off, because
>>> whenever an element is added to the cache, it will be thrown out
>>> immediately
>>> because the size exceeds the capacity?
>>> In other words: is this the way to choose between http response caching
>>> OR
>>> just the gadget/messagebundle caching?
>>>
>>
>> You should use both. The manifest files (gadget spec + message bundles)
>> should be cached in memory because the cost of parsing the xml is fairly
>> significant (in profiling, it's by far the most expensive operation in the
>> stack, even beating out link rewriting). The HTTP cache is more general
>> purpose, and in a production environment you would always want to use a
>> distributed cache such as memcache.
>>
>
> Thank you for clarifying that. So, just to be sure, by default the (parsed)
> manifest files are NOT cached as their cache capacity is set to 0 in the
> configuration, but it's advised to do change this number so they will be?
>
The defaults should probably actually be changed. Louis added this, so maybe
he had a good reason for defaulting to zero.
Re: Gadget/MessageBundle spec OR HttpResponse caching?
Posted by Marijn Speelman <ma...@hyves.nl>.
On 7-7-2008 21:25, Kevin Brown wrote:
> On Mon, Jul 7, 2008 at 10:49 AM, Marijn Speelman <ma...@hyves.nl> wrote:
>> Is it true that this effectively turns these two caches off, because
>> whenever an element is added to the cache, it will be thrown out immediately
>> because the size exceeds the capacity?
>> In other words: is this the way to choose between http response caching OR
>> just the gadget/messagebundle caching?
>
> You should use both. The manifest files (gadget spec + message bundles)
> should be cached in memory because the cost of parsing the xml is fairly
> significant (in profiling, it's by far the most expensive operation in the
> stack, even beating out link rewriting). The HTTP cache is more general
> purpose, and in a production environment you would always want to use a
> distributed cache such as memcache.
Thank you for clarifying that. So, just to be sure, by default the
(parsed) manifest files are NOT cached as their cache capacity is set to
0 in the configuration, but it's advised to do change this number so
they will be?
Re: Gadget/MessageBundle spec OR HttpResponse caching?
Posted by Kevin Brown <et...@google.com>.
On Mon, Jul 7, 2008 at 10:49 AM, Marijn Speelman <ma...@hyves.nl> wrote:
> I'm a bit confused about the configuration of caching in Java Shindig.
>
> From what I can see, when a gadget render request is made the spec can come
> from three locations:
> - In-memory gadget-spec cache.
> - In-memory http reponse cache.
> - The real http location.
>
> By default the gadget and message bundle spec capacities are:
>
> file: gadgets/conf/gadgets.properties:
> gadget-spec.cache.capacity=0
> message-bundle.cache.capacity=0
>
> I've implemented an extra memcached layer on top of BasicHttpCache and I
> want to be sure that I understand the caching process completely.
>
> Is it true that this effectively turns these two caches off, because
> whenever an element is added to the cache, it will be thrown out immediately
> because the size exceeds the capacity?
> In other words: is this the way to choose between http response caching OR
> just the gadget/messagebundle caching?
You should use both. The manifest files (gadget spec + message bundles)
should be cached in memory because the cost of parsing the xml is fairly
significant (in profiling, it's by far the most expensive operation in the
stack, even beating out link rewriting). The HTTP cache is more general
purpose, and in a production environment you would always want to use a
distributed cache such as memcache.
>
> Some more documentation about the different places and ways of caching and
> configuration would be also be greatly appreciated. For instance, is there
> any place where the use of the client-side caching parameter version ("v=")
> is documented? Although it's a simple concept, I could only find it by
> digging through the code.
There really isn't much by way of "user documentation" for Shindig right
now, though it would definitely be a welcomed contribution. We mostly keep
things on confluence (http://cwiki.apache.org/SHINDIGxSITE), but a wiki
solution with more open access would be nice.
>
>
> Thanks,
>
> Marijn
>