You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@shindig.apache.org by Eran Galili <er...@diffdoof.com> on 2008/09/07 08:19:46 UTC
Re: Gadget XML cache
There's something I don't understand.Why do you cache the XML for such a
short time?
Gadgets don't change every hour, or even every day.
Why not parse the XML once and save it for a few days?
That way the servers make fewer requests and parse XML less often.
On Fri, Aug 29, 2008 at 7:50 PM, Kevin Brown <et...@google.com> wrote:
> In addition to what Ian has said, there are actually two caches:
>
> - HTTP level. Generally best implemented as a distributed cache, since
> there
> will be lots of files here.
> - Manifest (Gadget / message bundle xml) level. Best implemented as an
> in-memory cache.
>
> The default implementations of both caches are in memory, but if you're
> scaling up I'd recommend using a distributed cache for HTTP (and strongly
> recommend keeping the in-memory cache for the xml files).
>
> As far as caching policy goes, the specification *requires* that we do two
> things:
>
> 1. Honor HTTP cache-control headers by default.
> 2. If the user specified a parameter in the request (say, in getProxyUrl or
> makeRequest, or by rewriting), honor the cache time that they've specified.
>
> For gadget specs, the rules are not well defined, but the de facto standard
> is to cache the files for a reasonably long time period (usually 1 hour).
>
> On Fri, Aug 29, 2008 at 3:45 AM, Ian Boston <ie...@tfd.co.uk> wrote:
>
> > For Java Shindig:
> >
> > The default cache is entirely in memory, wont page to disk and doesn't
> have
> > distributed capabilities, but is light fast and works perfectly well.
> >
> > However
> > In the current code base there is an implementation of the CacheProvider
> > for EhCache which will do distributed Cache for you.
> >
> > Its not active by default, you need to add the EhCacheGuiceCacheModule to
> > the list of modules that are loaded.
> > The configuration is in 2 places, ehcache.properties for the shindig
> > specific settings, like JMX stats and the location of the ehcache xml
> > configuration file, currently on the classpath at
> > /org/apache/shindig/cache/ehcache/ehcacheConfig.xml
> >
> > You should consult ehcache documentation to configure distribution and
> > paging to disk http://ehcache.sourceforge.net/EhcacheUserGuide.html
> >
> > I have an Apache JCS implementation in waiting to go in, and there may be
> a
> > memcachd impl in the pipeline.
> >
> >
> > Code:
> > http://svn.apache.org/repos/asf/incubator/shindig/trunk/java/common/
> > src/main/java/org/apache/shindig/common/cache/ehcache/
> > http://svn.apache.org/repos/asf/incubator/shindig/trunk/java/common/
> > src/main/bundle/ehcache.properties
> > http://svn.apache.org/repos/asf/incubator/shindig/trunk/java/common/
> > src/main/bundle/org/apache/shindig/common/cache/ehcache/ehcacheConfig.xml
> >
> > The location may change as we get closer to a 1.0 release.
> >
> > HTH
> > Ian
> >
> >
> >
> >
> >
> > On 29 Aug 2008, at 10:14, Eran Galili wrote:
> >
> > Hi,
> >>
> >> I started implementing a container using shindig as my gadget server and
> I
> >> have a few questions regarding the way shindig gets and caches the
> gadget
> >> XML.
> >>
> >> First, I'd like to know what's considered the best practice for the
> >> expiration time of cached gadgets? Does it have to be in some kind of
> >> range
> >> for my container to be spec compliant?
> >>
> >> Secondly, does shindig cache the gadgets only on the machine's memory,
> or
> >> does it also support using a distributed cache? If i set a very long
> time
> >> for my container to update the gadget XML (and perhaps allow the
> developer
> >> "update me" functionality), would it be reasonable to save it on my
> >> database
> >> (maybe save it already parsed)?
> >>
> >> Also if i keep an "Application Directory" of approved apps that i allow
> on
> >> my container, is it really necessary to send the gadget XML URL to when
> >> sending a gadget rendering request? If i implement my own gadget server,
> >> could i simply send an application Id (managed by my site) and on the
> >> server
> >> side get the appropriate URL for that application (or even the XML
> itself
> >> from the cache)?
> >>
> >> I'm asking these questions because i think I'm missing something. I
> would
> >> like to minimize the amount of times my servers have to parse the gadget
> >> xml. Saving the gadget spec already parsed would allow me to do that.
> >>
> >> Thanks in advance,
> >> Eran
> >>
> >
> >
>
Gadget/OpenSocial Management Interfaces
Posted by Paul Lindner <pl...@hi5.com>.
I'd be happy to see some kind of management/provisioning features
added to the protocol. Most containers try to associate an
application wiht an existing container user or users. Those
application owner users would be able to perform management operations
on the application:
* Disable/Enable the Application (for maintenance, etc)
* Refresh Cache
* Submit Application for Approval
* Get Metrics
* etc.
On Sep 8, 2008, at 10:36 AM, Kevin Brown wrote:
> And most containers do, but it's not a standard.
>
> On Sun, Sep 7, 2008 at 11:35 PM, Eran Galili <er...@diffdoof.com>
> wrote:
>
>> that can be easily treated with a "refresh me" feature.suppose we
>> expose a
>> URL that accepts the gadgets spec URL as a parameter and refreshes
>> the
>> cache.
>> it would deal with abusive requests of course and will be listed in
>> the
>> discovery document...
>>
>>
>> On Mon, Sep 8, 2008 at 12:03 AM, Kevin Brown <et...@google.com> wrote:
>>
>>> On Sat, Sep 6, 2008 at 11:19 PM, Eran Galili <er...@diffdoof.com>
>>> wrote:
>>>
>>>> There's something I don't understand.Why do you cache the XML for
>>>> such
>> a
>>>> short time?
>>>> Gadgets don't change every hour, or even every day.
>>>> Why not parse the XML once and save it for a few days?
>>>> That way the servers make fewer requests and parse XML less often.
>>>
>>>
>>> Because nothing pisses developers off more than being told that
>>> they have
>>> to
>>> wait days for their changes to take effect.
>>>
>>>
>>>>
>>>>
>>>>
>>>> On Fri, Aug 29, 2008 at 7:50 PM, Kevin Brown <et...@google.com>
>>>> wrote:
>>>>
>>>>> In addition to what Ian has said, there are actually two caches:
>>>>>
>>>>> - HTTP level. Generally best implemented as a distributed cache,
>> since
>>>>> there
>>>>> will be lots of files here.
>>>>> - Manifest (Gadget / message bundle xml) level. Best implemented
>>>>> as
>> an
>>>>> in-memory cache.
>>>>>
>>>>> The default implementations of both caches are in memory, but if
>> you're
>>>>> scaling up I'd recommend using a distributed cache for HTTP (and
>>> strongly
>>>>> recommend keeping the in-memory cache for the xml files).
>>>>>
>>>>> As far as caching policy goes, the specification *requires* that
>>>>> we
>> do
>>>> two
>>>>> things:
>>>>>
>>>>> 1. Honor HTTP cache-control headers by default.
>>>>> 2. If the user specified a parameter in the request (say, in
>>> getProxyUrl
>>>> or
>>>>> makeRequest, or by rewriting), honor the cache time that they've
>>>> specified.
>>>>>
>>>>> For gadget specs, the rules are not well defined, but the de facto
>>>> standard
>>>>> is to cache the files for a reasonably long time period (usually 1
>>> hour).
>>>>>
>>>>> On Fri, Aug 29, 2008 at 3:45 AM, Ian Boston <ie...@tfd.co.uk> wrote:
>>>>>
>>>>>> For Java Shindig:
>>>>>>
>>>>>> The default cache is entirely in memory, wont page to disk and
>>> doesn't
>>>>> have
>>>>>> distributed capabilities, but is light fast and works perfectly
>> well.
>>>>>>
>>>>>> However
>>>>>> In the current code base there is an implementation of the
>>>> CacheProvider
>>>>>> for EhCache which will do distributed Cache for you.
>>>>>>
>>>>>> Its not active by default, you need to add the
>>> EhCacheGuiceCacheModule
>>>> to
>>>>>> the list of modules that are loaded.
>>>>>> The configuration is in 2 places, ehcache.properties for the
>> shindig
>>>>>> specific settings, like JMX stats and the location of the ehcache
>> xml
>>>>>> configuration file, currently on the classpath at
>>>>>> /org/apache/shindig/cache/ehcache/ehcacheConfig.xml
>>>>>>
>>>>>> You should consult ehcache documentation to configure
>>>>>> distribution
>>> and
>>>>>> paging to disk
>> http://ehcache.sourceforge.net/EhcacheUserGuide.html
>>>>>>
>>>>>> I have an Apache JCS implementation in waiting to go in, and
>>>>>> there
>>> may
>>>> be
>>>>> a
>>>>>> memcachd impl in the pipeline.
>>>>>>
>>>>>>
>>>>>> Code:
>>>>>>
>> http://svn.apache.org/repos/asf/incubator/shindig/trunk/java/common/
>>>>>> src/main/java/org/apache/shindig/common/cache/ehcache/
>>>>>>
>> http://svn.apache.org/repos/asf/incubator/shindig/trunk/java/common/
>>>>>> src/main/bundle/ehcache.properties
>>>>>>
>> http://svn.apache.org/repos/asf/incubator/shindig/trunk/java/common/
>>>>>>
>>>>
>> src/main/bundle/org/apache/shindig/common/cache/ehcache/
>> ehcacheConfig.xml
>>>>>>
>>>>>> The location may change as we get closer to a 1.0 release.
>>>>>>
>>>>>> HTH
>>>>>> Ian
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 29 Aug 2008, at 10:14, Eran Galili wrote:
>>>>>>
>>>>>> Hi,
>>>>>>>
>>>>>>> I started implementing a container using shindig as my gadget
>> server
>>>> and
>>>>> I
>>>>>>> have a few questions regarding the way shindig gets and caches
>>>>>>> the
>>>>> gadget
>>>>>>> XML.
>>>>>>>
>>>>>>> First, I'd like to know what's considered the best practice for
>> the
>>>>>>> expiration time of cached gadgets? Does it have to be in some
>> kind
>>> of
>>>>>>> range
>>>>>>> for my container to be spec compliant?
>>>>>>>
>>>>>>> Secondly, does shindig cache the gadgets only on the machine's
>>> memory,
>>>>> or
>>>>>>> does it also support using a distributed cache? If i set a very
>> long
>>>>> time
>>>>>>> for my container to update the gadget XML (and perhaps allow the
>>>>> developer
>>>>>>> "update me" functionality), would it be reasonable to save it on
>> my
>>>>>>> database
>>>>>>> (maybe save it already parsed)?
>>>>>>>
>>>>>>> Also if i keep an "Application Directory" of approved apps
>>>>>>> that i
>>>> allow
>>>>> on
>>>>>>> my container, is it really necessary to send the gadget XML
>>>>>>> URL to
>>>> when
>>>>>>> sending a gadget rendering request? If i implement my own gadget
>>>> server,
>>>>>>> could i simply send an application Id (managed by my site) and
>>>>>>> on
>>> the
>>>>>>> server
>>>>>>> side get the appropriate URL for that application (or even the
>>>>>>> XML
>>>>> itself
>>>>>>> from the cache)?
>>>>>>>
>>>>>>> I'm asking these questions because i think I'm missing
>>>>>>> something.
>> I
>>>>> would
>>>>>>> like to minimize the amount of times my servers have to parse
>>>>>>> the
>>>> gadget
>>>>>>> xml. Saving the gadget spec already parsed would allow me to do
>>> that.
>>>>>>>
>>>>>>> Thanks in advance,
>>>>>>> Eran
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
Paul Lindner
plindner@hi5.com
Re: Gadget XML cache
Posted by Kevin Brown <et...@google.com>.
And most containers do, but it's not a standard.
On Sun, Sep 7, 2008 at 11:35 PM, Eran Galili <er...@diffdoof.com> wrote:
> that can be easily treated with a "refresh me" feature.suppose we expose a
> URL that accepts the gadgets spec URL as a parameter and refreshes the
> cache.
> it would deal with abusive requests of course and will be listed in the
> discovery document...
>
>
> On Mon, Sep 8, 2008 at 12:03 AM, Kevin Brown <et...@google.com> wrote:
>
> > On Sat, Sep 6, 2008 at 11:19 PM, Eran Galili <er...@diffdoof.com> wrote:
> >
> > > There's something I don't understand.Why do you cache the XML for such
> a
> > > short time?
> > > Gadgets don't change every hour, or even every day.
> > > Why not parse the XML once and save it for a few days?
> > > That way the servers make fewer requests and parse XML less often.
> >
> >
> > Because nothing pisses developers off more than being told that they have
> > to
> > wait days for their changes to take effect.
> >
> >
> > >
> > >
> > >
> > > On Fri, Aug 29, 2008 at 7:50 PM, Kevin Brown <et...@google.com> wrote:
> > >
> > > > In addition to what Ian has said, there are actually two caches:
> > > >
> > > > - HTTP level. Generally best implemented as a distributed cache,
> since
> > > > there
> > > > will be lots of files here.
> > > > - Manifest (Gadget / message bundle xml) level. Best implemented as
> an
> > > > in-memory cache.
> > > >
> > > > The default implementations of both caches are in memory, but if
> you're
> > > > scaling up I'd recommend using a distributed cache for HTTP (and
> > strongly
> > > > recommend keeping the in-memory cache for the xml files).
> > > >
> > > > As far as caching policy goes, the specification *requires* that we
> do
> > > two
> > > > things:
> > > >
> > > > 1. Honor HTTP cache-control headers by default.
> > > > 2. If the user specified a parameter in the request (say, in
> > getProxyUrl
> > > or
> > > > makeRequest, or by rewriting), honor the cache time that they've
> > > specified.
> > > >
> > > > For gadget specs, the rules are not well defined, but the de facto
> > > standard
> > > > is to cache the files for a reasonably long time period (usually 1
> > hour).
> > > >
> > > > On Fri, Aug 29, 2008 at 3:45 AM, Ian Boston <ie...@tfd.co.uk> wrote:
> > > >
> > > > > For Java Shindig:
> > > > >
> > > > > The default cache is entirely in memory, wont page to disk and
> > doesn't
> > > > have
> > > > > distributed capabilities, but is light fast and works perfectly
> well.
> > > > >
> > > > > However
> > > > > In the current code base there is an implementation of the
> > > CacheProvider
> > > > > for EhCache which will do distributed Cache for you.
> > > > >
> > > > > Its not active by default, you need to add the
> > EhCacheGuiceCacheModule
> > > to
> > > > > the list of modules that are loaded.
> > > > > The configuration is in 2 places, ehcache.properties for the
> shindig
> > > > > specific settings, like JMX stats and the location of the ehcache
> xml
> > > > > configuration file, currently on the classpath at
> > > > > /org/apache/shindig/cache/ehcache/ehcacheConfig.xml
> > > > >
> > > > > You should consult ehcache documentation to configure distribution
> > and
> > > > > paging to disk
> http://ehcache.sourceforge.net/EhcacheUserGuide.html
> > > > >
> > > > > I have an Apache JCS implementation in waiting to go in, and there
> > may
> > > be
> > > > a
> > > > > memcachd impl in the pipeline.
> > > > >
> > > > >
> > > > > Code:
> > > > >
> http://svn.apache.org/repos/asf/incubator/shindig/trunk/java/common/
> > > > > src/main/java/org/apache/shindig/common/cache/ehcache/
> > > > >
> http://svn.apache.org/repos/asf/incubator/shindig/trunk/java/common/
> > > > > src/main/bundle/ehcache.properties
> > > > >
> http://svn.apache.org/repos/asf/incubator/shindig/trunk/java/common/
> > > > >
> > >
> src/main/bundle/org/apache/shindig/common/cache/ehcache/ehcacheConfig.xml
> > > > >
> > > > > The location may change as we get closer to a 1.0 release.
> > > > >
> > > > > HTH
> > > > > Ian
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > On 29 Aug 2008, at 10:14, Eran Galili wrote:
> > > > >
> > > > > Hi,
> > > > >>
> > > > >> I started implementing a container using shindig as my gadget
> server
> > > and
> > > > I
> > > > >> have a few questions regarding the way shindig gets and caches the
> > > > gadget
> > > > >> XML.
> > > > >>
> > > > >> First, I'd like to know what's considered the best practice for
> the
> > > > >> expiration time of cached gadgets? Does it have to be in some
> kind
> > of
> > > > >> range
> > > > >> for my container to be spec compliant?
> > > > >>
> > > > >> Secondly, does shindig cache the gadgets only on the machine's
> > memory,
> > > > or
> > > > >> does it also support using a distributed cache? If i set a very
> long
> > > > time
> > > > >> for my container to update the gadget XML (and perhaps allow the
> > > > developer
> > > > >> "update me" functionality), would it be reasonable to save it on
> my
> > > > >> database
> > > > >> (maybe save it already parsed)?
> > > > >>
> > > > >> Also if i keep an "Application Directory" of approved apps that i
> > > allow
> > > > on
> > > > >> my container, is it really necessary to send the gadget XML URL to
> > > when
> > > > >> sending a gadget rendering request? If i implement my own gadget
> > > server,
> > > > >> could i simply send an application Id (managed by my site) and on
> > the
> > > > >> server
> > > > >> side get the appropriate URL for that application (or even the XML
> > > > itself
> > > > >> from the cache)?
> > > > >>
> > > > >> I'm asking these questions because i think I'm missing something.
> I
> > > > would
> > > > >> like to minimize the amount of times my servers have to parse the
> > > gadget
> > > > >> xml. Saving the gadget spec already parsed would allow me to do
> > that.
> > > > >>
> > > > >> Thanks in advance,
> > > > >> Eran
> > > > >>
> > > > >
> > > > >
> > > >
> > >
> >
>
Re: Gadget XML cache
Posted by Chris Chabot <ch...@google.com>.
Now that's something that should take place on the
gadgets-and-opensocial-spec list, shindig merely implements the spec :)
You could start the discussion there now (nothing with wrong with trying to
build some support for an idean) and then you can fire off a proposal there
at the next voting round (next scheduled one is for 0.9), if it gets 5 +1
votes, it'll be part of the spec, and something we can add to shindig.
I know it's a bit of a round-about way of trying to get solutions into
shindig, but shindig is not the only OpenSocial implementation, so defining
what should be available through the spec is essential.
-- Chris
On Mon, Sep 8, 2008 at 8:35 AM, Eran Galili <er...@diffdoof.com> wrote:
> that can be easily treated with a "refresh me" feature.suppose we expose a
> URL that accepts the gadgets spec URL as a parameter and refreshes the
> cache.
> it would deal with abusive requests of course and will be listed in the
> discovery document...
>
>
> On Mon, Sep 8, 2008 at 12:03 AM, Kevin Brown <et...@google.com> wrote:
>
> > On Sat, Sep 6, 2008 at 11:19 PM, Eran Galili <er...@diffdoof.com> wrote:
> >
> > > There's something I don't understand.Why do you cache the XML for such
> a
> > > short time?
> > > Gadgets don't change every hour, or even every day.
> > > Why not parse the XML once and save it for a few days?
> > > That way the servers make fewer requests and parse XML less often.
> >
> >
> > Because nothing pisses developers off more than being told that they have
> > to
> > wait days for their changes to take effect.
> >
> >
> > >
> > >
> > >
> > > On Fri, Aug 29, 2008 at 7:50 PM, Kevin Brown <et...@google.com> wrote:
> > >
> > > > In addition to what Ian has said, there are actually two caches:
> > > >
> > > > - HTTP level. Generally best implemented as a distributed cache,
> since
> > > > there
> > > > will be lots of files here.
> > > > - Manifest (Gadget / message bundle xml) level. Best implemented as
> an
> > > > in-memory cache.
> > > >
> > > > The default implementations of both caches are in memory, but if
> you're
> > > > scaling up I'd recommend using a distributed cache for HTTP (and
> > strongly
> > > > recommend keeping the in-memory cache for the xml files).
> > > >
> > > > As far as caching policy goes, the specification *requires* that we
> do
> > > two
> > > > things:
> > > >
> > > > 1. Honor HTTP cache-control headers by default.
> > > > 2. If the user specified a parameter in the request (say, in
> > getProxyUrl
> > > or
> > > > makeRequest, or by rewriting), honor the cache time that they've
> > > specified.
> > > >
> > > > For gadget specs, the rules are not well defined, but the de facto
> > > standard
> > > > is to cache the files for a reasonably long time period (usually 1
> > hour).
> > > >
> > > > On Fri, Aug 29, 2008 at 3:45 AM, Ian Boston <ie...@tfd.co.uk> wrote:
> > > >
> > > > > For Java Shindig:
> > > > >
> > > > > The default cache is entirely in memory, wont page to disk and
> > doesn't
> > > > have
> > > > > distributed capabilities, but is light fast and works perfectly
> well.
> > > > >
> > > > > However
> > > > > In the current code base there is an implementation of the
> > > CacheProvider
> > > > > for EhCache which will do distributed Cache for you.
> > > > >
> > > > > Its not active by default, you need to add the
> > EhCacheGuiceCacheModule
> > > to
> > > > > the list of modules that are loaded.
> > > > > The configuration is in 2 places, ehcache.properties for the
> shindig
> > > > > specific settings, like JMX stats and the location of the ehcache
> xml
> > > > > configuration file, currently on the classpath at
> > > > > /org/apache/shindig/cache/ehcache/ehcacheConfig.xml
> > > > >
> > > > > You should consult ehcache documentation to configure distribution
> > and
> > > > > paging to disk
> http://ehcache.sourceforge.net/EhcacheUserGuide.html
> > > > >
> > > > > I have an Apache JCS implementation in waiting to go in, and there
> > may
> > > be
> > > > a
> > > > > memcachd impl in the pipeline.
> > > > >
> > > > >
> > > > > Code:
> > > > >
> http://svn.apache.org/repos/asf/incubator/shindig/trunk/java/common/
> > > > > src/main/java/org/apache/shindig/common/cache/ehcache/
> > > > >
> http://svn.apache.org/repos/asf/incubator/shindig/trunk/java/common/
> > > > > src/main/bundle/ehcache.properties
> > > > >
> http://svn.apache.org/repos/asf/incubator/shindig/trunk/java/common/
> > > > >
> > >
> src/main/bundle/org/apache/shindig/common/cache/ehcache/ehcacheConfig.xml
> > > > >
> > > > > The location may change as we get closer to a 1.0 release.
> > > > >
> > > > > HTH
> > > > > Ian
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > On 29 Aug 2008, at 10:14, Eran Galili wrote:
> > > > >
> > > > > Hi,
> > > > >>
> > > > >> I started implementing a container using shindig as my gadget
> server
> > > and
> > > > I
> > > > >> have a few questions regarding the way shindig gets and caches the
> > > > gadget
> > > > >> XML.
> > > > >>
> > > > >> First, I'd like to know what's considered the best practice for
> the
> > > > >> expiration time of cached gadgets? Does it have to be in some
> kind
> > of
> > > > >> range
> > > > >> for my container to be spec compliant?
> > > > >>
> > > > >> Secondly, does shindig cache the gadgets only on the machine's
> > memory,
> > > > or
> > > > >> does it also support using a distributed cache? If i set a very
> long
> > > > time
> > > > >> for my container to update the gadget XML (and perhaps allow the
> > > > developer
> > > > >> "update me" functionality), would it be reasonable to save it on
> my
> > > > >> database
> > > > >> (maybe save it already parsed)?
> > > > >>
> > > > >> Also if i keep an "Application Directory" of approved apps that i
> > > allow
> > > > on
> > > > >> my container, is it really necessary to send the gadget XML URL to
> > > when
> > > > >> sending a gadget rendering request? If i implement my own gadget
> > > server,
> > > > >> could i simply send an application Id (managed by my site) and on
> > the
> > > > >> server
> > > > >> side get the appropriate URL for that application (or even the XML
> > > > itself
> > > > >> from the cache)?
> > > > >>
> > > > >> I'm asking these questions because i think I'm missing something.
> I
> > > > would
> > > > >> like to minimize the amount of times my servers have to parse the
> > > gadget
> > > > >> xml. Saving the gadget spec already parsed would allow me to do
> > that.
> > > > >>
> > > > >> Thanks in advance,
> > > > >> Eran
> > > > >>
> > > > >
> > > > >
> > > >
> > >
> >
>
Re: Gadget XML cache
Posted by Eran Galili <er...@diffdoof.com>.
that can be easily treated with a "refresh me" feature.suppose we expose a
URL that accepts the gadgets spec URL as a parameter and refreshes the
cache.
it would deal with abusive requests of course and will be listed in the
discovery document...
On Mon, Sep 8, 2008 at 12:03 AM, Kevin Brown <et...@google.com> wrote:
> On Sat, Sep 6, 2008 at 11:19 PM, Eran Galili <er...@diffdoof.com> wrote:
>
> > There's something I don't understand.Why do you cache the XML for such a
> > short time?
> > Gadgets don't change every hour, or even every day.
> > Why not parse the XML once and save it for a few days?
> > That way the servers make fewer requests and parse XML less often.
>
>
> Because nothing pisses developers off more than being told that they have
> to
> wait days for their changes to take effect.
>
>
> >
> >
> >
> > On Fri, Aug 29, 2008 at 7:50 PM, Kevin Brown <et...@google.com> wrote:
> >
> > > In addition to what Ian has said, there are actually two caches:
> > >
> > > - HTTP level. Generally best implemented as a distributed cache, since
> > > there
> > > will be lots of files here.
> > > - Manifest (Gadget / message bundle xml) level. Best implemented as an
> > > in-memory cache.
> > >
> > > The default implementations of both caches are in memory, but if you're
> > > scaling up I'd recommend using a distributed cache for HTTP (and
> strongly
> > > recommend keeping the in-memory cache for the xml files).
> > >
> > > As far as caching policy goes, the specification *requires* that we do
> > two
> > > things:
> > >
> > > 1. Honor HTTP cache-control headers by default.
> > > 2. If the user specified a parameter in the request (say, in
> getProxyUrl
> > or
> > > makeRequest, or by rewriting), honor the cache time that they've
> > specified.
> > >
> > > For gadget specs, the rules are not well defined, but the de facto
> > standard
> > > is to cache the files for a reasonably long time period (usually 1
> hour).
> > >
> > > On Fri, Aug 29, 2008 at 3:45 AM, Ian Boston <ie...@tfd.co.uk> wrote:
> > >
> > > > For Java Shindig:
> > > >
> > > > The default cache is entirely in memory, wont page to disk and
> doesn't
> > > have
> > > > distributed capabilities, but is light fast and works perfectly well.
> > > >
> > > > However
> > > > In the current code base there is an implementation of the
> > CacheProvider
> > > > for EhCache which will do distributed Cache for you.
> > > >
> > > > Its not active by default, you need to add the
> EhCacheGuiceCacheModule
> > to
> > > > the list of modules that are loaded.
> > > > The configuration is in 2 places, ehcache.properties for the shindig
> > > > specific settings, like JMX stats and the location of the ehcache xml
> > > > configuration file, currently on the classpath at
> > > > /org/apache/shindig/cache/ehcache/ehcacheConfig.xml
> > > >
> > > > You should consult ehcache documentation to configure distribution
> and
> > > > paging to disk http://ehcache.sourceforge.net/EhcacheUserGuide.html
> > > >
> > > > I have an Apache JCS implementation in waiting to go in, and there
> may
> > be
> > > a
> > > > memcachd impl in the pipeline.
> > > >
> > > >
> > > > Code:
> > > > http://svn.apache.org/repos/asf/incubator/shindig/trunk/java/common/
> > > > src/main/java/org/apache/shindig/common/cache/ehcache/
> > > > http://svn.apache.org/repos/asf/incubator/shindig/trunk/java/common/
> > > > src/main/bundle/ehcache.properties
> > > > http://svn.apache.org/repos/asf/incubator/shindig/trunk/java/common/
> > > >
> > src/main/bundle/org/apache/shindig/common/cache/ehcache/ehcacheConfig.xml
> > > >
> > > > The location may change as we get closer to a 1.0 release.
> > > >
> > > > HTH
> > > > Ian
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > On 29 Aug 2008, at 10:14, Eran Galili wrote:
> > > >
> > > > Hi,
> > > >>
> > > >> I started implementing a container using shindig as my gadget server
> > and
> > > I
> > > >> have a few questions regarding the way shindig gets and caches the
> > > gadget
> > > >> XML.
> > > >>
> > > >> First, I'd like to know what's considered the best practice for the
> > > >> expiration time of cached gadgets? Does it have to be in some kind
> of
> > > >> range
> > > >> for my container to be spec compliant?
> > > >>
> > > >> Secondly, does shindig cache the gadgets only on the machine's
> memory,
> > > or
> > > >> does it also support using a distributed cache? If i set a very long
> > > time
> > > >> for my container to update the gadget XML (and perhaps allow the
> > > developer
> > > >> "update me" functionality), would it be reasonable to save it on my
> > > >> database
> > > >> (maybe save it already parsed)?
> > > >>
> > > >> Also if i keep an "Application Directory" of approved apps that i
> > allow
> > > on
> > > >> my container, is it really necessary to send the gadget XML URL to
> > when
> > > >> sending a gadget rendering request? If i implement my own gadget
> > server,
> > > >> could i simply send an application Id (managed by my site) and on
> the
> > > >> server
> > > >> side get the appropriate URL for that application (or even the XML
> > > itself
> > > >> from the cache)?
> > > >>
> > > >> I'm asking these questions because i think I'm missing something. I
> > > would
> > > >> like to minimize the amount of times my servers have to parse the
> > gadget
> > > >> xml. Saving the gadget spec already parsed would allow me to do
> that.
> > > >>
> > > >> Thanks in advance,
> > > >> Eran
> > > >>
> > > >
> > > >
> > >
> >
>
Re: Gadget XML cache
Posted by Kevin Brown <et...@google.com>.
On Sat, Sep 6, 2008 at 11:19 PM, Eran Galili <er...@diffdoof.com> wrote:
> There's something I don't understand.Why do you cache the XML for such a
> short time?
> Gadgets don't change every hour, or even every day.
> Why not parse the XML once and save it for a few days?
> That way the servers make fewer requests and parse XML less often.
Because nothing pisses developers off more than being told that they have to
wait days for their changes to take effect.
>
>
>
> On Fri, Aug 29, 2008 at 7:50 PM, Kevin Brown <et...@google.com> wrote:
>
> > In addition to what Ian has said, there are actually two caches:
> >
> > - HTTP level. Generally best implemented as a distributed cache, since
> > there
> > will be lots of files here.
> > - Manifest (Gadget / message bundle xml) level. Best implemented as an
> > in-memory cache.
> >
> > The default implementations of both caches are in memory, but if you're
> > scaling up I'd recommend using a distributed cache for HTTP (and strongly
> > recommend keeping the in-memory cache for the xml files).
> >
> > As far as caching policy goes, the specification *requires* that we do
> two
> > things:
> >
> > 1. Honor HTTP cache-control headers by default.
> > 2. If the user specified a parameter in the request (say, in getProxyUrl
> or
> > makeRequest, or by rewriting), honor the cache time that they've
> specified.
> >
> > For gadget specs, the rules are not well defined, but the de facto
> standard
> > is to cache the files for a reasonably long time period (usually 1 hour).
> >
> > On Fri, Aug 29, 2008 at 3:45 AM, Ian Boston <ie...@tfd.co.uk> wrote:
> >
> > > For Java Shindig:
> > >
> > > The default cache is entirely in memory, wont page to disk and doesn't
> > have
> > > distributed capabilities, but is light fast and works perfectly well.
> > >
> > > However
> > > In the current code base there is an implementation of the
> CacheProvider
> > > for EhCache which will do distributed Cache for you.
> > >
> > > Its not active by default, you need to add the EhCacheGuiceCacheModule
> to
> > > the list of modules that are loaded.
> > > The configuration is in 2 places, ehcache.properties for the shindig
> > > specific settings, like JMX stats and the location of the ehcache xml
> > > configuration file, currently on the classpath at
> > > /org/apache/shindig/cache/ehcache/ehcacheConfig.xml
> > >
> > > You should consult ehcache documentation to configure distribution and
> > > paging to disk http://ehcache.sourceforge.net/EhcacheUserGuide.html
> > >
> > > I have an Apache JCS implementation in waiting to go in, and there may
> be
> > a
> > > memcachd impl in the pipeline.
> > >
> > >
> > > Code:
> > > http://svn.apache.org/repos/asf/incubator/shindig/trunk/java/common/
> > > src/main/java/org/apache/shindig/common/cache/ehcache/
> > > http://svn.apache.org/repos/asf/incubator/shindig/trunk/java/common/
> > > src/main/bundle/ehcache.properties
> > > http://svn.apache.org/repos/asf/incubator/shindig/trunk/java/common/
> > >
> src/main/bundle/org/apache/shindig/common/cache/ehcache/ehcacheConfig.xml
> > >
> > > The location may change as we get closer to a 1.0 release.
> > >
> > > HTH
> > > Ian
> > >
> > >
> > >
> > >
> > >
> > > On 29 Aug 2008, at 10:14, Eran Galili wrote:
> > >
> > > Hi,
> > >>
> > >> I started implementing a container using shindig as my gadget server
> and
> > I
> > >> have a few questions regarding the way shindig gets and caches the
> > gadget
> > >> XML.
> > >>
> > >> First, I'd like to know what's considered the best practice for the
> > >> expiration time of cached gadgets? Does it have to be in some kind of
> > >> range
> > >> for my container to be spec compliant?
> > >>
> > >> Secondly, does shindig cache the gadgets only on the machine's memory,
> > or
> > >> does it also support using a distributed cache? If i set a very long
> > time
> > >> for my container to update the gadget XML (and perhaps allow the
> > developer
> > >> "update me" functionality), would it be reasonable to save it on my
> > >> database
> > >> (maybe save it already parsed)?
> > >>
> > >> Also if i keep an "Application Directory" of approved apps that i
> allow
> > on
> > >> my container, is it really necessary to send the gadget XML URL to
> when
> > >> sending a gadget rendering request? If i implement my own gadget
> server,
> > >> could i simply send an application Id (managed by my site) and on the
> > >> server
> > >> side get the appropriate URL for that application (or even the XML
> > itself
> > >> from the cache)?
> > >>
> > >> I'm asking these questions because i think I'm missing something. I
> > would
> > >> like to minimize the amount of times my servers have to parse the
> gadget
> > >> xml. Saving the gadget spec already parsed would allow me to do that.
> > >>
> > >> Thanks in advance,
> > >> Eran
> > >>
> > >
> > >
> >
>