You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jetspeed-user@portals.apache.org by carlos beltran <cb...@aitek.it> on 2001/05/04 17:10:28 UTC
Jetspeed 1.3.a1 cache system: lighting the darkness!
We have been fighting a little with the cache system in Jetspeed 1.3.a1.
Our goal was that of showing real time data in a portlet presenting a
RSS page generated with JSP. But we had problems because the portlet
content was not refreshed when the user was hitting the refresh button
in his web browser.
Why?
Answer: cache system.
Two different kinds of cache are acting in the portlet; the first one is
caching the portlet object in memory and the second one is caching the
URL that the portlet is using. This second cache is called the
DiskCache. There is a third class of cache, but it is only working for
the channels defined in OCS and it is known as FeedCache.
The DiskCache has mainly two types of URL: locals and remotes. The
locals are not cached and are directly get from the URL source ( so if
your page is local you will not have problems). The DiskCache recognizes
as locals the next types of URLs:
/test/test
http://localhost
http://127.0.0.1
http://ourserver.domainame
Notice that the URL "http://ourserver" is considered as remote...
The remotes URL are refreshed by the DiskCacheDaemon. This daemon runs
with a interval defined in the JR.p; each time it runs it takes all the
URL from the DiskCache and renews that ones that are remote and have
passed over their "expiration time". This expiration time is hard coded
in the source of DiskCacheEntry.java and it is equal to 15 minutes.
So what happens if you want to refresh your remote URL in the DiskCache
with a interval lower than 15 minutes, ..well.., you can't, unless you
change DiskCacheEntry to modify the default expiration time.
Then, playing with the "expiration time" and the daemon interval, you
would be able to control the refresh of your portlet lower that 15
minutes ( in fact we wanted a refresh time of 1 minute).
Finally,In both, local and remote URL, we would suggest to "jump" the
"portlet cache". This can be done overwriting the isCacheable() method
as it has been explained in other messages.
Hope this help someone,
Greetings,
Carlos.
---------------------------------------------------------------------
To unsubscribe, e-mail: jetspeed-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jetspeed-user-help@jakarta.apache.org
Re: Jetspeed 1.3.a1 cache system: lighting the darkness!
Posted by Santiago Gala <sg...@hisitech.com>.
carlos beltran wrote:
> We have been fighting a little with the cache system in Jetspeed 1.3.a1.
> Our goal was that of showing real time data in a portlet presenting a
> RSS page generated with JSP. But we had problems because the portlet
> content was not refreshed when the user was hitting the refresh button
> in his web browser.
>
> Why?
> Answer: cache system.
>
> Two different kinds of cache are acting in the portlet; the first one is
> caching the portlet object in memory and the second one is caching the
> URL that the portlet is using. This second cache is called the
> DiskCache. There is a third class of cache, but it is only working for
> the channels defined in OCS and it is known as FeedCache.
The FeedCache is new for me. Either is something new coming from Raphael
changes or there is a wrong javadoc somewhere. Probably is the DiskCache
by other name.
> The DiskCache has mainly two types of URL: locals and remotes. The
> locals are not cached and are directly get from the URL source ( so if
> your page is local you will not have problems). The DiskCache recognizes
> as locals the next types of URLs:
>
> /test/test
> http://localhost
> http://127.0.0.1
> http://ourserver.domainame
>
> Notice that the URL "http://ourserver" is considered as remote...
>
And also protocols listed in a specific list in JR.p (file:, jndi:,
classloader:, ...)
> The remotes URL are refreshed by the DiskCacheDaemon. This daemon runs
> with a interval defined in the JR.p; each time it runs it takes all the
> URL from the DiskCache and renews that ones that are remote and have
> passed over their "expiration time". This expiration time is hard coded
> in the source of DiskCacheEntry.java and it is equal to 15 minutes.
>
Not exactly. The initial expiration time is set to 15 minutes. When the
diskcacheDaemon runs, it is set to the Expires: header of the URL (if it
exists and it is > 0)
But the important thing is that the URLs will only be checked when the
DiskCacheDaemon runs (all of them). This is every hour, by default.
> So what happens if you want to refresh your remote URL in the DiskCache
> with a interval lower than 15 minutes, ..well.., you can't, unless you
> change DiskCacheEntry to modify the default expiration time.
>
If the url returns a Expires: header it should be possible to have it
refreshed every time the daemon runs.
> Then, playing with the "expiration time" and the daemon interval, you
> would be able to control the refresh of your portlet lower that 15
> minutes ( in fact we wanted a refresh time of 1 minute).
>
> Finally,In both, local and remote URL, we would suggest to "jump" the
> "portlet cache". This can be done overwriting the isCacheable() method
> as it has been explained in other messages.
>
This brings a lot of overhead for some portlets, as a XSLT transform
will have to be performed for every getContent() call. So, it depends on
your concrete portlet if it is needed or not.
> Hope this help someone,
Yes, at least we are beginning to have reasonable documentation on the
behaviour.
For the next iteration I would have:
- resources can be cacheable or not. The selection will be done via
configurable regular expressions, and not through a fixed strategy. The
minimum and maximum expiration times will be settable for each regexp,
with a global default.
- the daemon will run with a much smaller granularity (every minute?)
but it will check only expired URLs. Currently it makes a request for
every resource with IfModifiedSince: set, and skips refreshing if it
gets a "Not Modified" answer. This saves a lot of work, but still is not
completely trustable (see Apacheweek below). Also, a lot of channels do
not handle these headers.
- The OCS entries will override update times using the OCS information
coming in the feed (they are like update --> once daily ...) They should
update global defaults, but NOT url-regexp specified values.
- A minimum and a maximum expiration time will be set for every entry.
The minimum will be used if Expires: is not meaningful (<=0 or in the
past), and the maximum to avoid pathological cases, where the date
returned is the script modification date, not the channel generation.
For instance, the ApacheWeek entry honors the IfModifiedSince: and
Exppires: headers BUT, when the war is expanded, the cache file gets the
modification date of the war expansion, not the true one in the cache.
So, unless you delete the cache entry, ApacheWeek will not be updated
until roughly 24 hours after you expand the war.
---------------------------------------------------------------------
To unsubscribe, e-mail: jetspeed-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jetspeed-user-help@jakarta.apache.org
Re: Jetspeed 1.3.a1 cache system: lighting the darkness!
Posted by Santiago Gala <sg...@hisitech.com>.
raphael.luta@networks.groupvu.com wrote:
> Santiago Gala wrote:
>
>>raphael.luta@networks.groupvu.com wrote:
>>
>>
>>I would like a lot that AbstractPortlet be really abstract. It is very
>>misleading that a class named Abstract has a concrete implementation.
>>Further, if it implements things that are not abstract at all, as a
>>caching behaviour coupled with a way to retrieve cached content. We have
>>had quite a few errors related with the fact that AbstractPortlet
>>implements too many things.
>>
>>
>
> I agree with the idea but do you see a mean to reduce the amount of functionality
> implemented by the AbstractPortlet (or Abstract*) without breaking compatibility ?
>
> I would think that AbstractPortlet should *not* implement Cacheable and should not
> provide an implementation for getContent() at the very least. If people are OK with
> dropping the Cacheable interface from AbstractPortlet, I'm definitely +1 but I fear
> this may have border effects on existing code.
>
This was also my concern. AbstractPortlet implies Cacheable, and a way
to retrieve content from cache.
I think we can wait until we go to the new Portlet API, which will break
most of the code, for such changes.
>
>>>OK, I'll see if I can integrate that into 1.3a2 without breaking existing code...
>>>
>>>While I'm testing waters here :
>>>
>>>Are there users out there who depend on PortletSet, PortletControl and PortletConfig
>>>being real classes rather that interfaces (for things like new PortletConfig() or
>>>new PortletSet() ) ?
>>>
>>>If not, I'd like to make these interfaces and provide the existing implementations
>>>as BasePortletConfig(), BasePortletSet() and AbstractPortletControl(). That will
>>>give us a lot of additionnal flexibility.
>>>
>>>
>>Again, I would pray that any "Abstract..." be really abstract. If it is
>>not abstract --> "Base..."
>>
>>
>
> The distinction I see between Base* and Abstract* is that a Base* class is likely
> to be the only implementation of this interface whereas Abstract* is a helper class
> for implementing an interface which is expected to have many implementations (Portlet,
> PortletControl, PortletController).
>
> Does that make sense to you ?
Yes. What I don't like is using Abstract for an implementation that
implements all methods. So if you subclass it with an empty behaviour,
you get something that compiles, but it will not run probably.
We have already had a bug related with this, as the caching behaviour
was not expected when a guy subclassed AbstractPortlet. So, for me the
problem is that the name of the class created a fake "implicit contract"
with the developer. If the class would have been called
"AbstractCacheablePortlet" this would not have been happened.
But I agree that it is not the time of moving this code too much. We
have already plenty to do. :)
---------------------------------------------------------------------
To unsubscribe, e-mail: jetspeed-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jetspeed-user-help@jakarta.apache.org
Re: Jetspeed 1.3.a1 cache system: lighting the darkness!
Posted by ra...@networks.groupvu.com.
Santiago Gala wrote:
>
> raphael.luta@networks.groupvu.com wrote:
>
>
> I would like a lot that AbstractPortlet be really abstract. It is very
> misleading that a class named Abstract has a concrete implementation.
> Further, if it implements things that are not abstract at all, as a
> caching behaviour coupled with a way to retrieve cached content. We have
> had quite a few errors related with the fact that AbstractPortlet
> implements too many things.
>
I agree with the idea but do you see a mean to reduce the amount of functionality
implemented by the AbstractPortlet (or Abstract*) without breaking compatibility ?
I would think that AbstractPortlet should *not* implement Cacheable and should not
provide an implementation for getContent() at the very least. If people are OK with
dropping the Cacheable interface from AbstractPortlet, I'm definitely +1 but I fear
this may have border effects on existing code.
> > OK, I'll see if I can integrate that into 1.3a2 without breaking existing code...
> >
> > While I'm testing waters here :
> >
> > Are there users out there who depend on PortletSet, PortletControl and PortletConfig
> > being real classes rather that interfaces (for things like new PortletConfig() or
> > new PortletSet() ) ?
> >
> > If not, I'd like to make these interfaces and provide the existing implementations
> > as BasePortletConfig(), BasePortletSet() and AbstractPortletControl(). That will
> > give us a lot of additionnal flexibility.
> >
>
> Again, I would pray that any "Abstract..." be really abstract. If it is
> not abstract --> "Base..."
>
The distinction I see between Base* and Abstract* is that a Base* class is likely
to be the only implementation of this interface whereas Abstract* is a helper class
for implementing an interface which is expected to have many implementations (Portlet,
PortletControl, PortletController).
Does that make sense to you ?
--
Raphaël Luta - luta.raphael@networks.vivendi.net
---------------------------------------------------------------------
To unsubscribe, e-mail: jetspeed-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jetspeed-user-help@jakarta.apache.org
Re: Jetspeed 1.3.a1 cache system: lighting the darkness!
Posted by Santiago Gala <sg...@hisitech.com>.
raphael.luta@networks.groupvu.com wrote:
> carlos beltran wrote:
>
>>> Very good explanation ! Do you think you code write this in xdocs ?
>>
>> Sure!!
>> But i dont have commit access. May be someone with access can do it?
>>
>
>
> Either send it to the jetspeed-dev list or send it directly to me raphael@apache.org,
> I'll review and commit it.
>
>
>>> BTW, I'm currently working on integrating Roberto Carrasco velocity work into
>>> Jetspeed and considering to remove the Cacheable interface from the base
>>> Portlet interface.
>>>
>>> This way only those portlet implementations that want to be Cacheable need to
>>> implement this feature.
>>>
>>> To keep backward compatibility, we could have an AbstractPortlet implementing
>>> Cacheable and a DynamicAbstractPortlet that would not implement it (I would
>>> prefer AbstractPortlet and CacheableAbstractPortlet but this may break portlets
>>> extending AbstractPortlet and relying on the caheable interface methods...)
>>>
>>> Would that suit you ?
>>
>> Yes, I think it is a good idea.
>>
>
>
I would like a lot that AbstractPortlet be really abstract. It is very
misleading that a class named Abstract has a concrete implementation.
Further, if it implements things that are not abstract at all, as a
caching behaviour coupled with a way to retrieve cached content. We have
had quite a few errors related with the fact that AbstractPortlet
implements too many things.
> OK, I'll see if I can integrate that into 1.3a2 without breaking existing code...
>
> While I'm testing waters here :
>
> Are there users out there who depend on PortletSet, PortletControl and PortletConfig
> being real classes rather that interfaces (for things like new PortletConfig() or
> new PortletSet() ) ?
>
> If not, I'd like to make these interfaces and provide the existing implementations
> as BasePortletConfig(), BasePortletSet() and AbstractPortletControl(). That will
> give us a lot of additionnal flexibility.
>
Again, I would pray that any "Abstract..." be really abstract. If it is
not abstract --> "Base..."
>
> --
> Raphaël Luta - luta.raphael@networks.vivendi.net
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: jetspeed-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: jetspeed-user-help@jakarta.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: jetspeed-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jetspeed-user-help@jakarta.apache.org
Re: Jetspeed 1.3.a1 cache system: lighting the darkness!
Posted by ra...@networks.groupvu.com.
carlos beltran wrote:
>
> > Very good explanation ! Do you think you code write this in xdocs ?
>
> Sure!!
> But i dont have commit access. May be someone with access can do it?
>
Either send it to the jetspeed-dev list or send it directly to me raphael@apache.org,
I'll review and commit it.
> > BTW, I'm currently working on integrating Roberto Carrasco velocity work into
> > Jetspeed and considering to remove the Cacheable interface from the base
> > Portlet interface.
> >
> > This way only those portlet implementations that want to be Cacheable need to
> > implement this feature.
> >
> > To keep backward compatibility, we could have an AbstractPortlet implementing
> > Cacheable and a DynamicAbstractPortlet that would not implement it (I would
> > prefer AbstractPortlet and CacheableAbstractPortlet but this may break portlets
> > extending AbstractPortlet and relying on the caheable interface methods...)
> >
> > Would that suit you ?
>
> Yes, I think it is a good idea.
>
OK, I'll see if I can integrate that into 1.3a2 without breaking existing code...
While I'm testing waters here :
Are there users out there who depend on PortletSet, PortletControl and PortletConfig
being real classes rather that interfaces (for things like new PortletConfig() or
new PortletSet() ) ?
If not, I'd like to make these interfaces and provide the existing implementations
as BasePortletConfig(), BasePortletSet() and AbstractPortletControl(). That will
give us a lot of additionnal flexibility.
--
Raphaël Luta - luta.raphael@networks.vivendi.net
---------------------------------------------------------------------
To unsubscribe, e-mail: jetspeed-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jetspeed-dev-help@jakarta.apache.org
Re: Jetspeed 1.3.a1 cache system: lighting the darkness!
Posted by ra...@networks.groupvu.com.
carlos beltran wrote:
>
> > Very good explanation ! Do you think you code write this in xdocs ?
>
> Sure!!
> But i dont have commit access. May be someone with access can do it?
>
Either send it to the jetspeed-dev list or send it directly to me raphael@apache.org,
I'll review and commit it.
> > BTW, I'm currently working on integrating Roberto Carrasco velocity work into
> > Jetspeed and considering to remove the Cacheable interface from the base
> > Portlet interface.
> >
> > This way only those portlet implementations that want to be Cacheable need to
> > implement this feature.
> >
> > To keep backward compatibility, we could have an AbstractPortlet implementing
> > Cacheable and a DynamicAbstractPortlet that would not implement it (I would
> > prefer AbstractPortlet and CacheableAbstractPortlet but this may break portlets
> > extending AbstractPortlet and relying on the caheable interface methods...)
> >
> > Would that suit you ?
>
> Yes, I think it is a good idea.
>
OK, I'll see if I can integrate that into 1.3a2 without breaking existing code...
While I'm testing waters here :
Are there users out there who depend on PortletSet, PortletControl and PortletConfig
being real classes rather that interfaces (for things like new PortletConfig() or
new PortletSet() ) ?
If not, I'd like to make these interfaces and provide the existing implementations
as BasePortletConfig(), BasePortletSet() and AbstractPortletControl(). That will
give us a lot of additionnal flexibility.
--
Raphaël Luta - luta.raphael@networks.vivendi.net
---------------------------------------------------------------------
To unsubscribe, e-mail: jetspeed-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jetspeed-user-help@jakarta.apache.org
Re: Jetspeed 1.3.a1 cache system: lighting the darkness!
Posted by carlos beltran <cb...@aitek.it>.
> Very good explanation ! Do you think you code write this in xdocs ?
Sure!!
But i dont have commit access. May be someone with access can do it?
> BTW, I'm currently working on integrating Roberto Carrasco velocity work into
> Jetspeed and considering to remove the Cacheable interface from the base
> Portlet interface.
>
> This way only those portlet implementations that want to be Cacheable need to
> implement this feature.
>
> To keep backward compatibility, we could have an AbstractPortlet implementing
> Cacheable and a DynamicAbstractPortlet that would not implement it (I would
> prefer AbstractPortlet and CacheableAbstractPortlet but this may break portlets
> extending AbstractPortlet and relying on the caheable interface methods...)
>
> Would that suit you ?
Yes, I think it is a good idea.
Carlos.
---------------------------------------------------------------------
To unsubscribe, e-mail: jetspeed-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jetspeed-user-help@jakarta.apache.org
Re: Jetspeed 1.3.a1 cache system: lighting the darkness!
Posted by ra...@networks.groupvu.com.
carlos beltran wrote:
>
> We have been fighting a little with the cache system in Jetspeed 1.3.a1.
> Our goal was that of showing real time data in a portlet presenting a
> RSS page generated with JSP. But we had problems because the portlet
> content was not refreshed when the user was hitting the refresh button
> in his web browser.
>
> Why?
> Answer: cache system.
>
> Two different kinds of cache are acting in the portlet; the first one is
> caching the portlet object in memory and the second one is caching the
> URL that the portlet is using. This second cache is called the
> DiskCache. There is a third class of cache, but it is only working for
> the channels defined in OCS and it is known as FeedCache.
> The DiskCache has mainly two types of URL: locals and remotes. The
> locals are not cached and are directly get from the URL source ( so if
> your page is local you will not have problems). The DiskCache recognizes
> as locals the next types of URLs:
>
> /test/test
> http://localhost
> http://127.0.0.1
> http://ourserver.domainame
>
> Notice that the URL "http://ourserver" is considered as remote...
>
> The remotes URL are refreshed by the DiskCacheDaemon. This daemon runs
> with a interval defined in the JR.p; each time it runs it takes all the
> URL from the DiskCache and renews that ones that are remote and have
> passed over their "expiration time". This expiration time is hard coded
> in the source of DiskCacheEntry.java and it is equal to 15 minutes.
>
> So what happens if you want to refresh your remote URL in the DiskCache
> with a interval lower than 15 minutes, ..well.., you can't, unless you
> change DiskCacheEntry to modify the default expiration time.
>
> Then, playing with the "expiration time" and the daemon interval, you
> would be able to control the refresh of your portlet lower that 15
> minutes ( in fact we wanted a refresh time of 1 minute).
>
> Finally,In both, local and remote URL, we would suggest to "jump" the
> "portlet cache". This can be done overwriting the isCacheable() method
> as it has been explained in other messages.
>
Very good explanation ! Do you think you code write this in xdocs ?
BTW, I'm currently working on integrating Roberto Carrasco velocity work into
Jetspeed and considering to remove the Cacheable interface from the base
Portlet interface.
This way only those portlet implementations that want to be Cacheable need to
implement this feature.
To keep backward compatibility, we could have an AbstractPortlet implementing
Cacheable and a DynamicAbstractPortlet that would not implement it (I would
prefer AbstractPortlet and CacheableAbstractPortlet but this may break portlets
extending AbstractPortlet and relying on the caheable interface methods...)
Would that suit you ?
--
Raphaël Luta - luta.raphael@networks.vivendi.net
---------------------------------------------------------------------
To unsubscribe, e-mail: jetspeed-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jetspeed-user-help@jakarta.apache.org