You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lenya.apache.org by Andreas Hartmann <an...@apache.org> on 2008/02/01 19:29:28 UTC
Caching of fallback:// sources
Hi Lenya devs,
yesterday I did some profiling again. IMO it's crucial and quite urgent
to improve the performance, especially of the authoring environment.
The biggest performance penalty is caused by the resolving of fallback
sources. They aren't cached by default, because a cascade of files is
involved and for each resolving of the source the system checks if a
file in this cascade has been added or removed.
As a proof of concept, I added an LRUMap as cache to the
FallbackSourceFactory. This improves the performance a lot. In
combination with the Ajax menubar the authoring environment feels
notably faster.
Unfortunately I don't see a way to prune stale sources from the cache
automatically. We could either add a button to the GUI which allows to
clear the cache when a static resource (sitemap/XSLT/...) has been
changed. IMO this would be useful and sufficient. Additionally, it could
be triggered by the build process.
Another option would be an expiration time, though I don't like this
idea much because it makes testing very cumbersome (you have to wait
after deploying something to see the results).
In development mode, the fallback:// source cache would be disabled anyway.
Do you think this approach is useful? I'll do some more testing and
provide a patch asap.
-- Andreas
--
Andreas Hartmann, CTO
BeCompany GmbH
http://www.becompany.ch
Tel.: +41 (0) 43 818 57 01
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lenya.apache.org
For additional commands, e-mail: dev-help@lenya.apache.org
Re: Caching of fallback:// sources
Posted by Richard Frovarp <ri...@sendit.nodak.edu>.
Richard Frovarp wrote:
>
> XSLT is already cached, even with fallbacks. In fact it can make
> things act quite weirdly. You can force a reload of those by using touch.
>
> Caching the other stuff would certainly be good as well. Having a
> method to dump the cache for the publication inside of the publication
> would be a good thing.
>
> Richard
Come to think of it, it's probably the import that is being cached, not
the fallback part.
Richard
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lenya.apache.org
For additional commands, e-mail: dev-help@lenya.apache.org
Re: Caching of fallback:// sources
Posted by Richard Frovarp <ri...@sendit.nodak.edu>.
Andreas Hartmann wrote:
> Hi Lenya devs,
>
> yesterday I did some profiling again. IMO it's crucial and quite
> urgent to improve the performance, especially of the authoring
> environment.
> The biggest performance penalty is caused by the resolving of fallback
> sources. They aren't cached by default, because a cascade of files is
> involved and for each resolving of the source the system checks if a
> file in this cascade has been added or removed.
>
> As a proof of concept, I added an LRUMap as cache to the
> FallbackSourceFactory. This improves the performance a lot. In
> combination with the Ajax menubar the authoring environment feels
> notably faster.
>
> Unfortunately I don't see a way to prune stale sources from the cache
> automatically. We could either add a button to the GUI which allows to
> clear the cache when a static resource (sitemap/XSLT/...) has been
> changed. IMO this would be useful and sufficient. Additionally, it
> could be triggered by the build process.
>
> Another option would be an expiration time, though I don't like this
> idea much because it makes testing very cumbersome (you have to wait
> after deploying something to see the results).
>
> In development mode, the fallback:// source cache would be disabled
> anyway.
>
> Do you think this approach is useful? I'll do some more testing and
> provide a patch asap.
>
> -- Andreas
>
>
XSLT is already cached, even with fallbacks. In fact it can make things
act quite weirdly. You can force a reload of those by using touch.
Caching the other stuff would certainly be good as well. Having a method
to dump the cache for the publication inside of the publication would be
a good thing.
Richard
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lenya.apache.org
For additional commands, e-mail: dev-help@lenya.apache.org
Re: Caching of fallback:// sources
Posted by Andreas Hartmann <an...@apache.org>.
solprovider@apache.org schrieb:
> 1.3 has fallback built into the module: protocol. The filename chosen
> is cached. Subsequent calls check that filename using File.exists()
> before searching again. The double-colon syntax (module::///filename)
> is available to reset the cache for a Module, and is expected to be
> useful for development and for functions that create local files in a
> Module, which will be needed when online editing of Modules is
> possible.
>
> I dislike expecting the dev environment to differ from production.
> Caching the filename rather than the contents avoids issues with the
> file being changed.
That's a good point. In fact, caching the filename already reduces the
performance penalty a lot. I added an MRUMemoryStore which caches the
file:// URI for a fallback:// source URI. The publication ID is added as
a prefix to the cache key.
The store is disabled by default, there's no GUI to clear it yet.
-- Andreas
> I agree expiration is poor design.
>
> solprovider
>
> On 2/1/08, Andreas Hartmann <an...@apache.org> wrote:
>> Hi Lenya devs,
>>
>> yesterday I did some profiling again. IMO it's crucial and quite urgent
>> to improve the performance, especially of the authoring environment.
>> The biggest performance penalty is caused by the resolving of fallback
>> sources. They aren't cached by default, because a cascade of files is
>> involved and for each resolving of the source the system checks if a
>> file in this cascade has been added or removed.
>>
>> As a proof of concept, I added an LRUMap as cache to the
>> FallbackSourceFactory. This improves the performance a lot. In
>> combination with the Ajax menubar the authoring environment feels
>> notably faster.
>>
>> Unfortunately I don't see a way to prune stale sources from the cache
>> automatically. We could either add a button to the GUI which allows to
>> clear the cache when a static resource (sitemap/XSLT/...) has been
>> changed. IMO this would be useful and sufficient. Additionally, it could
>> be triggered by the build process.
>>
>> Another option would be an expiration time, though I don't like this
>> idea much because it makes testing very cumbersome (you have to wait
>> after deploying something to see the results).
>>
>> In development mode, the fallback:// source cache would be disabled anyway.
>>
>> Do you think this approach is useful? I'll do some more testing and
>> provide a patch asap.
>> Andreas Hartmann, CTO
--
Andreas Hartmann, CTO
BeCompany GmbH
http://www.becompany.ch
Tel.: +41 (0) 43 818 57 01
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lenya.apache.org
For additional commands, e-mail: dev-help@lenya.apache.org
Re: Caching of fallback:// sources
Posted by so...@apache.org.
1.3 has fallback built into the module: protocol. The filename chosen
is cached. Subsequent calls check that filename using File.exists()
before searching again. The double-colon syntax (module::///filename)
is available to reset the cache for a Module, and is expected to be
useful for development and for functions that create local files in a
Module, which will be needed when online editing of Modules is
possible.
I dislike expecting the dev environment to differ from production.
Caching the filename rather than the contents avoids issues with the
file being changed.
I agree expiration is poor design.
solprovider
On 2/1/08, Andreas Hartmann <an...@apache.org> wrote:
> Hi Lenya devs,
>
> yesterday I did some profiling again. IMO it's crucial and quite urgent
> to improve the performance, especially of the authoring environment.
> The biggest performance penalty is caused by the resolving of fallback
> sources. They aren't cached by default, because a cascade of files is
> involved and for each resolving of the source the system checks if a
> file in this cascade has been added or removed.
>
> As a proof of concept, I added an LRUMap as cache to the
> FallbackSourceFactory. This improves the performance a lot. In
> combination with the Ajax menubar the authoring environment feels
> notably faster.
>
> Unfortunately I don't see a way to prune stale sources from the cache
> automatically. We could either add a button to the GUI which allows to
> clear the cache when a static resource (sitemap/XSLT/...) has been
> changed. IMO this would be useful and sufficient. Additionally, it could
> be triggered by the build process.
>
> Another option would be an expiration time, though I don't like this
> idea much because it makes testing very cumbersome (you have to wait
> after deploying something to see the results).
>
> In development mode, the fallback:// source cache would be disabled anyway.
>
> Do you think this approach is useful? I'll do some more testing and
> provide a patch asap.
> Andreas Hartmann, CTO
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lenya.apache.org
For additional commands, e-mail: dev-help@lenya.apache.org
Re: Caching of fallback:// sources
Posted by Andreas Hartmann <an...@apache.org>.
Thorsten Scherler schrieb:
> On Fri, 2008-02-01 at 19:29 +0100, Andreas Hartmann wrote:
>> Hi Lenya devs,
> ...
>> Unfortunately I don't see a way to prune stale sources from the cache
>> automatically.
>
> Validity would not work?
The validity check is so expensive that IMO we can't afford to do it,
because it requires checking the existence of all files in the cascade
(in the worst case). Profiling showed that this check is the single most
expensive operation among the org.apache.lenya.* classes. When
requesting the homepage, about 180 (!) fallback sources are checked.
-- Andreas
--
Andreas Hartmann, CTO
BeCompany GmbH
http://www.becompany.ch
Tel.: +41 (0) 43 818 57 01
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lenya.apache.org
For additional commands, e-mail: dev-help@lenya.apache.org
Re: Caching of fallback:// sources
Posted by Thorsten Scherler <th...@juntadeandalucia.es>.
On Fri, 2008-02-01 at 19:29 +0100, Andreas Hartmann wrote:
> Hi Lenya devs,
...
> Unfortunately I don't see a way to prune stale sources from the cache
> automatically.
Validity would not work?
salu2
--
Thorsten Scherler thorsten.at.apache.org
Open Source Java consulting, training and solutions
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lenya.apache.org
For additional commands, e-mail: dev-help@lenya.apache.org