[RFC] Altering the Execution Order in AxKit.pm

[Kip Hampton <kh...@totalcinema.com>]

Howdy AxDevers...

I think I've stumbled over a subtle logic flaw in AxKit.pm's 
main_handler method as it relates to Providers and caching. The 
"problem" lies in the fact that the Provider's process() method is 
called before determining whether or not the resource is cached.

The Provider's process() method is meant to indicate the Provider's 
intention and ability to get data for the current resource. That's fine 
and good, but for many non-file-based and/or dynamic Providers, 
process() needs to do *everything* required to serve the data in order 
to determine what to return (e,g. select from a DB-- return Declined if 
there's no data, run the app that generates the content-- return 
Declined if the app fails, or whatever).

In practice, this means that by having AxKit.pm's main_handler() call 
the Provider's process() before examining whether or not the resource is 
cached, the Providers will *still* have to hit the database, run the 
app, or whatever, even though the data for that resource may already be 
cached on the disk-- hence, the real benefit of caching is taken away 
from those Providers.

Not a hard fix, I think, but I wanted to invite discussion and check my 
blind spots before I whack into the main_handler() and friends.

Thoughts?

-kip

Re: [RFC] Altering the Execution Order in AxKit.pm

[Robin Berjon <ro...@knowscape.com>]

Kip Hampton wrote:
> I think that, after looking more closely still (too closely probably), 
> regenerating dynamic but unchanged content can be avoided as long as 1) 
> we limit the calls to Provider::process() to only the places where the 
> Provider is known to need to provide new content (e.g. where everything 
> is not already cached, including matching styles and so on). and 2) the 
> given Provider's key() method doesn't depend on the full content 
> generation cycle to happen just to get an appropriate unique value to 
> return.
> 
> Given that, Provider-level caching may be a solution in search of a 
> problem.

It's in search of documentation so that new users don't have to dig as deep into 
it as you did :)

--r

Re: [RFC] Altering the Execution Order in AxKit.pm

[Kip Hampton <kh...@totalcinema.com>]

Matt Sergeant wrote:
> On Thu, 13 Mar 2003, Kip Hampton wrote:

> So I'm happy for you to do whatever is required as long as it doesn't
> break regular file use. 

Agreed.

> Plus you can probably use the current AxKit::Cache
> module for doing provider level caching.

Yup, any other way would silly and wrong, IMO

I think that, after looking more closely still (too closely probably), 
regenerating dynamic but unchanged content can be avoided as long as 1) 
we limit the calls to Provider::process() to only the places where the 
Provider is known to need to provide new content (e.g. where everything 
is not already cached, including matching styles and so on). and 2) the 
given Provider's key() method doesn't depend on the full content 
generation cycle to happen just to get an appropriate unique value to 
return.

Given that, Provider-level caching may be a solution in search of a problem.

-kip

Re: [RFC] Altering the Execution Order in AxKit.pm

[Matt Sergeant <ma...@sergeant.org>]

On Thu, 13 Mar 2003, Kip Hampton wrote:

> Given this, I think that to cover the cases I'm trying to address, the
> *Provider* has to maintain its own content cache, then decide within
> itself whether to offer the cached copy of a resource or to reprocess to
> generate new content-- any other solution is either too fugly or leads
> to a brittle "chicken and egg" (which came first?) condition between the
> Cache and Provider classes. If I can think of a way to do it generically
> enough that other non-file-based/dynamic Providers can subclass and
> benefit from, I'll whip it out for review.
>
> Anyone have more thoughts? Matt?

Like most things that poke this area of AxKit, I don't always understand
the issues (mostly because I never got around to writing the axkit.com
CMS, which is the whole reason I designed it the way it is, but because I
never got around to developing it these subtleties never got uncovered).

So I'm happy for you to do whatever is required as long as it doesn't
break regular file use. Plus you can probably use the current AxKit::Cache
module for doing provider level caching.

-- 
<!-- Matt -->
<:->get a SMart net</:->
Spam trap - do not mail: spam-sig@spamtrap.messagelabs.com

Re: [RFC] Altering the Execution Order in AxKit.pm

[Kip Hampton <kh...@totalcinema.com>]

Jörg Walter wrote:
> On Thursday, 13. March 2003 22:53, Kip Hampton wrote:
> 
>>Howdy AxDevers...
>>
>>I think I've stumbled over a subtle logic flaw in AxKit.pm's
>>main_handler method as it relates to Providers and caching. The
>>"problem" lies in the fact that the Provider's process() method is
>>called before determining whether or not the resource is cached.

> Though, does it make a difference? The cache needs to call mtime(),
> which in turn is exensive as well... or not? Is there real money to be won, or 
> is this just a matter of cleanlyness?

Well, my goal is to make things friendler for alternate Providers that 
have to perform heavier-duty operations in order to construct the 
resource's content but, whose content may be static enough to want to 
take advantage of caching where appropriate.  After thinking more about 
it, though, it seems to me that there really *isn't* a flaw in the way 
things work now-- only a presumption that content will either be a 
static file or need to be generated for each request anyway.

Reality is that much of AxKit's behavior depends on the source content 
being parsed for every request (its how stylesheet PIs and most of the 
AxFooProcessor directives work) and what caching really avoids is the 
*transformation* overhead. The notion that the source of the content 
itself may be generated (and need to be cached in some cases) really 
isn't considered.

Given this, I think that to cover the cases I'm trying to address, the 
*Provider* has to maintain its own content cache, then decide within 
itself whether to offer the cached copy of a resource or to reprocess to 
generate new content-- any other solution is either too fugly or leads 
to a brittle "chicken and egg" (which came first?) condition between the 
Cache and Provider classes. If I can think of a way to do it generically 
enough that other non-file-based/dynamic Providers can subclass and 
benefit from, I'll whip it out for review.

Anyone have more thoughts? Matt?

-kip

Re: [RFC] Altering the Execution Order in AxKit.pm

[Jörg Walter <eh...@ich.bin.kein.hoschi.de>]

On Thursday, 13. March 2003 22:53, Kip Hampton wrote:
> Howdy AxDevers...
>
> I think I've stumbled over a subtle logic flaw in AxKit.pm's
> main_handler method as it relates to Providers and caching. The
> "problem" lies in the fact that the Provider's process() method is
> called before determining whether or not the resource is cached.
>
> The Provider's process() method is meant to indicate the Provider's
> intention and ability to get data for the current resource. That's fine
> and good, but for many non-file-based and/or dynamic Providers,
> process() needs to do *everything* required to serve the data in order
> to determine what to return (e,g. select from a DB-- return Declined if
> there's no data, run the app that generates the content-- return
> Declined if the app fails, or whatever).
>
> In practice, this means that by having AxKit.pm's main_handler() call
> the Provider's process() before examining whether or not the resource is
> cached, the Providers will *still* have to hit the database, run the
> app, or whatever, even though the data for that resource may already be
> cached on the disk-- hence, the real benefit of caching is taken away
> from those Providers.
>
> Not a hard fix, I think, but I wanted to invite discussion and check my
> blind spots before I whack into the main_handler() and friends.

Basically looks okay. Actually, I don't exactly understand the merits of 
process() anyways, since we have SetHandler/AddHandler. I never really got 
the difference between exists() and process().

Though, does it make a difference? The cache needs to call mtime(),
which in turn is exensive as well... or not? Is there real money to be won, or 
is this just a matter of cleanlyness?

-- 
CU
  Joerg

PGP Public Key at http://ich.bin.kein.hoschi.de/~trouble/public_key.asc
PGP Key fingerprint = D34F 57C4 99D8 8F16 E16E  7779 CDDC 41A4 4C48 6F94