Caching and key collision?

[Ku...@heinrich-partner.de]

Hi,

I noticed caching is done by generating a key with generateKey. This 
returns a long, which is commonly generated through hashing. The 
documentation says "This key must be unique inside the space of this 
component", which hashing does not guarantee. Won't this lead the cache to 
confusing two different cacheable objects (files/server pages/etc.) which 
accidently have the same cache value? This sure is an unlikely event, but 
ist still can happen. Also, on pipelines with many program steps, the 
cache generates very long file names some operatings systems can't cope 
with. 

Regards,
Jochen

RE: Caching and key collision?

[Gerhard Froehlich <g-...@gmx.de>]

Hi,

>From: Berin Loritsch [mailto:bloritsch@apache.org]
>
>
>Kuhnle@heinrich-partner.de wrote:
>
>> 
>> Hi,
>> 
>> I noticed caching is done by generating a key with generateKey. This 
>> returns a long, which is commonly generated through hashing. The 
>> documentation says "This key must be unique inside the space of this 
>> component", which hashing does not guarantee. Won't this lead the cache 
>> to confusing two different cacheable objects (files/server pages/etc.) 
>> which accidently have the same cache value? This sure is an unlikely 
>> event, but ist still can happen. Also, on pipelines with many program 
>> steps, the cache generates very long file names some operatings systems 
>> can't cope with.
>
>
>Regarding the last issue, I have been a victim of that.  Windows 9x
>cannot handle the long filenames generated by the cache.

I recommend to switch off the filesystem swaping in the current version (2.0).
It's buggy. In the next release (2.0.1) there will be a much better swapping
system and this problem will be solved. I'm sorry.

There is a switch called "filesystem" in the <store/> configuration where 
you can switch off the filesystem swapping!

>However, Windows NT and just about every Unix can, so to me this is a
>minor issue--who in their right mind would want to serve anything from
>Windows?
>
>
>A more fundamental issue however is in regards to how quickly an OS
>
>is on seeks.  Windows favors shallow directories and large numbers of files
>in a directory.  UNIX favors deep directories and small numbers of files
>in a directory.
>
>When you go to the extreme that each does not like, you *really* notice
>the slowness.  I have had my Windows machine block for 15 seconds at a
>time while navigating very deep hierarchies (this is especially true on
>Win 9x).  Again, while working on a Perl webapp that cached info on the
>drive, Linux would pause for a few seconds (5-6) when there were about
>1000 entries in a directory before displaying the contents of the directory.
>
>If we could tune our cache to favor what the OS favors, it would add
>just that extra something to make the cache even faster.

As I said in next version 2.0.1 there will be much better solution, which
fix this "long filename" issues. Please stand by!

Sorry, but I'm still learning!

Greets
  Gerhard
 
"God put me on this Earth to accomplish a certain number of things. 
Right now, I am so far behind I shall never die."


---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org

RE: Caching and key collision?

[Matt Sergeant <ma...@sergeant.org>]

On Tue, 15 Jan 2002, Vadim Gritsenko wrote:

> > When you go to the extreme that each does not like, you *really*
> notice
> > the slowness.  I have had my Windows machine block for 15 seconds at a
> > time while navigating very deep hierarchies (this is especially true
> on
> > Win 9x).  Again, while working on a Perl webapp that cached info on
> the
> > drive, Linux would pause for a few seconds (5-6) when there were about
> > 1000 entries in a directory before displaying the contents of the
> directory.
> >
> > If we could tune our cache to favor what the OS favors, it would add
> > just that extra something to make the cache even faster.
>
> Have you seen Gerhard's JispFilesystemStore? It promises to solve issue
> on both platforms, on Win as well as on *NIX.

[as a side note I see people's quoting ability *still* sucks here]

A simple implementation that works well on both Windows and Unix is to
simply use two directories deep. That's what AxKit does, and it improves
performance a bit (though it makes cache management tools slightly more
complex).

-- 
<!-- Matt -->
<:->Get a smart net</:->



---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org

RE: Caching and key collision?

[Vadim Gritsenko <va...@verizon.net>]

> From: Berin Loritsch [mailto:bloritsch@apache.org]
> 
> Kuhnle@heinrich-partner.de wrote:
> 
> >
> > Hi,
> >
> > I noticed caching is done by generating a key with generateKey. This
> > returns a long, which is commonly generated through hashing. The
> > documentation says "This key must be unique inside the space of this
> > component", which hashing does not guarantee. Won't this lead the
cache
> > to confusing two different cacheable objects (files/server
pages/etc.)
> > which accidently have the same cache value? This sure is an unlikely
> > event, but ist still can happen. Also, on pipelines with many
program
> > steps, the cache generates very long file names some operatings
systems
> > can't cope with.
> 
> 
> Regarding the last issue, I have been a victim of that.  Windows 9x
> cannot handle the long filenames generated by the cache.
> 
> However, Windows NT and just about every Unix can, so to me this is a
> minor issue--who in their right mind would want to serve anything from
> Windows?
> 
> 
> A more fundamental issue however is in regards to how quickly an OS
> 
> is on seeks.  Windows favors shallow directories and large numbers of
files
> in a directory.  UNIX favors deep directories and small numbers of
files
> in a directory.
> 
> When you go to the extreme that each does not like, you *really*
notice
> the slowness.  I have had my Windows machine block for 15 seconds at a
> time while navigating very deep hierarchies (this is especially true
on
> Win 9x).  Again, while working on a Perl webapp that cached info on
the
> drive, Linux would pause for a few seconds (5-6) when there were about
> 1000 entries in a directory before displaying the contents of the
directory.
> 
> If we could tune our cache to favor what the OS favors, it would add
> just that extra something to make the cache even faster.

Have you seen Gerhard's JispFilesystemStore? It promises to solve issue
on both platforms, on Win as well as on *NIX.

Vadim


---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org

Re: Caching and key collision?

[Berin Loritsch <bl...@apache.org>]

Kuhnle@heinrich-partner.de wrote:

> 
> Hi,
> 
> I noticed caching is done by generating a key with generateKey. This 
> returns a long, which is commonly generated through hashing. The 
> documentation says "This key must be unique inside the space of this 
> component", which hashing does not guarantee. Won't this lead the cache 
> to confusing two different cacheable objects (files/server pages/etc.) 
> which accidently have the same cache value? This sure is an unlikely 
> event, but ist still can happen. Also, on pipelines with many program 
> steps, the cache generates very long file names some operatings systems 
> can't cope with.


Regarding the last issue, I have been a victim of that.  Windows 9x
cannot handle the long filenames generated by the cache.

However, Windows NT and just about every Unix can, so to me this is a
minor issue--who in their right mind would want to serve anything from
Windows?


A more fundamental issue however is in regards to how quickly an OS

is on seeks.  Windows favors shallow directories and large numbers of files
in a directory.  UNIX favors deep directories and small numbers of files
in a directory.

When you go to the extreme that each does not like, you *really* notice
the slowness.  I have had my Windows machine block for 15 seconds at a
time while navigating very deep hierarchies (this is especially true on
Win 9x).  Again, while working on a Perl webapp that cached info on the
drive, Linux would pause for a few seconds (5-6) when there were about
1000 entries in a directory before displaying the contents of the directory.

If we could tune our cache to favor what the OS favors, it would add
just that extra something to make the cache even faster.




-- 

"They that give up essential liberty to obtain a little temporary safety
  deserve neither liberty nor safety."
                 - Benjamin Franklin


---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org