You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cocoon.apache.org by Gerhard Froehlich <g-...@gmx.de> on 2001/12/12 11:55:34 UTC
[VOTE] adding Jisp based Store persistence to the Scratchpad
Hi,
as I said there's much work to do and
as a fresh baked Committer I call
a Vote, now:
I want to improve the Store Persistence
of the current Store implementation. There-
fore I want to try to implement a
JispFilesystemStore
http://www.coyotegulch.com/jisp/index.html
But whole thing must be evaluated and tried
first. Therefore I want to put this new
Implementation in the Scratchpad. So we can
play around and don't bother the current
main branch.
Please give your votes.
Gerhard
----------------------------
I just found the last bug...
----------------------------
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org
RE: [VOTE] adding Jisp based Store persistence to the Scratchpad
Posted by Gerhard Froehlich <g-...@gmx.de>.
>-----Original Message-----
>From: Peter Royal [mailto:proyal@managingpartners.com]
>Sent: Wednesday, December 12, 2001 5:01 PM
>To: cocoon-dev@xml.apache.org
>Subject: Re: [VOTE] adding Jisp based Store persistence to the
>Scratchpad
>
>
>On Wednesday 12 December 2001 09:13 am, you wrote:
>> >What would be the benefits of using B-Tree indexed file vs filesystem
>> >directory?
>>
>> - When you have many files in one directory that can be problematic
>> for maintainence.
>> - Some Caching system like Squid split that up into the different
>> subdirectories, others hold the data in one file.
>> - With this Implementation we would be able to turn -if wished-
>> the MRUMemoryStore (or something equal) off and use only the Filesystem
>> for Storing. Maybe for machines with limited memory,.... For this approach
>> you need some speedy access for the data on the FS.
>> - Multiple indexes per data file, so you can access data with
>> a variety of criteria.
>> - Indexed data does not need to be homogenous; data files can contain
>> mixed serialized types, all using the same indexes.
>> - 100% portable between platforms.
>> - ...
>
>- No problems with filenames that exceed target platform limitations :)
>
>A deep cacheable pipeline can generate a very long FS store key.
>-pete
Yepp :)!
Cheers
Gerhard
------------------------------------------------------------------
Calvin: People think it must be fun to be a super genius, but they
don't realize how hard it is to put up with all the idiots in the
world.
Hobbes: Isn't your pants' zipper supposed to be in the front?
------------------------------------------------------------------
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org
Re: [VOTE] adding Jisp based Store persistence to the Scratchpad
Posted by Peter Royal <pr...@managingpartners.com>.
On Wednesday 12 December 2001 09:13 am, you wrote:
> >What would be the benefits of using B-Tree indexed file vs filesystem
> >directory?
>
> - When you have many files in one directory that can be problematic
> for maintainence.
> - Some Caching system like Squid split that up into the different
> subdirectories, others hold the data in one file.
> - With this Implementation we would be able to turn -if wished-
> the MRUMemoryStore (or something equal) off and use only the Filesystem
> for Storing. Maybe for machines with limited memory,.... For this approach
> you need some speedy access for the data on the FS.
> - Multiple indexes per data file, so you can access data with
> a variety of criteria.
> - Indexed data does not need to be homogenous; data files can contain
> mixed serialized types, all using the same indexes.
> - 100% portable between platforms.
> - ...
- No problems with filenames that exceed target platform limitations :)
A deep cacheable pipeline can generate a very long FS store key.
-pete
--
peter royal -> proyal@managingpartners.com
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org
RE: [VOTE] adding Jisp based Store persistence to the Scratchpad
Posted by Gerhard Froehlich <g-...@gmx.de>.
Adrian,
>From: Adrian Geissel [mailto:ageissel@zenark.com]
>
>----- Original Message -----
>From: Gerhard Froehlich <g-...@gmx.de>
>To: <co...@xml.apache.org>
>Sent: Wednesday, December 12, 2001 2:13 PM
>Subject: RE: [VOTE] adding Jisp based Store persistence to the Scratchpad
>
>
>> Vadim,
>> >From: Vadim Gritsenko [mailto:vadim.gritsenko@verizon.net]
>> >
>> >
>> >What would be the benefits of using B-Tree indexed file vs filesystem
>> >directory?
>>
>> - When you have many files in one directory that can be problematic
>> for maintainence.
>> - Some Caching system like Squid split that up into the different
>> subdirectories, others hold the data in one file.
>> - With this Implementation we would be able to turn -if wished-
>> the MRUMemoryStore (or something equal) off and use only the Filesystem
>> for Storing. Maybe for machines with limited memory,.... For this approach
>> you need some speedy access for the data on the FS.
>
>If you are going to persue this thought process, bear in mind that storing
>large numbers of files in a single directory has serious performance
>considerations, unless you are using a specialized file system - I'm
>thinking of ReiserFS (for Linux) which advertises exceptional performance
>both for large numbers of files, especially for files <4k in size.
Yes, you're correct. In the current implementation the Cocoon objects are
serialized down the fs as they are. Zig little files on the filesystem.
To avoid this my idea is (just an idea) to store this objects in one data
file and using indexes to access this file.
>Also, my experience with disk-based caching is that it is inherently slower
>than memory-based, and so should only be used in situations where the data
>either needs to be guaranteed failsafe (QOS-type requirements), or where it
>is known in advance that the data will not be required for some time to
>come. In practice, I expect that it is cheaper and possibly more elegant to
>upgrade the physical memory, especially at current market prices.
That's right. But it uses the same Store Interface, therefor it is a
elegant (elegant in fs context) alternative to a memory based storing.
It's up to you using it.
Default this new Jisp based Store will be used combined with the
MRUMemoryStore!
>Appologies for stepping on toes,
Didn't hurt ;-).
>Adrian
Gerhard
-----------------------------------------------
Consciousness: that annoying time between naps.
-----------------------------------------------
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org
Re: [VOTE] adding Jisp based Store persistence to the Scratchpad
Posted by Adrian Geissel <ag...@zenark.com>.
----- Original Message -----
From: Gerhard Froehlich <g-...@gmx.de>
To: <co...@xml.apache.org>
Sent: Wednesday, December 12, 2001 2:13 PM
Subject: RE: [VOTE] adding Jisp based Store persistence to the Scratchpad
> Vadim,
> >From: Vadim Gritsenko [mailto:vadim.gritsenko@verizon.net]
> >
> >
> >What would be the benefits of using B-Tree indexed file vs filesystem
> >directory?
>
> - When you have many files in one directory that can be problematic
> for maintainence.
> - Some Caching system like Squid split that up into the different
> subdirectories, others hold the data in one file.
> - With this Implementation we would be able to turn -if wished-
> the MRUMemoryStore (or something equal) off and use only the Filesystem
> for Storing. Maybe for machines with limited memory,.... For this approach
> you need some speedy access for the data on the FS.
If you are going to persue this thought process, bear in mind that storing
large numbers of files in a single directory has serious performance
considerations, unless you are using a specialized file system - I'm
thinking of ReiserFS (for Linux) which advertises exceptional performance
both for large numbers of files, especially for files <4k in size.
Also, my experience with disk-based caching is that it is inherently slower
than memory-based, and so should only be used in situations where the data
either needs to be guaranteed failsafe (QOS-type requirements), or where it
is known in advance that the data will not be required for some time to
come. In practice, I expect that it is cheaper and possibly more elegant to
upgrade the physical memory, especially at current market prices.
Appologies for stepping on toes,
Adrian
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org
RE: [VOTE] adding Jisp based Store persistence to the Scratchpad
Posted by Vadim Gritsenko <va...@verizon.net>.
> -----Original Message-----
> From: Gerhard Froehlich [mailto:g-froehlich@gmx.de]
> Sent: Wednesday, December 12, 2001 9:13 AM
> To: cocoon-dev@xml.apache.org
> Subject: RE: [VOTE] adding Jisp based Store persistence to the
Scratchpad
>
> Vadim,
> >From: Vadim Gritsenko [mailto:vadim.gritsenko@verizon.net]
> >
> >
> >What would be the benefits of using B-Tree indexed file vs filesystem
> >directory?
>
> - When you have many files in one directory that can be problematic
> for maintainence.
> - Some Caching system like Squid split that up into the different
> subdirectories, others hold the data in one file.
> - With this Implementation we would be able to turn -if wished-
> the MRUMemoryStore (or something equal) off and use only the
Filesystem
> for Storing. Maybe for machines with limited memory,.... For this
approach
> you need some speedy access for the data on the FS.
> - Multiple indexes per data file, so you can access data with
> a variety of criteria.
> - Indexed data does not need to be homogenous; data files can contain
> mixed serialized types, all using the same indexes.
> - 100% portable between platforms.
> - ...
>
> Just a try... Maybe it rules, maybe it fails ;)
Go ahead! :)
>
> >Anyway, you can put something into scratchpad without voting...
>
> Ahh I see...
>
> >Vadim
>
> Gerhard
>
> ---------------------------------
> Me, Ambivalent? Well, yes and no.
> ---------------------------------
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org
RE: [VOTE] adding Jisp based Store persistence to the Scratchpad
Posted by Gerhard Froehlich <g-...@gmx.de>.
Vadim,
>From: Vadim Gritsenko [mailto:vadim.gritsenko@verizon.net]
>
>
>What would be the benefits of using B-Tree indexed file vs filesystem
>directory?
- When you have many files in one directory that can be problematic
for maintainence.
- Some Caching system like Squid split that up into the different
subdirectories, others hold the data in one file.
- With this Implementation we would be able to turn -if wished-
the MRUMemoryStore (or something equal) off and use only the Filesystem
for Storing. Maybe for machines with limited memory,.... For this approach
you need some speedy access for the data on the FS.
- Multiple indexes per data file, so you can access data with
a variety of criteria.
- Indexed data does not need to be homogenous; data files can contain
mixed serialized types, all using the same indexes.
- 100% portable between platforms.
- ...
Just a try... Maybe it rules, maybe it fails ;)
>Anyway, you can put something into scratchpad without voting...
Ahh I see...
>Vadim
Gerhard
---------------------------------
Me, Ambivalent? Well, yes and no.
---------------------------------
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org
RE: [VOTE] adding Jisp based Store persistence to the Scratchpad
Posted by Vadim Gritsenko <va...@verizon.net>.
What would be the benefits of using B-Tree indexed file vs filesystem
directory?
Anyway, you can put something into scratchpad without voting...
Vadim
> -----Original Message-----
> From: Gerhard Froehlich [mailto:g-froehlich@gmx.de]
> Sent: Wednesday, December 12, 2001 5:56 AM
> To: cocoon-Dev
> Subject: [VOTE] adding Jisp based Store persistence to the Scratchpad
>
> Hi,
> as I said there's much work to do and
> as a fresh baked Committer I call
> a Vote, now:
>
> I want to improve the Store Persistence
> of the current Store implementation. There-
> fore I want to try to implement a
> JispFilesystemStore
> http://www.coyotegulch.com/jisp/index.html
>
> But whole thing must be evaluated and tried
> first. Therefore I want to put this new
> Implementation in the Scratchpad. So we can
> play around and don't bother the current
> main branch.
>
> Please give your votes.
>
> Gerhard
>
> ----------------------------
> I just found the last bug...
> ----------------------------
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org