You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by Blair Zajac <bl...@orcaware.com> on 2008/03/07 23:48:09 UTC
Re: svn commit: r29773 - branches/in-memory-cache
glasser@tigris.org wrote:
> Author: glasser
> Date: Fri Mar 7 10:23:54 2008
> New Revision: 29773
>
> Log:
> On the in-memory-cache branch:
>
> * README.branch
> Add a README, including a basic outline of the API I'm going to add
> and the implementation it's going to use.
>
>
> Added:
> branches/in-memory-cache/README.branch
>
> Added: branches/in-memory-cache/README.branch
> URL: http://svn.collab.net/viewvc/svn/branches/in-memory-cache/README.branch?pathrev=29773
> ==============================================================================
> --- (empty file)
> +++ branches/in-memory-cache/README.branch Fri Mar 7 10:23:54 2008
> @@ -0,0 +1,102 @@
> +This branch adds a simple in-memory cache library that handles
> +annoying memory-management details for you. FSFS has several
> +hand-written caches already; this will reduce duplicate code and make
> +it easier to add more.
> +
> +NULL is a legitimate value. There is no cache_delete (but you can set
> +to NULL, which may be good enough).
It seems that any cache would want a cache_delete method. I've written a server
that exposes svn via Ice and it would be handy to have a cache.
What about adding some fields in the structures to make an LRU cache possible
such as adding to the cache_entry the last time it was looked up. I guess one
could also wrap the value in another value that has those fields, so leaving
this alone would be good.
But I think you'll always want a way of removing entries.
Blair
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn commit: r29773 - branches/in-memory-cache
Posted by David Glasser <gl...@davidglasser.net>.
On Tue, Mar 11, 2008 at 5:14 PM, Daniel L. Rall <dl...@finemaltcoding.com> wrote:
> On Fri, 07 Mar 2008, David Glasser wrote:
>
> > On Fri, Mar 7, 2008 at 4:22 PM, Blair Zajac <bl...@orcaware.com> wrote:
> ...
>
> > > What about thread safety? Are you going to address that?
> >
> > Good point. Options:
> >
> > * Not thread safe at all. Caller must do mutexes.
> >
> > * Always thread safe (if APR_HAS_THREADS).
> >
> > * Boolean to constructor.
>
> Would all current users go one way with that? If so, which?
I implemented it as "boolean to constructor". There are things in the
FSFS code that are shared between all threads accessing a FS, and
things that are single-threaded.
--dave
--
David Glasser | glasser@davidglasser.net | http://www.davidglasser.net/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn commit: r29773 - branches/in-memory-cache
Posted by "Daniel L. Rall" <dl...@finemaltcoding.com>.
On Fri, 07 Mar 2008, David Glasser wrote:
> On Fri, Mar 7, 2008 at 4:22 PM, Blair Zajac <bl...@orcaware.com> wrote:
...
> > What about thread safety? Are you going to address that?
>
> Good point. Options:
>
> * Not thread safe at all. Caller must do mutexes.
>
> * Always thread safe (if APR_HAS_THREADS).
>
> * Boolean to constructor.
Would all current users go one way with that? If so, which?
Re: svn commit: r29773 - branches/in-memory-cache
Posted by David Glasser <gl...@davidglasser.net>.
On Fri, Mar 7, 2008 at 4:22 PM, Blair Zajac <bl...@orcaware.com> wrote:
>
> David Glasser wrote:
> > On Fri, Mar 7, 2008 at 3:48 PM, Blair Zajac <bl...@orcaware.com> wrote:
> >> glasser@tigris.org wrote:
> >> > Author: glasser
> >> > Date: Fri Mar 7 10:23:54 2008
> >> > New Revision: 29773
> >> >
> >> > Log:
> >> > On the in-memory-cache branch:
> >> >
> >> > * README.branch
> >> > Add a README, including a basic outline of the API I'm going to add
> >> > and the implementation it's going to use.
> >> >
> >> >
> >> > Added:
> >> > branches/in-memory-cache/README.branch
> >> >
> >> > Added: branches/in-memory-cache/README.branch
> >> > URL: http://svn.collab.net/viewvc/svn/branches/in-memory-cache/README.branch?pathrev=29773
> >> > ==============================================================================
> >> > --- (empty file)
> >> > +++ branches/in-memory-cache/README.branch Fri Mar 7 10:23:54 2008
> >> > @@ -0,0 +1,102 @@
> >> > +This branch adds a simple in-memory cache library that handles
> >> > +annoying memory-management details for you. FSFS has several
> >> > +hand-written caches already; this will reduce duplicate code and make
> >> > +it easier to add more.
> >> > +
> >> > +NULL is a legitimate value. There is no cache_delete (but you can set
> >> > +to NULL, which may be good enough).
> >>
> >> It seems that any cache would want a cache_delete method. I've written a server
> >> that exposes svn via Ice and it would be handy to have a cache.
> >
> > When we need it, sure. FSFS does have one bit of cache invalidation.
> >
> > However, the whole reason that this data structure is needed at all is
> > that with APR pools, "deleting" (freeing memory) is non-trivial.
> >
> >> What about adding some fields in the structures to make an LRU cache possible
> >> such as adding to the cache_entry the last time it was looked up. I guess one
> >> could also wrap the value in another value that has those fields, so leaving
> >> this alone would be good.
> >
> > The plan as designed is already LRU (though at the "page" level of
> > granularity). If items_per_page=1 then this is a normal LRU cache,
> > but then you have the overhead of one pool (at least 8K) per item...
> >
> >> But I think you'll always want a way of removing entries.
> >
> > Not for some of the actual applications I am planning this for. There
> > are so many low-hanging trees of immutable but uncached data in
> > FSFS...
> >
> > --dave
>
> OK. Great, that takes care of my concerns.
>
> What about thread safety? Are you going to address that?
Good point. Options:
* Not thread safe at all. Caller must do mutexes.
* Always thread safe (if APR_HAS_THREADS).
* Boolean to constructor.
--dave
--
David Glasser | glasser@davidglasser.net | http://www.davidglasser.net/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn commit: r29773 - branches/in-memory-cache
Posted by Blair Zajac <bl...@orcaware.com>.
David Glasser wrote:
> On Fri, Mar 7, 2008 at 3:48 PM, Blair Zajac <bl...@orcaware.com> wrote:
>> glasser@tigris.org wrote:
>> > Author: glasser
>> > Date: Fri Mar 7 10:23:54 2008
>> > New Revision: 29773
>> >
>> > Log:
>> > On the in-memory-cache branch:
>> >
>> > * README.branch
>> > Add a README, including a basic outline of the API I'm going to add
>> > and the implementation it's going to use.
>> >
>> >
>> > Added:
>> > branches/in-memory-cache/README.branch
>> >
>> > Added: branches/in-memory-cache/README.branch
>> > URL: http://svn.collab.net/viewvc/svn/branches/in-memory-cache/README.branch?pathrev=29773
>> > ==============================================================================
>> > --- (empty file)
>> > +++ branches/in-memory-cache/README.branch Fri Mar 7 10:23:54 2008
>> > @@ -0,0 +1,102 @@
>> > +This branch adds a simple in-memory cache library that handles
>> > +annoying memory-management details for you. FSFS has several
>> > +hand-written caches already; this will reduce duplicate code and make
>> > +it easier to add more.
>> > +
>> > +NULL is a legitimate value. There is no cache_delete (but you can set
>> > +to NULL, which may be good enough).
>>
>> It seems that any cache would want a cache_delete method. I've written a server
>> that exposes svn via Ice and it would be handy to have a cache.
>
> When we need it, sure. FSFS does have one bit of cache invalidation.
>
> However, the whole reason that this data structure is needed at all is
> that with APR pools, "deleting" (freeing memory) is non-trivial.
>
>> What about adding some fields in the structures to make an LRU cache possible
>> such as adding to the cache_entry the last time it was looked up. I guess one
>> could also wrap the value in another value that has those fields, so leaving
>> this alone would be good.
>
> The plan as designed is already LRU (though at the "page" level of
> granularity). If items_per_page=1 then this is a normal LRU cache,
> but then you have the overhead of one pool (at least 8K) per item...
>
>> But I think you'll always want a way of removing entries.
>
> Not for some of the actual applications I am planning this for. There
> are so many low-hanging trees of immutable but uncached data in
> FSFS...
>
> --dave
OK. Great, that takes care of my concerns.
What about thread safety? Are you going to address that?
Blair
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn commit: r29773 - branches/in-memory-cache
Posted by David Glasser <gl...@davidglasser.net>.
On Fri, Mar 7, 2008 at 3:48 PM, Blair Zajac <bl...@orcaware.com> wrote:
> glasser@tigris.org wrote:
> > Author: glasser
> > Date: Fri Mar 7 10:23:54 2008
> > New Revision: 29773
> >
> > Log:
> > On the in-memory-cache branch:
> >
> > * README.branch
> > Add a README, including a basic outline of the API I'm going to add
> > and the implementation it's going to use.
> >
> >
> > Added:
> > branches/in-memory-cache/README.branch
> >
> > Added: branches/in-memory-cache/README.branch
> > URL: http://svn.collab.net/viewvc/svn/branches/in-memory-cache/README.branch?pathrev=29773
> > ==============================================================================
> > --- (empty file)
> > +++ branches/in-memory-cache/README.branch Fri Mar 7 10:23:54 2008
> > @@ -0,0 +1,102 @@
> > +This branch adds a simple in-memory cache library that handles
> > +annoying memory-management details for you. FSFS has several
> > +hand-written caches already; this will reduce duplicate code and make
> > +it easier to add more.
> > +
> > +NULL is a legitimate value. There is no cache_delete (but you can set
> > +to NULL, which may be good enough).
>
> It seems that any cache would want a cache_delete method. I've written a server
> that exposes svn via Ice and it would be handy to have a cache.
When we need it, sure. FSFS does have one bit of cache invalidation.
However, the whole reason that this data structure is needed at all is
that with APR pools, "deleting" (freeing memory) is non-trivial.
> What about adding some fields in the structures to make an LRU cache possible
> such as adding to the cache_entry the last time it was looked up. I guess one
> could also wrap the value in another value that has those fields, so leaving
> this alone would be good.
The plan as designed is already LRU (though at the "page" level of
granularity). If items_per_page=1 then this is a normal LRU cache,
but then you have the overhead of one pool (at least 8K) per item...
> But I think you'll always want a way of removing entries.
Not for some of the actual applications I am planning this for. There
are so many low-hanging trees of immutable but uncached data in
FSFS...
--dave
--
David Glasser | glasser@davidglasser.net | http://www.davidglasser.net/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org