Re: Caching in Maven core

[John Casey <jd...@commonjava.org>]

Just to be clear, the ProjectBuildCache is currently stored in a  
ThreadLocal, by way of the BuildContextManager.

The BuildContext stuff needs a little more love, definitely, but this  
cache should not be a memory leak now, unless I'm missing something...

-john

On Sep 24, 2007, at 2:42 AM, Milos Kleint wrote:

> On 9/24/07, Jason van Zyl <ja...@maven.org> wrote:
>>
>> On 23 Sep 07, at 8:54 PM 23 Sep 07, Carlos Sanchez wrote:
>>
>>> that's exactly what I wanted to know. So we then agree than this  
>>> must
>>> be cleared at the end of each execution. If that's the case then no
>>> caching is needed, but we would need a cache per execution  
>>> request to
>>> allow concurrent executions
>>>
>>
>> I have been trying to use the session for everything internally so we
>> could either put it in the session, or add a thread local to the
>> cache. I'm working on a thread local instantiation strategy for
>> plexus but I'm sure you can do either of those two things above and
>> it will work perfectly fine.
>
> ThreadLocal will solve problems with concurrent access for sure,
> however it will not handle subsequent requests on the same thread.
>
> That's currently my only usecase when the cache hits me. I usually
> create an embedder instance per project load/build request, however on
> IDE startup with many projects opened, it slows down the startup
> significantly.
>
>
> Milos
>
>>
>> The logging is the other problem. I'll probably do the thread local
>> thing in plexus for the logging.
>>
>>> On 9/23/07, Jason van Zyl <ja...@maven.org> wrote:
>>>>
>>>> On 23 Sep 07, at 8:28 PM 23 Sep 07, Carlos Sanchez wrote:
>>>>
>>>>> Hi, this is a thread to gather other people's opinions on the
>>>>> usage of
>>>>> caching inside core maven
>>>>>
>>>>> While using the embedder I've come across some caching issues,
>>>>> mainly
>>>>> MNG-3008, but also MNG-3170 and MNG-3013
>>>>>
>>>>> For instance in org.apache.maven.project.build.ProjectBuildCache
>>>>> projects are being cached but never removed (please correct me if
>>>>> this
>>>>> assumption is wrong), so we would need some features like
>>>>> - maximum size of the cache
>>>>> - eviction policy (FIFO)
>>>>> - programatically evict projects (eg. when the pom file changes)
>>>>> ...
>>>>>
>>>>
>>>> It was only intended for the session and should be flushed after  
>>>> the
>>>> session. We tried to remove all optimization so that we could focus
>>>> on correctness and tackle the refactoring of the project  
>>>> builder. If
>>>> it's not flushed at the end of the reactor run then that's all that
>>>> needs to be done.
>>>>
>>>>> These requirements mean to me that a "real" cache would need to be
>>>>> used instead of HashMaps, something like ehcache or Apache JCS  
>>>>> that
>>>>> already have all that implemented.
>>>>>
>>>>
>>>> -1
>>>>
>>>> That's way too much overkill, especially for the core in a CLI  
>>>> run. I
>>>> mean really what are the rules?
>>>>
>>>> 1) If it's a release it shouldn't change and you can hold on to it
>>>> the entire life of the embedder.
>>>> 2) If it's a snapshot hold on to it during the session and then  
>>>> flush
>>>> it so that you get a new one the next session. If you wanted to get
>>>> fancy and check timestamps, or md5s for snapshots without the local
>>>> reactor you probably could here, but the simple rule of tossing  
>>>> post-
>>>> session would probably work just fine.
>>>>
>>>> How much more complicated is it then that?
>>>>
>>>> We already discussed this when Joakim attempted this and it's  
>>>> just so
>>>> much overkill.
>>>>
>>>>> What do other people think?
>>>>>
>>>>> --
>>>>> I could give you my word as a Spaniard.
>>>>> No good. I've known too many Spaniards.
>>>>>                              -- The Princess Bride
>>>>>
>>>>> ------------------------------------------------------------------ 
>>>>> --
>>>>> -
>>>>> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
>>>>> For additional commands, e-mail: dev-help@maven.apache.org
>>>>>
>>>>
>>>> Thanks,
>>>>
>>>> Jason
>>>>
>>>> ----------------------------------------------------------
>>>> Jason van Zyl
>>>> Founder and PMC Chair, Apache Maven
>>>> jason at sonatype dot com
>>>> ----------------------------------------------------------
>>>>
>>>>
>>>>
>>>>
>>>> ------------------------------------------------------------------- 
>>>> --
>>>> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
>>>> For additional commands, e-mail: dev-help@maven.apache.org
>>>>
>>>>
>>>
>>>
>>> --
>>> I could give you my word as a Spaniard.
>>> No good. I've known too many Spaniards.
>>>                              -- The Princess Bride
>>>
>>> -------------------------------------------------------------------- 
>>> -
>>> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
>>> For additional commands, e-mail: dev-help@maven.apache.org
>>>
>>
>> Thanks,
>>
>> Jason
>>
>> ----------------------------------------------------------
>> Jason van Zyl
>> Founder and PMC Chair, Apache Maven
>> jason at sonatype dot com
>> ----------------------------------------------------------
>>
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
>> For additional commands, e-mail: dev-help@maven.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>

---
John Casey
Committer and PMC Member, Apache Maven
mail: jdcasey at commonjava dot org
blog: http://www.ejlife.net/blogs/john
rss: http://feeds.feedburner.com/ejlife/john