Re: Geronimo Repository (was Re: Unable to build using m2)

["Alan D. Cabrera" <li...@toolazydogs.com>]

Jason, who are you replying to?  You accidentally cut out the person's name.


Regards,
Alan

Jason Dillon wrote:
>> I agree with your comments but I also recognize that it is a slippery 
>> slope.  Who is responsible for tracking the changes and lobbying the 
>> other groups?  My limited experience tells me that this will likely 
>> end up being a set of forked code from other projects that will be 
>> very difficult to manage.  for instance...say a quick change is made 
>> to XMLBeans and it get's put into our "local" copy.  4 months later 
>> the XMLBeans folks have not done anything with the patch and now 
>> after discussing it further decide that it doens't fit with their 
>> direction.  During that time Geronimo has relied on the patch and 
>> changing direction is going to be painful.
>
> I would expect that most of the localized changes would be to pom's 
> only and to install the correct version of the dependency in a 
> repository for our build to work.
>
> While I think that we may need to have a few cases where some code is 
> required to change, but in order for that to work, there must be 
> someone who is actively working with the external community to get the 
> changes in.  Otherwise, as you suggested it would turn into a private 
> fork... and IMO that is not acceptable.
>
> I believe that it would be the responsibility of the developer who 
> introduces the new version of the dependency to create a JIRA (in our 
> project) to manage the state of the dependency... AND be active about 
> getting the changes introduced in the target communities codebase.
>
> Peer review will also be used to help ensure that we keep things 
> moving smoothly.  Like, if a custom version of a dependency exists for 
> months and months and goes no where, then we need to react and either 
> help the developer get the changes moved or reevaluate the need for 
> those changes, etc.
>
>
>> I realize this is a worse case scenario but we had a similar 
>> situation with the Axis guys in cutting a 1.4 release for us.  It 
>> took about 6 months and I think was only done because David Blevins 
>> was made a committer.
>
> Um, did it take 6 months because he was a committer?  Meaning it would 
> have taken more than 6 months if he was not?  Yikes!
>
>
>> In order to consider this it would be great if you could flush out 
>> the different scenarios as you see them and how we would handle 
>> them.  I'm not the most administrative person on the planet and what 
>> you are proposing sounds great and scare the heck out of me at the 
>> same time ;-0
>
> Well...  I'm not sure I know of all of the scenarios, but mostly I do 
> not see that we will have custom forked versions of others work in our 
> repository, though on some occasions yes, we will... but only for a 
> very limited and controlled period.
>
> For example, I have been using the RetroTranslator m2 plugin in GShell 
> for many weeks now.  I have been very active about creating patches 
> for the Mojo team, getting changes needed into this plugin so that it 
> can be more useful.  But, it takes the Mojo team time (some times a 
> long time) to pull in those changes... or as is the current case, to 
> publish the plugin with those changes already accepted and committed.
>
> So we have already gotten the code in the target communities 
> codebase... but it is taking a long time to get a new release (or 
> SNAPSHOT) pushed out to a repo so that we can actually use it.
>
> This is where our repository comes in.  We publish a SNAPSHOT of the 
> plugin to our repository so that we can continue to move forward using 
> the plugin, and in the background keep on the Mojo peeps to get the 
> thing official.
>
>  * * *
>
> There is also the other-side of the repository, which is to allow 
> builds to be reproducible in future months, years.  To have 100% build 
> reproducibility we need to have much more control over the repository 
> which dependencies are pulled from.
>
> The reliance on so many external repositories is a build anti-pattern 
> and make is very difficult (some times impossible) to ensure that 
> builds will just work in future days, months, years.  Machines crash, 
> data gets lost, which we have recently seen and suffered from.  Having 
> a repository that is controlled by the same system as our sources will 
> generally mean that at any point in time when when have a valid scm 
> codebase, then we also have the valid dependencies.  Then when the 
> Codehaus disks die again, or a worm eats thought all of Ibiblio, then 
> we can still function and continue to work.
>
>  * * *
>
> I believe that both of these reasons (staging w/external groups and 
> build reproduction and isolation from external failure) warrant the 
> creation of an internal SVN-backed repository that exists solely for 
> the use by Geronimo, and its child projects.
>
> --jason

Re: Geronimo Repository (was Re: Unable to build using m2)

[Jason Dillon <ja...@planet57.com>]

Ooops... I got trim happy.  This was a reply to Matt.

--jason


On Jul 1, 2006, at 2:05 PM, Alan D. Cabrera wrote:

> Jason, who are you replying to?  You accidentally cut out the  
> person's name.
>
>
> Regards,
> Alan
>
> Jason Dillon wrote:
>>> I agree with your comments but I also recognize that it is a  
>>> slippery slope.  Who is responsible for tracking the changes and  
>>> lobbying the other groups?  My limited experience tells me that  
>>> this will likely end up being a set of forked code from other  
>>> projects that will be very difficult to manage.  for  
>>> instance...say a quick change is made to XMLBeans and it get's  
>>> put into our "local" copy.  4 months later the XMLBeans folks  
>>> have not done anything with the patch and now after discussing it  
>>> further decide that it doens't fit with their direction.  During  
>>> that time Geronimo has relied on the patch and changing direction  
>>> is going to be painful.
>>
>> I would expect that most of the localized changes would be to  
>> pom's only and to install the correct version of the dependency in  
>> a repository for our build to work.
>>
>> While I think that we may need to have a few cases where some code  
>> is required to change, but in order for that to work, there must  
>> be someone who is actively working with the external community to  
>> get the changes in.  Otherwise, as you suggested it would turn  
>> into a private fork... and IMO that is not acceptable.
>>
>> I believe that it would be the responsibility of the developer who  
>> introduces the new version of the dependency to create a JIRA (in  
>> our project) to manage the state of the dependency... AND be  
>> active about getting the changes introduced in the target  
>> communities codebase.
>>
>> Peer review will also be used to help ensure that we keep things  
>> moving smoothly.  Like, if a custom version of a dependency exists  
>> for months and months and goes no where, then we need to react and  
>> either help the developer get the changes moved or reevaluate the  
>> need for those changes, etc.
>>
>>
>>> I realize this is a worse case scenario but we had a similar  
>>> situation with the Axis guys in cutting a 1.4 release for us.  It  
>>> took about 6 months and I think was only done because David  
>>> Blevins was made a committer.
>>
>> Um, did it take 6 months because he was a committer?  Meaning it  
>> would have taken more than 6 months if he was not?  Yikes!
>>
>>
>>> In order to consider this it would be great if you could flush  
>>> out the different scenarios as you see them and how we would  
>>> handle them.  I'm not the most administrative person on the  
>>> planet and what you are proposing sounds great and scare the heck  
>>> out of me at the same time ;-0
>>
>> Well...  I'm not sure I know of all of the scenarios, but mostly I  
>> do not see that we will have custom forked versions of others work  
>> in our repository, though on some occasions yes, we will... but  
>> only for a very limited and controlled period.
>>
>> For example, I have been using the RetroTranslator m2 plugin in  
>> GShell for many weeks now.  I have been very active about creating  
>> patches for the Mojo team, getting changes needed into this plugin  
>> so that it can be more useful.  But, it takes the Mojo team time  
>> (some times a long time) to pull in those changes... or as is the  
>> current case, to publish the plugin with those changes already  
>> accepted and committed.
>>
>> So we have already gotten the code in the target communities  
>> codebase... but it is taking a long time to get a new release (or  
>> SNAPSHOT) pushed out to a repo so that we can actually use it.
>>
>> This is where our repository comes in.  We publish a SNAPSHOT of  
>> the plugin to our repository so that we can continue to move  
>> forward using the plugin, and in the background keep on the Mojo  
>> peeps to get the thing official.
>>
>>  * * *
>>
>> There is also the other-side of the repository, which is to allow  
>> builds to be reproducible in future months, years.  To have 100%  
>> build reproducibility we need to have much more control over the  
>> repository which dependencies are pulled from.
>>
>> The reliance on so many external repositories is a build anti- 
>> pattern and make is very difficult (some times impossible) to  
>> ensure that builds will just work in future days, months, years.   
>> Machines crash, data gets lost, which we have recently seen and  
>> suffered from.  Having a repository that is controlled by the same  
>> system as our sources will generally mean that at any point in  
>> time when when have a valid scm codebase, then we also have the  
>> valid dependencies.  Then when the Codehaus disks die again, or a  
>> worm eats thought all of Ibiblio, then we can still function and  
>> continue to work.
>>
>>  * * *
>>
>> I believe that both of these reasons (staging w/external groups  
>> and build reproduction and isolation from external failure)  
>> warrant the creation of an internal SVN-backed repository that  
>> exists solely for the use by Geronimo, and its child projects.
>>
>> --jason
>