Re: [VOTE] Should we create sub-projects for HDFS and Map/Reduce?

[Nigel Daley <nd...@yahoo-inc.com>]

On Aug 15, 2008, at 10:03 PM, Nigel Daley wrote:

>> Another benefit is that it would increase the separation of these  
>> technologies, so that, e.g., folks could more easily run different  
>> versions of mapreduce on top of different versions of HDFS.   
>> Currently we make no such guarantees.  Folks would be able to  
>> upgrade to, e.g., the next release of mapreduce on a subset of  
>> their cluster without upgrading their HDFS.  That's not currently  
>> supported.  As we move towards splitting mapreduce into a scheduler  
>> and runtime, where folks can specify a different runtime per job,  
>> this will be even more critical.
>
> Sounds like we simply need to create separate jar files for these  
> different components.  This can be done in the current project.
>
> Wouldn't the amount of effort to make this split and get it right be  
> better spent on getting all components of Hadoop to 1.0 (API  
> stability)?  The proposal feels like a distraction to me at this  
> point in the project.
>
> Nige

I'd like to retract the -1 vote that I gave this proposal earlier.   
One compelling reason (for me) to split HDFS and Map/Reduce into  
separate sub-projects is that (hopefully) the *configs* for each layer  
will be clearer and simpler.

So I'm now +1 on this proposal.

Nige

Re: [VOTE] Should we create sub-projects for HDFS and Map/Reduce?

[Tom White <to...@gmail.com>]

BTW The initial work to sort out the module dependencies (but not
actually split the projects) is being carried out in
https://issues.apache.org/jira/browse/HADOOP-3750.

Tom

On Mon, Sep 15, 2008 at 11:09 PM, Owen O'Malley <om...@apache.org> wrote:
> Ok, so in very protracted voting, the results are:
>
> PMC +1's: Arun, Dhruba, Doug, Nigel, Owen, Tom
>
> So the vote passes.
>
> -- Owen
>

Re: [VOTE] Should we create sub-projects for HDFS and Map/Reduce?

[Owen O'Malley <om...@apache.org>]

Ok, so in very protracted voting, the results are:

PMC +1's: Arun, Dhruba, Doug, Nigel, Owen, Tom

So the vote passes.

-- Owen

Re: [VOTE] Should we create sub-projects for HDFS and Map/Reduce?

[Dhruba Borthakur <dh...@gmail.com>]

+1

I would prefer to keep hdfs and mapreduce together because I believe
that this arrangement catches incompatability sooner that later. But
with the coming of age of both these modules separately, I guess it is
time to grow them on their own individual turf!

thanks,
dhruba

On Fri, Sep 5, 2008 at 5:21 PM, Nigel Daley <nd...@yahoo-inc.com> wrote:
>
> On Aug 15, 2008, at 10:03 PM, Nigel Daley wrote:
>
>>> Another benefit is that it would increase the separation of these
>>> technologies, so that, e.g., folks could more easily run different versions
>>> of mapreduce on top of different versions of HDFS.  Currently we make no
>>> such guarantees.  Folks would be able to upgrade to, e.g., the next release
>>> of mapreduce on a subset of their cluster without upgrading their HDFS.
>>>  That's not currently supported.  As we move towards splitting mapreduce
>>> into a scheduler and runtime, where folks can specify a different runtime
>>> per job, this will be even more critical.
>>
>> Sounds like we simply need to create separate jar files for these
>> different components.  This can be done in the current project.
>>
>> Wouldn't the amount of effort to make this split and get it right be
>> better spent on getting all components of Hadoop to 1.0 (API stability)?
>>  The proposal feels like a distraction to me at this point in the project.
>>
>> Nige
>
> I'd like to retract the -1 vote that I gave this proposal earlier.  One
> compelling reason (for me) to split HDFS and Map/Reduce into separate
> sub-projects is that (hopefully) the *configs* for each layer will be
> clearer and simpler.
>
> So I'm now +1 on this proposal.
>
> Nige
>