Apache-hadoop => CDH3 conversion: permissions

["J. Ryan Earl" <os...@jryanearl.us>]

Hello,

I'm trying to nail down a process for converting existing Apache-hadoop
clusters with significant amounts of pre-existing data to CDH3.  While I've
found documentation for upgrading between CDH versions, I haven't seen one
for standard Apache-hadoop => CDH3 with the "new" mapred and hdfs users and
groups.  I'm looking for the proper method to manually convert permissions
on existing data from a single user&group setup (hadoop/hadoop) to the 2user
& 3 group setup of hdfs/mapred users with hdfs/mapred/hadoop groups.

What I'm thinking needs to happen is something like this:

   1. Shutdown cluster.
   2. Perform full configuration and HDFS data backup.
   3. Delete existing hadoop user/group while leaving HDFS data/mapred
   folders untouched.
   4. Install CDH3 packages (which sets up new users and groups).
   5. Manually adjust permissions/groups/ownership of old data files to
   match new CDH3 security setup.
   6. Flow into standard hadoop-upgrade process.

 Basically, I'm trying to nail down setup 5, and would appreciate any
guidance on this.  I'm guessing I just haven't found the correct document
since this seems like it would be a common endeavor.

Thanks in advance,
-JR

Re: Apache-hadoop => CDH3 conversion: permissions

[Konstantin Boudnik <co...@apache.org>]

I'm pretty sure you'll find more support on a cdh specific mailing lists.
Apparently, such a conversion won't be covered by Apache Hadoop documentation.

Cos

On Fri, Jun 17, 2011 at 04:09PM, J. Ryan Earl wrote:
>    Hello,
>    I'm trying to nail down a process for converting existing Apache-hadoop
>    clusters with significant amounts of pre-existing data to CDH3.  While
>    I've found documentation for upgrading between CDH versions, I haven't
>    seen one for standard Apache-hadoop => CDH3 with the "new" mapred and hdfs
>    users and groups.  I'm looking for the proper method to manually convert
>    permissions on existing data from a single user&group setup
>    (hadoop/hadoop) to the 2user & 3 group setup of hdfs/mapred users with
>    hdfs/mapred/hadoop groups.
>    What I'm thinking needs to happen is something like this:
>     1. Shutdown cluster.
>     2. Perform full configuration and HDFS data backup.
>     3. Delete existing hadoop user/group while leaving HDFS data/mapred
>        folders untouched.
>     4. Install CDH3 packages (which sets up new users and groups).
>     5. Manually adjust permissions/groups/ownership of old data files to
>        match new CDH3 security setup.
>     6. Flow into standard hadoop-upgrade process.
>     Basically, I'm trying to nail down setup 5, and would appreciate any
>    guidance on this.  I'm guessing I just haven't found the correct document
>    since this seems like it would be a common endeavor.
>    Thanks in advance,
>    -JR