You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Brock Palen <br...@mlds-networks.com> on 2009/06/17 17:35:21 UTC
hadoop interaction with AFS
Ran into an issue with running hadoop on a cluster that also has AFS
installed. When a user ssh's in they get an 'extra' group id, I
think it is called a 'pag',
Problem is when you try to start hadoop from a shell that has a pag,
one of the checks in newer versions stops because there is no group
name to go with the gid,
as an example from the 'groups' command:
[brockp@nyx-login1 ~]$ groups
cacstaff vasp stata charmm amber8 siesta eecs587f06 nolimit gti
hyades adina chemkinpro coe molpro aces2 helios mcnp5 matlab id:
cannot find name for group ID 1093742985
1093742985
If you ssh to the machine its self using ssh keys, the pag is not
created, which our current work around, but is kinda kludgy,
If you need more information let me know.
Brock Palen
brockp@mlds-networks.com
www.mlds-networks.com
MLDS Owner Senior Tech.
Re: hadoop interaction with AFS
Posted by Faisal Khan <fa...@gmail.com>.
If re-building of Hadoop is an option then I guess you can replace 'groups'
command in Shell.java (src/core/org/apache/hadoop/util/Shell.java) with 'id
-rgn' to get the correct group name.
--
Faisal
On Wed, Jun 17, 2009 at 10:35 AM, Brock Palen <br...@mlds-networks.com>wrote:
> Ran into an issue with running hadoop on a cluster that also has AFS
> installed. When a user ssh's in they get an 'extra' group id, I think it
> is called a 'pag',
>
> Problem is when you try to start hadoop from a shell that has a pag, one of
> the checks in newer versions stops because there is no group name to go with
> the gid,
>
> as an example from the 'groups' command:
>
> [brockp@nyx-login1 ~]$ groups
> cacstaff vasp stata charmm amber8 siesta eecs587f06 nolimit gti hyades
> adina chemkinpro coe molpro aces2 helios mcnp5 matlab id: cannot find name
> for group ID 1093742985
> 1093742985
>
>
> If you ssh to the machine its self using ssh keys, the pag is not created,
> which our current work around, but is kinda kludgy,
>
> If you need more information let me know.
>
>
> Brock Palen
> brockp@mlds-networks.com
> www.mlds-networks.com
> MLDS Owner Senior Tech.
>
>
>
Re: hadoop interaction with AFS
Posted by Brian Bockelman <bb...@cse.unl.edu>.
Hey Brock,
Looking through my notes, OpenAFS 1.4.1 through 1.4.8 have an issue
with the GIDs. Maybe you can upgrade to the latest maintenance
release for the 1.4.x series and get lucky?
Otherwise, I think I might have a patch somewhere for this.
Brian
On Jun 17, 2009, at 8:41 AM, Brock Palen wrote:
>
>
>> Hey Brock,
>>
>> I've seen a similar problem at another site. They were able to
>> solve this by upgrading their version of OpenAFS. Is that an
>> option for you?
>
> It might be, I see we are running 1.4.8 and they have 1.5 out, not
> sure what our cell is running and compatibility, good to know.
>
>>
>> Brian
>>
>> On Jun 17, 2009, at 8:35 AM, Brock Palen wrote:
>>
>>> Ran into an issue with running hadoop on a cluster that also has
>>> AFS installed. When a user ssh's in they get an 'extra' group
>>> id, I think it is called a 'pag',
>>>
>>> Problem is when you try to start hadoop from a shell that has a
>>> pag, one of the checks in newer versions stops because there is no
>>> group name to go with the gid,
>>>
>>> as an example from the 'groups' command:
>>>
>>> [brockp@nyx-login1 ~]$ groups
>>> cacstaff vasp stata charmm amber8 siesta eecs587f06 nolimit gti
>>> hyades adina chemkinpro coe molpro aces2 helios mcnp5 matlab id:
>>> cannot find name for group ID 1093742985
>>> 1093742985
>>>
>>>
>>> If you ssh to the machine its self using ssh keys, the pag is not
>>> created, which our current work around, but is kinda kludgy,
>>>
>>> If you need more information let me know.
>>>
>>>
>>> Brock Palen
>>> brockp@mlds-networks.com
>>> www.mlds-networks.com
>>> MLDS Owner Senior Tech.
>>>
>>
Re: hadoop interaction with AFS
Posted by Brock Palen <br...@mlds-networks.com>.
> Hey Brock,
>
> I've seen a similar problem at another site. They were able to
> solve this by upgrading their version of OpenAFS. Is that an
> option for you?
It might be, I see we are running 1.4.8 and they have 1.5 out, not
sure what our cell is running and compatibility, good to know.
>
> Brian
>
> On Jun 17, 2009, at 8:35 AM, Brock Palen wrote:
>
>> Ran into an issue with running hadoop on a cluster that also has
>> AFS installed. When a user ssh's in they get an 'extra' group
>> id, I think it is called a 'pag',
>>
>> Problem is when you try to start hadoop from a shell that has a
>> pag, one of the checks in newer versions stops because there is no
>> group name to go with the gid,
>>
>> as an example from the 'groups' command:
>>
>> [brockp@nyx-login1 ~]$ groups
>> cacstaff vasp stata charmm amber8 siesta eecs587f06 nolimit gti
>> hyades adina chemkinpro coe molpro aces2 helios mcnp5 matlab id:
>> cannot find name for group ID 1093742985
>> 1093742985
>>
>>
>> If you ssh to the machine its self using ssh keys, the pag is not
>> created, which our current work around, but is kinda kludgy,
>>
>> If you need more information let me know.
>>
>>
>> Brock Palen
>> brockp@mlds-networks.com
>> www.mlds-networks.com
>> MLDS Owner Senior Tech.
>>
>
Re: hadoop interaction with AFS
Posted by Brian Bockelman <bb...@cse.unl.edu>.
Hey Brock,
I've seen a similar problem at another site. They were able to solve
this by upgrading their version of OpenAFS. Is that an option for you?
Brian
On Jun 17, 2009, at 8:35 AM, Brock Palen wrote:
> Ran into an issue with running hadoop on a cluster that also has AFS
> installed. When a user ssh's in they get an 'extra' group id, I
> think it is called a 'pag',
>
> Problem is when you try to start hadoop from a shell that has a pag,
> one of the checks in newer versions stops because there is no group
> name to go with the gid,
>
> as an example from the 'groups' command:
>
> [brockp@nyx-login1 ~]$ groups
> cacstaff vasp stata charmm amber8 siesta eecs587f06 nolimit gti
> hyades adina chemkinpro coe molpro aces2 helios mcnp5 matlab id:
> cannot find name for group ID 1093742985
> 1093742985
>
>
> If you ssh to the machine its self using ssh keys, the pag is not
> created, which our current work around, but is kinda kludgy,
>
> If you need more information let me know.
>
>
> Brock Palen
> brockp@mlds-networks.com
> www.mlds-networks.com
> MLDS Owner Senior Tech.
>