Locks in hadoop

[Sagar Naik <sn...@attributor.com>]

I would like to implement a  locking mechanism across the hdfs cluster
I assume there is no inherent support for it

I was going to do it with files. According to my knowledge, file 
creation is an atomic operation. So the file-based lock should work.
I need to think through with all conditions but if some one has better 
idea/solution, pl share

Thanks
-Sagar

Re: Locks in hadoop

[Konstantin Shvachko <sh...@yahoo-inc.com>]

Did you look at Zookeeper?
Thanks,
--Konstantin

Sagar Naik wrote:
> I would like to implement a  locking mechanism across the hdfs cluster
> I assume there is no inherent support for it
> 
> I was going to do it with files. According to my knowledge, file 
> creation is an atomic operation. So the file-based lock should work.
> I need to think through with all conditions but if some one has better 
> idea/solution, pl share
> 
> Thanks
> -Sagar
> 
> 

RE: Locks in hadoop

[Koji Noguchi <kn...@yahoo-inc.com>]

> I was going to do it with files. According to my knowledge, 
> file creation is an atomic operation.
>
You might want to look at hdfs balancer.
It uses hdfs file to make sure only one instance of balancer is up.

Koji


-----Original Message-----
From: Sagar Naik [mailto:snaik@attributor.com] 
Sent: Thursday, January 15, 2009 5:44 PM
To: core-user@hadoop.apache.org
Subject: Locks in hadoop

I would like to implement a  locking mechanism across the hdfs cluster
I assume there is no inherent support for it

I was going to do it with files. According to my knowledge, file 
creation is an atomic operation. So the file-based lock should work.
I need to think through with all conditions but if some one has better 
idea/solution, pl share

Thanks
-Sagar