You are viewing a plain text version of this content. The canonical link for it is here.
Posted to general@hadoop.apache.org by Christophe Bisciglia <ch...@cloudera.com> on 2009/09/15 23:40:59 UTC
Hadoop World: NYC - Speakers and Schedule Up - New: Lightning Talks
at Welcome Reception!
Hadoop Fans, we're getting down to the wire for Hadoop World. We
couldn't be happier with how the schedule has come together.
For a full list of speakers and registration details, see
http://www.cloudera.com/hadoop-world-nyc - and if you plan to attend,
register this week, as prices go up $100 after September 21st.
We received nearly 50 submissions, and it was hard to trim this down,
even after creating a 3rd track. To provide more speaking
opportunities and time for the community to share their personal
experiences, Cloudera engineering is hosting a welcome reception with
lightning talks, an open bar, and live entertainment between talks.
There will be no sales or marketing from Cloudera or anyone else, and
talks are limited to 5 minutes to keep things lively. You'll find the
welcome reception in the Roosevelt Grill on the main floor, just off
the hotel lobby.
We'll get started at 6PM on October 1st. We'll also let you
pre-register for the conference so you can skip the line in the
morning.
At the main event, we're particularly excited to hear from Yahoo!
about upcoming API enhancements and security work. The whole team over
there is doing great stuff in this department, and it's features like
this that will help pave the way for serious enterprise adoption of
Hadoop. Facebook is also going to do a deep dive into HDFS
improvements and Hive - other key features for delivering a scalable
platform for a wide variety of data intensive applications.
You'll also hear from leading techologists at traditional enterprises
using Hadoop to solve real business problems today. Of particular note
are VISA on Large Scale Transaction Analysis, IBM on Ad-hoc Web Scale
Analytics, JP Morgan Chase on applications in Financial Services, Booz
Allen on Protein Alignment and Amazon Web Services on Bioinformatics.
These are in addition to all the usual suspects crunching big data in
the web space, and demonstrate how much closer we are to the beginning
of the Hadoop upswing than a plateau. These are exciting times, and we
hope you can all join us.
Cheers,
Christophe and the Cloudera Team
--
get hadoop: cloudera.com/hadoop
online training: cloudera.com/hadoop-training
blog: cloudera.com/blog
twitter: twitter.com/cloudera
Re: ACL question w/ Zookeeper 3.1.1
Posted by Benjamin Reed <br...@yahoo-inc.com>.
what error do you get?
ben
Todd Greenwood wrote:
> I'm attempting to secure a zookeeper installation using zookeeper ACLs.
> However, I'm finding that while Ids.OPEN_ACL_UNSAFE works great, my
> attempts at using Ids.CREATOR_ALL_ACL are failing. Here's a code
> snippet:
>
>
> public class ZooWrapper
> {
>
> /*
> 1. Here I'm setting up my authentication. I've got an ACL list, and my
> authentication strings.
> */
> private final List<ACL> acl = new ArrayList<ACL>( 1 );
> private static final String authentication_type = "digest";
> private static final String authentication =
> "audiencescience:gravy";
>
>
> public ZooWrapper( final String connection_string,
> final String path,
> final int connectiontimeout ) throws
> ZooWrapperException
> {
> ...
> /*
> 2. Here I'm adding the acls
> */
>
> // This works (creates nodes, sets data on nodes)
> for ( ACL ids_acl : Ids.OPEN_ACL_UNSAFE )
> {
> acl.add( ids_acl);
> }
>
> /*
> NOTE: This does not work (nodes are not created, cannot set data on
> nodes b/c nodes do not exist)
> */
>
> // for ( ACL ids_acl : Ids.CREATOR_ALL_ACL )
> // {
> // acl.add( ids_acl );
> // }
>
> /*
> 3. Finally, I create a new zookeeper instance and add my authorization
> info to it.
> */
> zoo = new ZooKeeper( connection_string, connectiontimeout, this );
> zoo.addAuthInfo( authentication_type, authentication.getBytes() )
>
> /*
> 4. Later, I try to write some data into zookeeper by first creating the
> node, and then calling setdata...
> */
> zoo.create( path, new byte[0], acl, CreateMode.PERSISTENT );
>
> zoo.setData( path, bytes, -1 )
>
> As I mentioned above, when I add Ids.OPEN_ACL_UNSAFE to acl, then both
> the create and setData succeed. However, when I use Ids.CREATOR_ALL_ACL,
> then the nodes are not created. Am I missing something obvious w/
> respect to configuring ACLs?
>
> I've used the following references:
>
> http://hadoop.apache.org/zookeeper/docs/r3.1.1/zookeeperProgrammers.html
>
> http://mail-archives.apache.org/mod_mbox/hadoop-zookeeper-commits/200807
> .mbox/%3C20080731201025.C62092388873@eris.apache.org%3E
>
> http://books.google.com/books?id=bKPEwR-Pt6EC&pg=PT404&lpg=PT404&dq=zook
> eeper+ACL+digest+%22new+Id%22&source=bl&ots=kObz0y8eFk&sig=VFCAsNW0mBJyZ
> swoweJDI31iNlo&hl=en&ei=Z82ySojRFsqRlAeqxsyIDw&sa=X&oi=book_result&ct=re
> sult&resnum=6#v=onepage&q=zookeeper%20ACL%20digest%20%22new%20Id%22&f=fa
> lse
>
> -Todd
>
Re: ACL question w/ Zookeeper 3.1.1
Posted by Mahadev Konar <ma...@yahoo-inc.com>.
HI todd,
From what I understand, you are sayin that a creator_all_acl does not work
with auth?
I tried the following with CREATOR_ALL_ACL and it seemed to work for me...
import org.apache.zookeeper.CreateMode;
import org.apache.zookeeper.WatchedEvent;
import org.apache.zookeeper.Watcher;
import org.apache.zookeeper.ZooKeeper;
import org.apache.zookeeper.data.ACL;
import org.apache.zookeeper.ZooDefs.Ids;
import java.util.ArrayList;
import java.util.List;
public class TestACl implements Watcher {
public static void main(String[] argv) throws Exception {
List<ACL> acls = new ArrayList<ACL>(1);
String authentication_type = "digest";
String authentication = "mahadev:some";
for (ACL ids_acl : Ids.CREATOR_ALL_ACL) {
acls.add(ids_acl);
}
TestACl tacl = new TestACl();
ZooKeeper zoo = new ZooKeeper("localhost:2181", 3000, tacl);
zoo.addAuthInfo(authentication_type, authentication.getBytes());
zoo.create("/some", new byte[0], acls, CreateMode.PERSISTENT);
zoo.setData("/some", new byte[0], -1);
}
@Override
public void process(WatchedEvent event) {
}
}
And it worked on my set of zookeeper servers....
And then
I tried
Without auth
Getdata("/some")
Which correctly gave me the error:
Exception in thread "main"
org.apache.zookeeper.KeeperException$NoAuthException: KeeperErrorCode =
NoAuth for /some
at org.apache.zookeeper.KeeperException.create(KeeperException.java:104)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:42)
at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:892)
at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:921)
at
org.apache.zookeeper.ZooKeeperMain.processZKCmd(ZooKeeperMain.java:692)
at org.apache.zookeeper.ZooKeeperMain.processCmd(ZooKeeperMain.java:579)
at
org.apache.zookeeper.ZooKeeperMain.executeLine(ZooKeeperMain.java:351)
at org.apache.zookeeper.ZooKeeperMain.run(ZooKeeperMain.java:309)
at org.apache.zookeeper.ZooKeeperMain.main(ZooKeeperMain.java:268)
Is this what you are trying to do?
Thanks
mahadev
On 9/17/09 5:05 PM, "Todd Greenwood" <to...@audiencescience.com> wrote:
> I'm attempting to secure a zookeeper installation using zookeeper ACLs.
> However, I'm finding that while Ids.OPEN_ACL_UNSAFE works great, my
> attempts at using Ids.CREATOR_ALL_ACL are failing. Here's a code
> snippet:
>
>
> public class ZooWrapper
> {
>
> /*
> 1. Here I'm setting up my authentication. I've got an ACL list, and my
> authentication strings.
> */
> private final List<ACL> acl = new ArrayList<ACL>( 1 );
> private static final String authentication_type = "digest";
> private static final String authentication =
> "audiencescience:gravy";
>
>
> public ZooWrapper( final String connection_string,
> final String path,
> final int connectiontimeout ) throws
> ZooWrapperException
> {
> ...
> /*
> 2. Here I'm adding the acls
> */
>
> // This works (creates nodes, sets data on nodes)
> for ( ACL ids_acl : Ids.OPEN_ACL_UNSAFE )
> {
> acl.add( ids_acl);
> }
>
> /*
> NOTE: This does not work (nodes are not created, cannot set data on
> nodes b/c nodes do not exist)
> */
>
> // for ( ACL ids_acl : Ids.CREATOR_ALL_ACL )
> // {
> // acl.add( ids_acl );
> // }
>
> /*
> 3. Finally, I create a new zookeeper instance and add my authorization
> info to it.
> */
> zoo = new ZooKeeper( connection_string, connectiontimeout, this );
> zoo.addAuthInfo( authentication_type, authentication.getBytes() )
>
> /*
> 4. Later, I try to write some data into zookeeper by first creating the
> node, and then calling setdata...
> */
> zoo.create( path, new byte[0], acl, CreateMode.PERSISTENT );
>
> zoo.setData( path, bytes, -1 )
>
> As I mentioned above, when I add Ids.OPEN_ACL_UNSAFE to acl, then both
> the create and setData succeed. However, when I use Ids.CREATOR_ALL_ACL,
> then the nodes are not created. Am I missing something obvious w/
> respect to configuring ACLs?
>
> I've used the following references:
>
> http://hadoop.apache.org/zookeeper/docs/r3.1.1/zookeeperProgrammers.html
>
> http://mail-archives.apache.org/mod_mbox/hadoop-zookeeper-commits/200807
> .mbox/%3C20080731201025.C62092388873@eris.apache.org%3E
>
> http://books.google.com/books?id=bKPEwR-Pt6EC&pg=PT404&lpg=PT404&dq=zook
> eeper+ACL+digest+%22new+Id%22&source=bl&ots=kObz0y8eFk&sig=VFCAsNW0mBJyZ
> swoweJDI31iNlo&hl=en&ei=Z82ySojRFsqRlAeqxsyIDw&sa=X&oi=book_result&ct=re
> sult&resnum=6#v=onepage&q=zookeeper%20ACL%20digest%20%22new%20Id%22&f=fa
> lse
>
> -Todd
ACL question w/ Zookeeper 3.1.1
Posted by Todd Greenwood <to...@audiencescience.com>.
I'm attempting to secure a zookeeper installation using zookeeper ACLs.
However, I'm finding that while Ids.OPEN_ACL_UNSAFE works great, my
attempts at using Ids.CREATOR_ALL_ACL are failing. Here's a code
snippet:
public class ZooWrapper
{
/*
1. Here I'm setting up my authentication. I've got an ACL list, and my
authentication strings.
*/
private final List<ACL> acl = new ArrayList<ACL>( 1 );
private static final String authentication_type = "digest";
private static final String authentication =
"audiencescience:gravy";
public ZooWrapper( final String connection_string,
final String path,
final int connectiontimeout ) throws
ZooWrapperException
{
...
/*
2. Here I'm adding the acls
*/
// This works (creates nodes, sets data on nodes)
for ( ACL ids_acl : Ids.OPEN_ACL_UNSAFE )
{
acl.add( ids_acl);
}
/*
NOTE: This does not work (nodes are not created, cannot set data on
nodes b/c nodes do not exist)
*/
// for ( ACL ids_acl : Ids.CREATOR_ALL_ACL )
// {
// acl.add( ids_acl );
// }
/*
3. Finally, I create a new zookeeper instance and add my authorization
info to it.
*/
zoo = new ZooKeeper( connection_string, connectiontimeout, this );
zoo.addAuthInfo( authentication_type, authentication.getBytes() )
/*
4. Later, I try to write some data into zookeeper by first creating the
node, and then calling setdata...
*/
zoo.create( path, new byte[0], acl, CreateMode.PERSISTENT );
zoo.setData( path, bytes, -1 )
As I mentioned above, when I add Ids.OPEN_ACL_UNSAFE to acl, then both
the create and setData succeed. However, when I use Ids.CREATOR_ALL_ACL,
then the nodes are not created. Am I missing something obvious w/
respect to configuring ACLs?
I've used the following references:
http://hadoop.apache.org/zookeeper/docs/r3.1.1/zookeeperProgrammers.html
http://mail-archives.apache.org/mod_mbox/hadoop-zookeeper-commits/200807
.mbox/%3C20080731201025.C62092388873@eris.apache.org%3E
http://books.google.com/books?id=bKPEwR-Pt6EC&pg=PT404&lpg=PT404&dq=zook
eeper+ACL+digest+%22new+Id%22&source=bl&ots=kObz0y8eFk&sig=VFCAsNW0mBJyZ
swoweJDI31iNlo&hl=en&ei=Z82ySojRFsqRlAeqxsyIDw&sa=X&oi=book_result&ct=re
sult&resnum=6#v=onepage&q=zookeeper%20ACL%20digest%20%22new%20Id%22&f=fa
lse
-Todd