Hbase with AWS EMR

[Nitin Gupta <Ni...@e-zest.in>]

Dear All,

Would like to know if there is a good Java example to connect to Hbase deployed on AWS EMR through a Java client.

I am able to connect through the Hbase console using ssh but I am not sure where and how should I provide the AWS credentials to connect through Java client.

Any reference would be appreciable. Thanks!

Thanks & Regards,
Nitin Gupta

RE: Hbase with AWS EMR

[Nitin Gupta <Ni...@e-zest.in>]

Thanks Iain,



I have opened the port 80 for HTTP request . When I try to make a connection I get the below error



Exception in thread "main" java.io.IOException: create request to /namespace3:t4/schema returned 403

                at org.apache.hadoop.hbase.rest.client.RemoteAdmin.createTable(RemoteAdmin.java:310)



Below is the code snippet:



Configuration hBaseConfiguration=HBaseConfiguration.create();

hBaseConfiguration.set("AWSAccessKeyId", "ACTUAL KEY");

hBaseConfiguration.set("AWSSecretKey", "ACTUAL SECRET");



RemoteAdmin admin = new RemoteAdmin(new Client(new Cluster().add("DNS of Master Node on AWS", 8080)), hBaseConfiguration);

HTableDescriptor table = new HTableDescriptor("namespace3:t4".getBytes());

//creating column family descriptor

HColumnDescriptor family = new HColumnDescriptor("cf1".getBytes());



//adding coloumn family to HTable

table.addFamily(family);

admin.createTable(table);



Thanks & Regards,

Nitin Gupta



-----Original Message-----
From: iain wright [mailto:iainwrig@gmail.com]
Sent: Wednesday, January 31, 2018 11:25 AM
To: user@hbase.apache.org
Subject: Re: Hbase with AWS EMR



Not sure re:example but you will need network access to the cluster on relevant ports (via AWS security groups), there is no authentication/authorization afaik for EMR, esp. not integrated with IAM, unless you are trying to add steps to a cluster or bring up a cluster/etc.

from java



You will likely want to just copy the relevant config dirs ( HADOOP_CONF_DIR, YARN_CONF_DIR, HBASE_CONF_DIR or wherever hbase_site.xml is, etc.) to wherever you are establishing your client from, and set them as env variables/get them into your classpath before starting the client





--

Iain Wright



This email message is confidential, intended only for the recipient(s) named above and may contain information that is privileged, exempt from disclosure under applicable law. If you are not the intended recipient, do not disclose or disseminate the message to anyone except the intended recipient. If you have received this message in error, or are not the named recipient(s), please immediately notify the sender by return email, and delete all copies of this message.



On Tue, Jan 30, 2018 at 8:27 PM, Nitin Gupta <Ni...@e-zest.in>> wrote:



> Dear All,

>

> Would like to know if there is a good Java example to connect to Hbase

> deployed on AWS EMR through a Java client.

>

> I am able to connect through the Hbase console using ssh but I am not

> sure where and how should I provide the AWS credentials to connect

> through Java client.

>

> Any reference would be appreciable. Thanks!

>

> Thanks & Regards,

> Nitin Gupta

>

>

Re: Hbase with AWS EMR

[iain wright <ia...@gmail.com>]

Not sure re:example but you will need network access to the cluster on
relevant ports (via AWS security groups), there is no
authentication/authorization afaik for EMR, esp. not integrated with IAM,
unless you are trying to add steps to a cluster or bring up a cluster/etc.
from java

You will likely want to just copy the relevant config dirs (
HADOOP_CONF_DIR, YARN_CONF_DIR, HBASE_CONF_DIR or wherever hbase_site.xml
is, etc.) to wherever you are establishing your client from, and set them
as env variables/get them into your classpath before starting the client


-- 
Iain Wright

This email message is confidential, intended only for the recipient(s)
named above and may contain information that is privileged, exempt from
disclosure under applicable law. If you are not the intended recipient, do
not disclose or disseminate the message to anyone except the intended
recipient. If you have received this message in error, or are not the named
recipient(s), please immediately notify the sender by return email, and
delete all copies of this message.

On Tue, Jan 30, 2018 at 8:27 PM, Nitin Gupta <Ni...@e-zest.in> wrote:

> Dear All,
>
> Would like to know if there is a good Java example to connect to Hbase
> deployed on AWS EMR through a Java client.
>
> I am able to connect through the Hbase console using ssh but I am not sure
> where and how should I provide the AWS credentials to connect through Java
> client.
>
> Any reference would be appreciable. Thanks!
>
> Thanks & Regards,
> Nitin Gupta
>
>