You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by sh...@accenture.com on 2013/09/05 12:22:56 UTC
error executing the tutorial ->
https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
Hi,
I am executing the example, but got initial roadblock while trying out "Setting up tables (DDL Statements)" section.
When I executed the command "hive> create external table kv (key int, values string) location 's3n://data.s3ndemo.hive/kv';"
I got the error
FAILED: Error in metadata: MetaException(message:java.lang.IllegalArgumentException: AWS Access Key ID and Secret Access Key must be specified as the username or password (respectively) of a s3n URL, or by setting the fs.s3n.awsAccessKeyId or fs.s3n.awsSecretAccessKey properties (respectively).)
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
hive>
Please help.
Thanks,
Shouvanik
________________________________
This message is for the designated recipient only and may contain privileged, proprietary, or otherwise confidential information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of the e-mail by you is prohibited.
Where allowed by local law, electronic communications with Accenture and its affiliates, including e-mail and instant messaging (including content), may be scanned by our systems for the purposes of information security and assessment of internal compliance with Accenture policy.
______________________________________________________________________________________
www.accenture.com
RE: error executing the tutorial ->
https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
Posted by sh...@accenture.com.
Once I have access keys & ID in hand, I will execute this step and let you know.
But, I suspect that the access secret key might have "/" in it. I came across this JIRA-> https://issues.apache.org/jira/browse/HADOOP-3733 .
Do you know if there is a workaround other than regenerating secret key without a slash.
Thanks again for your reply.
Thanks,
Shouvanik
From: Jitendra Yadav [mailto:jeetuyadav200890@gmail.com]
Sent: Thursday, September 05, 2013 6:02 PM
To: user@hive.apache.org
Subject: Re: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
OK got your point, As an alternative you can set these variables in hive environment before creating the table
Hive > set fs.s3n.awsSecretAccessKey='value'
Hive > set fs.s3n.awsAccessKeyId='Value'
Hope this will help you.
Thanks
Jitendra
On Thu, Sep 5, 2013 at 5:19 PM, <sh...@accenture.com>> wrote:
Hi,
Thanks for the reply.
I had seen in some posts, that you could mention these keys on the Hive CLI. If things work, then I am later planning to include it in core-site.xml of Hadoop config. Presently, I am not allowed to restart Hadoop services.
"create external table ....
.....
LOCATION 's3n:// <fs.s3n.awsAccessKeyId: fs.s3n>.<awsSecretAccessKey >@ap-hw-dev/service-names';
Please help me.
Thanks,
Shouvanik
From: Jitendra Yadav [mailto:jeetuyadav200890@gmail.com<ma...@gmail.com>]
Sent: Thursday, September 05, 2013 5:11 PM
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: Re: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
Hi,
Did you configured fs.s3n.awsAccessKeyId or fs.s3n.awsSecretAccessKey properties
in your core-site.xml file?
Regards
Jitendra
On Thu, Sep 5, 2013 at 3:52 PM, <sh...@accenture.com>> wrote:
Hi,
I am executing the example, but got initial roadblock while trying out "Setting up tables (DDL Statements)" section.
When I executed the command "hive> create external table kv (key int, values string) location 's3n://data.s3ndemo.hive/kv';"
I got the error
FAILED: Error in metadata: MetaException(message:java.lang.IllegalArgumentException: AWS Access Key ID and Secret Access Key must be specified as the username or password (respectively) of a s3n URL, or by setting the fs.s3n.awsAccessKeyId or fs.s3n.awsSecretAccessKey properties (respectively).)
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
hive>
Please help.
Thanks,
Shouvanik
________________________________
This message is for the designated recipient only and may contain privileged, proprietary, or otherwise confidential information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of the e-mail by you is prohibited.
Where allowed by local law, electronic communications with Accenture and its affiliates, including e-mail and instant messaging (including content), may be scanned by our systems for the purposes of information security and assessment of internal compliance with Accenture policy.
______________________________________________________________________________________
www.accenture.com<http://www.accenture.com/>
RE: error executing the tutorial ->
https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
Posted by sh...@accenture.com.
Hi Nitin,
I was not able to find the metastore db. i.e. where it was located?
Finally, I found that out & started metastore db.
But before that, I added the properties, specific to AWS for s3 access, to hive-site.xml where metastore db resides.
Once metastore db is up (i.e. issuing command hive -service metastore), I was able to successfully create the table in HIVE.
Next, when I wanted to fire a "select * from .." query, again I was facing with the error that properties pertaining to AWS not set. That led me to change the hive-site.xml file on the node where HIVE is installed. Once properties are added, I restarted HIVE, and now was able to read rows of the HIVE table.
So, the HIVE issue is resolved.
Thanks a lot for your mail and support.
Thanks,
Shouvanik
From: Nitin Pawar [mailto:nitinpawar432@gmail.com]
Sent: Thursday, September 12, 2013 1:55 AM
To: user@hive.apache.org
Subject: Re: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
Did you try putting those values inside core-site.xml for hdfs?
On Tue, Sep 10, 2013 at 8:11 PM, <sh...@accenture.com>> wrote:
Hi Nitin,
Latest update. Am able to start metastore and table getting created. But when I execute the command "hive> select * from table_name", hive compaints
Failed with exception java.io.IOException:java.lang.IllegalArgumentException: AWS Access Key ID and Secret Access Key must be specified as the username or password (respectively) of a s3n URL, or by setting the fs.s3n.awsAccessKeyId or fs.s3n.awsSecretAccessKey properties (respectively).
Time taken: 1.158 seconds
Thanks,
Shouvanik
From: Haldar, Shouvanik
Sent: Tuesday, September 10, 2013 5:04 PM
To: Haldar, Shouvanik; user@hive.apache.org<ma...@hive.apache.org>
Subject: RE: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
Sorry again. File system will be hadoop
Sent from my Windows Phone
________________________________
From: Haldar, Shouvanik<ma...@accenture.com>
Sent: 10-09-2013 16:59
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: RE: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
No. File system will be s3n, or native file system or hadoop. But using HIVE we are storing data that is already existing inside s3
From: Nitin Pawar [mailto:nitinpawar432@gmail.com]
Sent: Tuesday, September 10, 2013 4:56 PM
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: Re: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
Ok, so back to basics.
correct me If I am wrong
You are trying to setup YARN using s3 as file system?
Then you want to use hive on top of it.
On Tue, Sep 10, 2013 at 4:48 PM, <sh...@accenture.com>> wrote:
The issue is if I paste it in core-site.xml of Hadoop on the node in question, then will starting datanode would do only? Actually,. I am a bit confused what all daemons to start. I have a YARN architecture running 1 separate - Resource Manager, 1 NameNode & 4 datanodes.
From: Nitin Pawar [mailto:nitinpawar432@gmail.com<ma...@gmail.com>]
Sent: Tuesday, September 10, 2013 4:43 PM
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: Re: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
There is not more to start. If hive is not starting properly, it will not start at all so you are doing ok.
Problem is when you create table, hive tries to create a directory with that name if its not existing. Just try to put those two settings inside core-site.xml and see if the hdfs can use it to create directory
On Tue, Sep 10, 2013 at 4:06 PM, <sh...@accenture.com>> wrote:
The values are showing when I run "hive> set;" command. But it's not getting identified. I guess I am not starting HIVE properly. Do you know what are the steps?
From: Nitin Pawar [mailto:nitinpawar432@gmail.com<ma...@gmail.com>]
Sent: Tuesday, September 10, 2013 4:04 PM
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: Re: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
>From the config file it looks ok. Not really sure why it is not picking up when you run it via hive cli.
Can you try putting these in core-site.xml of hdfs.
<property>
<name>fs.s3n.awsAccessKeyId</name>
<value>ID</value>
</property>
<property>
<name>fs.s3n.awsSecretAccessKey</name>
<value>SECRET</value>
</property>
On Tue, Sep 10, 2013 at 3:11 PM, <sh...@accenture.com>> wrote:
Hi Nitin,
PFA hive-site.xml. I wanted to start Hortonworks HIVE. Can you please help.
Thanks,
Shouvanik
From: Nitin Pawar [mailto:nitinpawar432@gmail.com<ma...@gmail.com>]
Sent: Tuesday, September 10, 2013 11:15 AM
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: Re: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
if you have set these values inside hive-site.xml and restarted hive service. It should work.
Share your hive-site.xml (please remove values for s3naccess key and secret )
On Tue, Sep 10, 2013 at 3:17 AM, <sh...@accenture.com>> wrote:
Please help. I am badly stuck on this. Please help.
From: Haldar, Shouvanik
Sent: Tuesday, September 10, 2013 1:03 AM
To: 'user@hive.apache.org<ma...@hive.apache.org>'
Subject: RE: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
When I set values inside hive-site.xml & test whether values are set by typing hive> set; I see that the values are set.
Yet, when I run "CREATE EXTERNAL TABLE..." command, I get the same error
FAILED: Error in metadata: MetaException(message:java.lang.IllegalArgumentException: AWS Access Key ID and Secret Access Key must be specified as the username or password (respectively) of a s3n URL, or by setting the fs.s3n.awsAccessKeyId or fs.s3n.awsSecretAccessKey properties (respectively).)
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
Please help.
Thanks,
Shouvanik
From: Haldar, Shouvanik
Sent: Tuesday, September 10, 2013 12:39 AM
To: 'user@hive.apache.org<ma...@hive.apache.org>'
Subject: RE: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
Hi Nitin,
Please help me with the below?
Thanks,
Shouvanik
From: Haldar, Shouvanik
Sent: Saturday, September 07, 2013 2:30 AM
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: RE: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
Hi ,
I tried to set the parameters inside hive-site.xml on the node from where I am running hive. Even "hive>set;" command showed me that the parameters were successfully set. Yet, when running hive "CREATE EXTERNAL TABLE...", I am getting the following error
FAILED: Error in metadata: MetaException(message:java.lang.IllegalArgumentException: AWS Access Key ID and Secret Access Key must be specified as the username or password (respectively) of a s3n URL, or by setting the fs.s3n.awsAccessKeyId or fs.s3n.awsSecretAccessKey properties (respectively).)
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
Please help.
Thanks,
Shouvanik
From: Nitin Pawar [mailto:nitinpawar432@gmail.com]
Sent: Saturday, September 07, 2013 2:25 AM
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: Re: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
did you try setting them in hive-site.xml file ?
On Sat, Sep 7, 2013 at 2:18 AM, <sh...@accenture.com>> wrote:
Hi,
Can anybody please reply to this below? I am stuck up very bad.
Regards,
Shouvanik
From: Haldar, Shouvanik
Sent: Friday, September 06, 2013 11:22 PM
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: RE: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
Hi,
Is it possible to permanently set the below parameters? Please help.
Thanks,
SHouvanik
From: Jitendra Yadav [mailto:jeetuyadav200890@gmail.com]
Sent: Thursday, September 05, 2013 6:02 PM
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: Re: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
OK got your point, As an alternative you can set these variables in hive environment before creating the table
Hive > set fs.s3n.awsSecretAccessKey='value'
Hive > set fs.s3n.awsAccessKeyId='Value'
Hope this will help you.
Thanks
Jitendra
On Thu, Sep 5, 2013 at 5:19 PM, <sh...@accenture.com>> wrote:
Hi,
Thanks for the reply.
I had seen in some posts, that you could mention these keys on the Hive CLI. If things work, then I am later planning to include it in core-site.xml of Hadoop config. Presently, I am not allowed to restart Hadoop services.
"create external table ....
.....
LOCATION 's3n:// <fs.s3n.awsAccessKeyId: fs.s3n>.<awsSecretAccessKey >@ap-hw-dev/service-names';
Please help me.
Thanks,
Shouvanik
From: Jitendra Yadav [mailto:jeetuyadav200890@gmail.com<ma...@gmail.com>]
Sent: Thursday, September 05, 2013 5:11 PM
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: Re: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
Hi,
Did you configured fs.s3n.awsAccessKeyId or fs.s3n.awsSecretAccessKey properties
in your core-site.xml file?
Regards
Jitendra
On Thu, Sep 5, 2013 at 3:52 PM, <sh...@accenture.com>> wrote:
Hi,
I am executing the example, but got initial roadblock while trying out "Setting up tables (DDL Statements)" section.
When I executed the command "hive> create external table kv (key int, values string) location 's3n://data.s3ndemo.hive/kv';"
I got the error
FAILED: Error in metadata: MetaException(message:java.lang.IllegalArgumentException: AWS Access Key ID and Secret Access Key must be specified as the username or password (respectively) of a s3n URL, or by setting the fs.s3n.awsAccessKeyId or fs.s3n.awsSecretAccessKey properties (respectively).)
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
hive>
Please help.
Thanks,
Shouvanik
________________________________
This message is for the designated recipient only and may contain privileged, proprietary, or otherwise confidential information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of the e-mail by you is prohibited.
Where allowed by local law, electronic communications with Accenture and its affiliates, including e-mail and instant messaging (including content), may be scanned by our systems for the purposes of information security and assessment of internal compliance with Accenture policy.
______________________________________________________________________________________
www.accenture.com<http://www.accenture.com/>
--
Nitin Pawar
--
Nitin Pawar
--
Nitin Pawar
--
Nitin Pawar
--
Nitin Pawar
--
Nitin Pawar
Re: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
Posted by Nitin Pawar <ni...@gmail.com>.
Did you try putting those values inside core-site.xml for hdfs?
On Tue, Sep 10, 2013 at 8:11 PM, <sh...@accenture.com> wrote:
> Hi Nitin,****
>
> Latest update. Am able to start metastore and table getting created. But
> when I execute the command “hive> select * from table_name”, hive compaints
> ****
>
> ** **
>
> Failed with exception
> java.io.IOException:java.lang.IllegalArgumentException: AWS Access Key ID
> and Secret Access Key must be specified as the username or password
> (respectively) of a s3n URL, or by setting the fs.s3n.awsAccessKeyId or
> fs.s3n.awsSecretAccessKey properties (respectively).****
>
> Time taken: 1.158 seconds****
>
> ** **
>
> ** **
>
> Thanks,****
>
> Shouvanik****
>
> ** **
>
> *From:* Haldar, Shouvanik
> *Sent:* Tuesday, September 10, 2013 5:04 PM
> *To:* Haldar, Shouvanik; user@hive.apache.org
>
> *Subject:* RE: error executing the tutorial ->
> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
> ****
>
> ** **
>
> Sorry again. File system will be hadoop
>
> Sent from my Windows Phone****
> ------------------------------
>
> *From: *Haldar, Shouvanik <sh...@accenture.com>
> *Sent: *10-09-2013 16:59
> *To: *user@hive.apache.org
> *Subject: *RE: error executing the tutorial ->
> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
> ****
>
> No. File system will be s3n, or native file system or hadoop. But using
> HIVE we are storing data that is already existing inside s3****
>
> ****
>
> *From:* Nitin Pawar [mailto:nitinpawar432@gmail.com<ni...@gmail.com>]
>
> *Sent:* Tuesday, September 10, 2013 4:56 PM
> *To:* user@hive.apache.org
> *Subject:* Re: error executing the tutorial ->
> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
> ****
>
> ****
>
> Ok, so back to basics. ****
>
> ****
>
> correct me If I am wrong****
>
> ****
>
> You are trying to setup YARN using s3 as file system? ****
>
> Then you want to use hive on top of it. ****
>
> ****
>
> ****
>
> On Tue, Sep 10, 2013 at 4:48 PM, <sh...@accenture.com> wrote:**
> **
>
> The issue is if I paste it in core-site.xml of Hadoop on the node in
> question, then will starting datanode would do only? Actually,. I am a bit
> confused what all daemons to start. I have a YARN architecture running 1
> separate – Resource Manager, 1 NameNode & 4 datanodes.****
>
> ****
>
> *From:* Nitin Pawar [mailto:nitinpawar432@gmail.com]
> *Sent:* Tuesday, September 10, 2013 4:43 PM****
>
>
> *To:* user@hive.apache.org
> *Subject:* Re: error executing the tutorial ->
> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
> ****
>
> ****
>
> There is not more to start. If hive is not starting properly, it will not
> start at all so you are doing ok. ****
>
> ****
>
> Problem is when you create table, hive tries to create a directory with
> that name if its not existing. Just try to put those two settings inside
> core-site.xml and see if the hdfs can use it to create directory ****
>
> ****
>
> On Tue, Sep 10, 2013 at 4:06 PM, <sh...@accenture.com> wrote:**
> **
>
> The values are showing when I run “hive> set;” command. But it’s not
> getting identified. I guess I am not starting HIVE properly. Do you know
> what are the steps?****
>
> ****
>
> *From:* Nitin Pawar [mailto:nitinpawar432@gmail.com]
> *Sent:* Tuesday, September 10, 2013 4:04 PM****
>
>
> *To:* user@hive.apache.org
> *Subject:* Re: error executing the tutorial ->
> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
> ****
>
> ****
>
> From the config file it looks ok. Not really sure why it is not picking up
> when you run it via hive cli. ****
>
> ****
>
> Can you try putting these in core-site.xml of hdfs. ****
>
> ****
>
> ****
>
> <property>****
>
> <name>fs.s3n.awsAccessKeyId</name>****
>
> <value>ID</value>****
>
> </property>****
>
> <property>****
>
> <name>fs.s3n.awsSecretAccessKey</name>****
>
> <value>SECRET</value>****
>
> </property>****
>
> ****
>
> On Tue, Sep 10, 2013 at 3:11 PM, <sh...@accenture.com> wrote:**
> **
>
> Hi Nitin,****
>
> ****
>
> PFA hive-site.xml. I wanted to start Hortonworks HIVE. Can you please help.
> ****
>
> ****
>
> Thanks,****
>
> Shouvanik****
>
> ****
>
> *From:* Nitin Pawar [mailto:nitinpawar432@gmail.com]
> *Sent:* Tuesday, September 10, 2013 11:15 AM****
>
>
> *To:* user@hive.apache.org
> *Subject:* Re: error executing the tutorial ->
> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
> ****
>
> ****
>
> if you have set these values inside hive-site.xml and restarted hive
> service. It should work. ****
>
> ****
>
> Share your hive-site.xml (please remove values for s3naccess key and
> secret ) ****
>
> ****
>
> On Tue, Sep 10, 2013 at 3:17 AM, <sh...@accenture.com> wrote:**
> **
>
> Please help. I am badly stuck on this. Please help.****
>
> ****
>
> *From:* Haldar, Shouvanik
> *Sent:* Tuesday, September 10, 2013 1:03 AM****
>
>
> *To:* 'user@hive.apache.org'
> *Subject:* RE: error executing the tutorial ->
> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
> ****
>
> ****
>
> When I set values inside hive-site.xml & test whether values are set by
> typing hive> set; I see that the values are set.****
>
> ****
>
> Yet, when I run “CREATE EXTERNAL TABLE…” command, I get the same error****
>
> ****
>
> FAILED: Error in metadata:
> MetaException(message:java.lang.IllegalArgumentException: AWS Access Key ID
> and Secret Access Key must be specified as the username or password
> (respectively) of a s3n URL, or by setting the fs.s3n.awsAccessKeyId or
> fs.s3n.awsSecretAccessKey properties (respectively).)****
>
> FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.DDLTask****
>
> ****
>
> Please help.****
>
> ****
>
> Thanks,****
>
> Shouvanik****
>
> ****
>
> *From:* Haldar, Shouvanik ****
>
> *Sent:* Tuesday, September 10, 2013 12:39 AM****
>
> *To:* 'user@hive.apache.org'
> *Subject:* RE: error executing the tutorial ->
> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
> ****
>
> ****
>
> Hi Nitin,****
>
> ****
>
> Please help me with the below?****
>
> ****
>
> Thanks,****
>
> Shouvanik****
>
> ****
>
> *From:* Haldar, Shouvanik ****
>
> *Sent:* Saturday, September 07, 2013 2:30 AM****
>
> *To:* user@hive.apache.org
> *Subject:* RE: error executing the tutorial ->
> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
> ****
>
> ****
>
> Hi ,****
>
> ****
>
> I tried to set the parameters inside hive-site.xml on the node from where
> I am running hive. Even “hive>set;” command showed me that the parameters
> were successfully set. Yet, when running hive “CREATE EXTERNAL TABLE…”, I
> am getting the following error****
>
> ****
>
> FAILED: Error in metadata:
> MetaException(message:java.lang.IllegalArgumentException: AWS Access Key ID
> and Secret Access Key must be specified as the username or password
> (respectively) of a s3n URL, or by setting the fs.s3n.awsAccessKeyId or
> fs.s3n.awsSecretAccessKey properties (respectively).)****
>
> FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.DDLTask ****
>
> ****
>
> ****
>
> Please help.****
>
> ****
>
> Thanks,****
>
> Shouvanik****
>
> ****
>
> *From:* Nitin Pawar [mailto:nitinpawar432@gmail.com<ni...@gmail.com>]
> ****
>
> *Sent:* Saturday, September 07, 2013 2:25 AM****
>
> *To:* user@hive.apache.org
> *Subject:* Re: error executing the tutorial ->
> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
> ****
>
> ****
>
> did you try setting them in hive-site.xml file ? ****
>
> ****
>
> On Sat, Sep 7, 2013 at 2:18 AM, <sh...@accenture.com> wrote:***
> *
>
> Hi,****
>
> ****
>
> Can anybody please reply to this below? I am stuck up very bad.****
>
> ****
>
> Regards,****
>
> Shouvanik****
>
> ****
>
> *From:* Haldar, Shouvanik
> *Sent:* Friday, September 06, 2013 11:22 PM
> *To:* user@hive.apache.org
> *Subject:* RE: error executing the tutorial ->
> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
> ****
>
> ****
>
> Hi,****
>
> ****
>
> Is it possible to permanently set the below parameters? Please help.****
>
> ****
>
> Thanks,****
>
> SHouvanik****
>
> ****
>
> *From:* Jitendra Yadav [mailto:jeetuyadav200890@gmail.com<je...@gmail.com>]
>
> *Sent:* Thursday, September 05, 2013 6:02 PM****
>
>
> *To:* user@hive.apache.org
> *Subject:* Re: error executing the tutorial ->
> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
> ****
>
> ****
>
> OK got your point, As an alternative you can set these variables in hive
> environment before creating the table****
>
> ****
>
> Hive > set fs.s3n.awsSecretAccessKey='value'
> Hive > set fs.s3n.awsAccessKeyId='Value'****
>
> ****
>
> Hope this will help you.****
>
> ****
>
> Thanks****
>
> Jitendra
> On Thu, Sep 5, 2013 at 5:19 PM, <sh...@accenture.com> wrote:***
> *
>
> Hi,****
>
> ****
>
> Thanks for the reply.****
>
> ****
>
> I had seen in some posts, that you could mention these keys on the Hive
> CLI. If things work, then I am later planning to include it in
> core-site.xml of Hadoop config. Presently, I am not allowed to restart
> Hadoop services. ****
>
> ****
>
> “create external table ….****
>
> …..****
>
> LOCATION 's3n:// <fs.s3n.awsAccessKeyId: fs.s3n>.<awsSecretAccessKey>@ap-hw-dev/service-names';
> ****
>
> ****
>
> Please help me.****
>
> ****
>
> Thanks,****
>
> Shouvanik****
>
> ****
>
> *From:* Jitendra Yadav [mailto:jeetuyadav200890@gmail.com]
> *Sent:* Thursday, September 05, 2013 5:11 PM
> *To:* user@hive.apache.org
> *Subject:* Re: error executing the tutorial ->
> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
> ****
>
> ****
>
> Hi,****
>
> ****
>
> Did you configured fs.s3n.awsAccessKeyId or fs.s3n.awsSecretAccessKey
> properties****
>
> in your core-site.xml file?****
>
> ****
>
> Regards****
>
> Jitendra****
>
> On Thu, Sep 5, 2013 at 3:52 PM, <sh...@accenture.com> wrote:***
> *
>
> Hi,****
>
> ****
>
> I am executing the example, but got initial roadblock while trying out
> “Setting up tables (DDL Statements)” section.****
>
> ****
>
> When I executed the command “hive> create external table kv (key int,
> values string) location 's3n://data.s3ndemo.hive/kv';”****
>
> ****
>
> I got the error****
>
> ****
>
> FAILED: Error in metadata:
> MetaException(message:java.lang.IllegalArgumentException: AWS Access Key ID
> and Secret Access Key must be specified as the username or password
> (respectively) of a s3n URL, or by setting the fs.s3n.awsAccessKeyId or
> fs.s3n.awsSecretAccessKey properties (respectively).)****
>
> FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.DDLTask****
>
> hive>****
>
> ****
>
> Please help.****
>
> ****
>
> ****
>
> Thanks,****
>
> Shouvanik****
>
> ****
> ------------------------------
>
> This message is for the designated recipient only and may contain
> privileged, proprietary, or otherwise confidential information. If you have
> received it in error, please notify the sender immediately and delete the
> original. Any other use of the e-mail by you is prohibited.
>
> Where allowed by local law, electronic communications with Accenture and
> its affiliates, including e-mail and instant messaging (including content),
> may be scanned by our systems for the purposes of information security and
> assessment of internal compliance with Accenture policy.
>
>
> ______________________________________________________________________________________
>
> www.accenture.com****
>
> ****
>
> ****
>
>
>
> ****
>
> ****
>
> --
> Nitin Pawar****
>
>
>
> ****
>
> ****
>
> --
> Nitin Pawar****
>
>
>
> ****
>
> ****
>
> --
> Nitin Pawar****
>
>
>
> ****
>
> ****
>
> --
> Nitin Pawar****
>
>
>
> ****
>
> ****
>
> --
> Nitin Pawar****
>
--
Nitin Pawar
RE: error executing the tutorial ->
https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
Posted by sh...@accenture.com.
Hi Nitin,
Latest update. Am able to start metastore and table getting created. But when I execute the command “hive> select * from table_name”, hive compaints
Failed with exception java.io.IOException:java.lang.IllegalArgumentException: AWS Access Key ID and Secret Access Key must be specified as the username or password (respectively) of a s3n URL, or by setting the fs.s3n.awsAccessKeyId or fs.s3n.awsSecretAccessKey properties (respectively).
Time taken: 1.158 seconds
Thanks,
Shouvanik
From: Haldar, Shouvanik
Sent: Tuesday, September 10, 2013 5:04 PM
To: Haldar, Shouvanik; user@hive.apache.org
Subject: RE: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
Sorry again. File system will be hadoop
Sent from my Windows Phone
________________________________
From: Haldar, Shouvanik<ma...@accenture.com>
Sent: 10-09-2013 16:59
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: RE: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
No. File system will be s3n, or native file system or hadoop. But using HIVE we are storing data that is already existing inside s3
From: Nitin Pawar [mailto:nitinpawar432@gmail.com]
Sent: Tuesday, September 10, 2013 4:56 PM
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: Re: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
Ok, so back to basics.
correct me If I am wrong
You are trying to setup YARN using s3 as file system?
Then you want to use hive on top of it.
On Tue, Sep 10, 2013 at 4:48 PM, <sh...@accenture.com>> wrote:
The issue is if I paste it in core-site.xml of Hadoop on the node in question, then will starting datanode would do only? Actually,. I am a bit confused what all daemons to start. I have a YARN architecture running 1 separate – Resource Manager, 1 NameNode & 4 datanodes.
From: Nitin Pawar [mailto:nitinpawar432@gmail.com<ma...@gmail.com>]
Sent: Tuesday, September 10, 2013 4:43 PM
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: Re: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
There is not more to start. If hive is not starting properly, it will not start at all so you are doing ok.
Problem is when you create table, hive tries to create a directory with that name if its not existing. Just try to put those two settings inside core-site.xml and see if the hdfs can use it to create directory
On Tue, Sep 10, 2013 at 4:06 PM, <sh...@accenture.com>> wrote:
The values are showing when I run “hive> set;” command. But it’s not getting identified. I guess I am not starting HIVE properly. Do you know what are the steps?
From: Nitin Pawar [mailto:nitinpawar432@gmail.com<ma...@gmail.com>]
Sent: Tuesday, September 10, 2013 4:04 PM
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: Re: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
>From the config file it looks ok. Not really sure why it is not picking up when you run it via hive cli.
Can you try putting these in core-site.xml of hdfs.
<property>
<name>fs.s3n.awsAccessKeyId</name>
<value>ID</value>
</property>
<property>
<name>fs.s3n.awsSecretAccessKey</name>
<value>SECRET</value>
</property>
On Tue, Sep 10, 2013 at 3:11 PM, <sh...@accenture.com>> wrote:
Hi Nitin,
PFA hive-site.xml. I wanted to start Hortonworks HIVE. Can you please help.
Thanks,
Shouvanik
From: Nitin Pawar [mailto:nitinpawar432@gmail.com<ma...@gmail.com>]
Sent: Tuesday, September 10, 2013 11:15 AM
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: Re: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
if you have set these values inside hive-site.xml and restarted hive service. It should work.
Share your hive-site.xml (please remove values for s3naccess key and secret )
On Tue, Sep 10, 2013 at 3:17 AM, <sh...@accenture.com>> wrote:
Please help. I am badly stuck on this. Please help.
From: Haldar, Shouvanik
Sent: Tuesday, September 10, 2013 1:03 AM
To: 'user@hive.apache.org<ma...@hive.apache.org>'
Subject: RE: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
When I set values inside hive-site.xml & test whether values are set by typing hive> set; I see that the values are set.
Yet, when I run “CREATE EXTERNAL TABLE…” command, I get the same error
FAILED: Error in metadata: MetaException(message:java.lang.IllegalArgumentException: AWS Access Key ID and Secret Access Key must be specified as the username or password (respectively) of a s3n URL, or by setting the fs.s3n.awsAccessKeyId or fs.s3n.awsSecretAccessKey properties (respectively).)
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
Please help.
Thanks,
Shouvanik
From: Haldar, Shouvanik
Sent: Tuesday, September 10, 2013 12:39 AM
To: 'user@hive.apache.org<ma...@hive.apache.org>'
Subject: RE: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
Hi Nitin,
Please help me with the below?
Thanks,
Shouvanik
From: Haldar, Shouvanik
Sent: Saturday, September 07, 2013 2:30 AM
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: RE: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
Hi ,
I tried to set the parameters inside hive-site.xml on the node from where I am running hive. Even “hive>set;” command showed me that the parameters were successfully set. Yet, when running hive “CREATE EXTERNAL TABLE…”, I am getting the following error
FAILED: Error in metadata: MetaException(message:java.lang.IllegalArgumentException: AWS Access Key ID and Secret Access Key must be specified as the username or password (respectively) of a s3n URL, or by setting the fs.s3n.awsAccessKeyId or fs.s3n.awsSecretAccessKey properties (respectively).)
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
Please help.
Thanks,
Shouvanik
From: Nitin Pawar [mailto:nitinpawar432@gmail.com]
Sent: Saturday, September 07, 2013 2:25 AM
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: Re: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
did you try setting them in hive-site.xml file ?
On Sat, Sep 7, 2013 at 2:18 AM, <sh...@accenture.com>> wrote:
Hi,
Can anybody please reply to this below? I am stuck up very bad.
Regards,
Shouvanik
From: Haldar, Shouvanik
Sent: Friday, September 06, 2013 11:22 PM
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: RE: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
Hi,
Is it possible to permanently set the below parameters? Please help.
Thanks,
SHouvanik
From: Jitendra Yadav [mailto:jeetuyadav200890@gmail.com]
Sent: Thursday, September 05, 2013 6:02 PM
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: Re: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
OK got your point, As an alternative you can set these variables in hive environment before creating the table
Hive > set fs.s3n.awsSecretAccessKey='value'
Hive > set fs.s3n.awsAccessKeyId='Value'
Hope this will help you.
Thanks
Jitendra
On Thu, Sep 5, 2013 at 5:19 PM, <sh...@accenture.com>> wrote:
Hi,
Thanks for the reply.
I had seen in some posts, that you could mention these keys on the Hive CLI. If things work, then I am later planning to include it in core-site.xml of Hadoop config. Presently, I am not allowed to restart Hadoop services.
“create external table ….
…..
LOCATION 's3n:// <fs.s3n.awsAccessKeyId: fs.s3n>.<awsSecretAccessKey >@ap-hw-dev/service-names';
Please help me.
Thanks,
Shouvanik
From: Jitendra Yadav [mailto:jeetuyadav200890@gmail.com<ma...@gmail.com>]
Sent: Thursday, September 05, 2013 5:11 PM
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: Re: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
Hi,
Did you configured fs.s3n.awsAccessKeyId or fs.s3n.awsSecretAccessKey properties
in your core-site.xml file?
Regards
Jitendra
On Thu, Sep 5, 2013 at 3:52 PM, <sh...@accenture.com>> wrote:
Hi,
I am executing the example, but got initial roadblock while trying out “Setting up tables (DDL Statements)” section.
When I executed the command “hive> create external table kv (key int, values string) location 's3n://data.s3ndemo.hive/kv';”
I got the error
FAILED: Error in metadata: MetaException(message:java.lang.IllegalArgumentException: AWS Access Key ID and Secret Access Key must be specified as the username or password (respectively) of a s3n URL, or by setting the fs.s3n.awsAccessKeyId or fs.s3n.awsSecretAccessKey properties (respectively).)
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
hive>
Please help.
Thanks,
Shouvanik
________________________________
This message is for the designated recipient only and may contain privileged, proprietary, or otherwise confidential information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of the e-mail by you is prohibited.
Where allowed by local law, electronic communications with Accenture and its affiliates, including e-mail and instant messaging (including content), may be scanned by our systems for the purposes of information security and assessment of internal compliance with Accenture policy.
______________________________________________________________________________________
www.accenture.com<http://www.accenture.com/>
--
Nitin Pawar
--
Nitin Pawar
--
Nitin Pawar
--
Nitin Pawar
--
Nitin Pawar
RE: error executing the tutorial ->
https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
Posted by sh...@accenture.com.
Sorry again. File system will be hadoop
Sent from my Windows Phone
________________________________
From: Haldar, Shouvanik<ma...@accenture.com>
Sent: 10-09-2013 16:59
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: RE: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
No. File system will be s3n, or native file system or hadoop. But using HIVE we are storing data that is already existing inside s3
From: Nitin Pawar [mailto:nitinpawar432@gmail.com]
Sent: Tuesday, September 10, 2013 4:56 PM
To: user@hive.apache.org
Subject: Re: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
Ok, so back to basics.
correct me If I am wrong
You are trying to setup YARN using s3 as file system?
Then you want to use hive on top of it.
On Tue, Sep 10, 2013 at 4:48 PM, <sh...@accenture.com>> wrote:
The issue is if I paste it in core-site.xml of Hadoop on the node in question, then will starting datanode would do only? Actually,. I am a bit confused what all daemons to start. I have a YARN architecture running 1 separate – Resource Manager, 1 NameNode & 4 datanodes.
From: Nitin Pawar [mailto:nitinpawar432@gmail.com<ma...@gmail.com>]
Sent: Tuesday, September 10, 2013 4:43 PM
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: Re: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
There is not more to start. If hive is not starting properly, it will not start at all so you are doing ok.
Problem is when you create table, hive tries to create a directory with that name if its not existing. Just try to put those two settings inside core-site.xml and see if the hdfs can use it to create directory
On Tue, Sep 10, 2013 at 4:06 PM, <sh...@accenture.com>> wrote:
The values are showing when I run “hive> set;” command. But it’s not getting identified. I guess I am not starting HIVE properly. Do you know what are the steps?
From: Nitin Pawar [mailto:nitinpawar432@gmail.com<ma...@gmail.com>]
Sent: Tuesday, September 10, 2013 4:04 PM
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: Re: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
>From the config file it looks ok. Not really sure why it is not picking up when you run it via hive cli.
Can you try putting these in core-site.xml of hdfs.
<property>
<name>fs.s3n.awsAccessKeyId</name>
<value>ID</value>
</property>
<property>
<name>fs.s3n.awsSecretAccessKey</name>
<value>SECRET</value>
</property>
On Tue, Sep 10, 2013 at 3:11 PM, <sh...@accenture.com>> wrote:
Hi Nitin,
PFA hive-site.xml. I wanted to start Hortonworks HIVE. Can you please help.
Thanks,
Shouvanik
From: Nitin Pawar [mailto:nitinpawar432@gmail.com<ma...@gmail.com>]
Sent: Tuesday, September 10, 2013 11:15 AM
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: Re: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
if you have set these values inside hive-site.xml and restarted hive service. It should work.
Share your hive-site.xml (please remove values for s3naccess key and secret )
On Tue, Sep 10, 2013 at 3:17 AM, <sh...@accenture.com>> wrote:
Please help. I am badly stuck on this. Please help.
From: Haldar, Shouvanik
Sent: Tuesday, September 10, 2013 1:03 AM
To: 'user@hive.apache.org<ma...@hive.apache.org>'
Subject: RE: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
When I set values inside hive-site.xml & test whether values are set by typing hive> set; I see that the values are set.
Yet, when I run “CREATE EXTERNAL TABLE…” command, I get the same error
FAILED: Error in metadata: MetaException(message:java.lang.IllegalArgumentException: AWS Access Key ID and Secret Access Key must be specified as the username or password (respectively) of a s3n URL, or by setting the fs.s3n.awsAccessKeyId or fs.s3n.awsSecretAccessKey properties (respectively).)
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
Please help.
Thanks,
Shouvanik
From: Haldar, Shouvanik
Sent: Tuesday, September 10, 2013 12:39 AM
To: 'user@hive.apache.org<ma...@hive.apache.org>'
Subject: RE: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
Hi Nitin,
Please help me with the below?
Thanks,
Shouvanik
From: Haldar, Shouvanik
Sent: Saturday, September 07, 2013 2:30 AM
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: RE: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
Hi ,
I tried to set the parameters inside hive-site.xml on the node from where I am running hive. Even “hive>set;” command showed me that the parameters were successfully set. Yet, when running hive “CREATE EXTERNAL TABLE…”, I am getting the following error
FAILED: Error in metadata: MetaException(message:java.lang.IllegalArgumentException: AWS Access Key ID and Secret Access Key must be specified as the username or password (respectively) of a s3n URL, or by setting the fs.s3n.awsAccessKeyId or fs.s3n.awsSecretAccessKey properties (respectively).)
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
Please help.
Thanks,
Shouvanik
From: Nitin Pawar [mailto:nitinpawar432@gmail.com]
Sent: Saturday, September 07, 2013 2:25 AM
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: Re: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
did you try setting them in hive-site.xml file ?
On Sat, Sep 7, 2013 at 2:18 AM, <sh...@accenture.com>> wrote:
Hi,
Can anybody please reply to this below? I am stuck up very bad.
Regards,
Shouvanik
From: Haldar, Shouvanik
Sent: Friday, September 06, 2013 11:22 PM
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: RE: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
Hi,
Is it possible to permanently set the below parameters? Please help.
Thanks,
SHouvanik
From: Jitendra Yadav [mailto:jeetuyadav200890@gmail.com]
Sent: Thursday, September 05, 2013 6:02 PM
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: Re: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
OK got your point, As an alternative you can set these variables in hive environment before creating the table
Hive > set fs.s3n.awsSecretAccessKey='value'
Hive > set fs.s3n.awsAccessKeyId='Value'
Hope this will help you.
Thanks
Jitendra
On Thu, Sep 5, 2013 at 5:19 PM, <sh...@accenture.com>> wrote:
Hi,
Thanks for the reply.
I had seen in some posts, that you could mention these keys on the Hive CLI. If things work, then I am later planning to include it in core-site.xml of Hadoop config. Presently, I am not allowed to restart Hadoop services.
“create external table ….
…..
LOCATION 's3n:// <fs.s3n.awsAccessKeyId: fs.s3n>.<awsSecretAccessKey >@ap-hw-dev/service-names';
Please help me.
Thanks,
Shouvanik
From: Jitendra Yadav [mailto:jeetuyadav200890@gmail.com<ma...@gmail.com>]
Sent: Thursday, September 05, 2013 5:11 PM
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: Re: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
Hi,
Did you configured fs.s3n.awsAccessKeyId or fs.s3n.awsSecretAccessKey properties
in your core-site.xml file?
Regards
Jitendra
On Thu, Sep 5, 2013 at 3:52 PM, <sh...@accenture.com>> wrote:
Hi,
I am executing the example, but got initial roadblock while trying out “Setting up tables (DDL Statements)” section.
When I executed the command “hive> create external table kv (key int, values string) location 's3n://data.s3ndemo.hive/kv';”
I got the error
FAILED: Error in metadata: MetaException(message:java.lang.IllegalArgumentException: AWS Access Key ID and Secret Access Key must be specified as the username or password (respectively) of a s3n URL, or by setting the fs.s3n.awsAccessKeyId or fs.s3n.awsSecretAccessKey properties (respectively).)
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
hive>
Please help.
Thanks,
Shouvanik
________________________________
This message is for the designated recipient only and may contain privileged, proprietary, or otherwise confidential information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of the e-mail by you is prohibited.
Where allowed by local law, electronic communications with Accenture and its affiliates, including e-mail and instant messaging (including content), may be scanned by our systems for the purposes of information security and assessment of internal compliance with Accenture policy.
______________________________________________________________________________________
www.accenture.com<http://www.accenture.com/>
--
Nitin Pawar
--
Nitin Pawar
--
Nitin Pawar
--
Nitin Pawar
--
Nitin Pawar
RE: error executing the tutorial ->
https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
Posted by sh...@accenture.com.
No. File system will be s3n, or native file system or hadoop. But using HIVE we are storing data that is already existing inside s3
From: Nitin Pawar [mailto:nitinpawar432@gmail.com]
Sent: Tuesday, September 10, 2013 4:56 PM
To: user@hive.apache.org
Subject: Re: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
Ok, so back to basics.
correct me If I am wrong
You are trying to setup YARN using s3 as file system?
Then you want to use hive on top of it.
On Tue, Sep 10, 2013 at 4:48 PM, <sh...@accenture.com>> wrote:
The issue is if I paste it in core-site.xml of Hadoop on the node in question, then will starting datanode would do only? Actually,. I am a bit confused what all daemons to start. I have a YARN architecture running 1 separate - Resource Manager, 1 NameNode & 4 datanodes.
From: Nitin Pawar [mailto:nitinpawar432@gmail.com<ma...@gmail.com>]
Sent: Tuesday, September 10, 2013 4:43 PM
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: Re: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
There is not more to start. If hive is not starting properly, it will not start at all so you are doing ok.
Problem is when you create table, hive tries to create a directory with that name if its not existing. Just try to put those two settings inside core-site.xml and see if the hdfs can use it to create directory
On Tue, Sep 10, 2013 at 4:06 PM, <sh...@accenture.com>> wrote:
The values are showing when I run "hive> set;" command. But it's not getting identified. I guess I am not starting HIVE properly. Do you know what are the steps?
From: Nitin Pawar [mailto:nitinpawar432@gmail.com<ma...@gmail.com>]
Sent: Tuesday, September 10, 2013 4:04 PM
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: Re: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
>From the config file it looks ok. Not really sure why it is not picking up when you run it via hive cli.
Can you try putting these in core-site.xml of hdfs.
<property>
<name>fs.s3n.awsAccessKeyId</name>
<value>ID</value>
</property>
<property>
<name>fs.s3n.awsSecretAccessKey</name>
<value>SECRET</value>
</property>
On Tue, Sep 10, 2013 at 3:11 PM, <sh...@accenture.com>> wrote:
Hi Nitin,
PFA hive-site.xml. I wanted to start Hortonworks HIVE. Can you please help.
Thanks,
Shouvanik
From: Nitin Pawar [mailto:nitinpawar432@gmail.com<ma...@gmail.com>]
Sent: Tuesday, September 10, 2013 11:15 AM
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: Re: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
if you have set these values inside hive-site.xml and restarted hive service. It should work.
Share your hive-site.xml (please remove values for s3naccess key and secret )
On Tue, Sep 10, 2013 at 3:17 AM, <sh...@accenture.com>> wrote:
Please help. I am badly stuck on this. Please help.
From: Haldar, Shouvanik
Sent: Tuesday, September 10, 2013 1:03 AM
To: 'user@hive.apache.org<ma...@hive.apache.org>'
Subject: RE: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
When I set values inside hive-site.xml & test whether values are set by typing hive> set; I see that the values are set.
Yet, when I run "CREATE EXTERNAL TABLE..." command, I get the same error
FAILED: Error in metadata: MetaException(message:java.lang.IllegalArgumentException: AWS Access Key ID and Secret Access Key must be specified as the username or password (respectively) of a s3n URL, or by setting the fs.s3n.awsAccessKeyId or fs.s3n.awsSecretAccessKey properties (respectively).)
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
Please help.
Thanks,
Shouvanik
From: Haldar, Shouvanik
Sent: Tuesday, September 10, 2013 12:39 AM
To: 'user@hive.apache.org<ma...@hive.apache.org>'
Subject: RE: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
Hi Nitin,
Please help me with the below?
Thanks,
Shouvanik
From: Haldar, Shouvanik
Sent: Saturday, September 07, 2013 2:30 AM
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: RE: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
Hi ,
I tried to set the parameters inside hive-site.xml on the node from where I am running hive. Even "hive>set;" command showed me that the parameters were successfully set. Yet, when running hive "CREATE EXTERNAL TABLE...", I am getting the following error
FAILED: Error in metadata: MetaException(message:java.lang.IllegalArgumentException: AWS Access Key ID and Secret Access Key must be specified as the username or password (respectively) of a s3n URL, or by setting the fs.s3n.awsAccessKeyId or fs.s3n.awsSecretAccessKey properties (respectively).)
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
Please help.
Thanks,
Shouvanik
From: Nitin Pawar [mailto:nitinpawar432@gmail.com]
Sent: Saturday, September 07, 2013 2:25 AM
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: Re: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
did you try setting them in hive-site.xml file ?
On Sat, Sep 7, 2013 at 2:18 AM, <sh...@accenture.com>> wrote:
Hi,
Can anybody please reply to this below? I am stuck up very bad.
Regards,
Shouvanik
From: Haldar, Shouvanik
Sent: Friday, September 06, 2013 11:22 PM
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: RE: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
Hi,
Is it possible to permanently set the below parameters? Please help.
Thanks,
SHouvanik
From: Jitendra Yadav [mailto:jeetuyadav200890@gmail.com]
Sent: Thursday, September 05, 2013 6:02 PM
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: Re: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
OK got your point, As an alternative you can set these variables in hive environment before creating the table
Hive > set fs.s3n.awsSecretAccessKey='value'
Hive > set fs.s3n.awsAccessKeyId='Value'
Hope this will help you.
Thanks
Jitendra
On Thu, Sep 5, 2013 at 5:19 PM, <sh...@accenture.com>> wrote:
Hi,
Thanks for the reply.
I had seen in some posts, that you could mention these keys on the Hive CLI. If things work, then I am later planning to include it in core-site.xml of Hadoop config. Presently, I am not allowed to restart Hadoop services.
"create external table ....
.....
LOCATION 's3n:// <fs.s3n.awsAccessKeyId: fs.s3n>.<awsSecretAccessKey >@ap-hw-dev/service-names';
Please help me.
Thanks,
Shouvanik
From: Jitendra Yadav [mailto:jeetuyadav200890@gmail.com<ma...@gmail.com>]
Sent: Thursday, September 05, 2013 5:11 PM
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: Re: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
Hi,
Did you configured fs.s3n.awsAccessKeyId or fs.s3n.awsSecretAccessKey properties
in your core-site.xml file?
Regards
Jitendra
On Thu, Sep 5, 2013 at 3:52 PM, <sh...@accenture.com>> wrote:
Hi,
I am executing the example, but got initial roadblock while trying out "Setting up tables (DDL Statements)" section.
When I executed the command "hive> create external table kv (key int, values string) location 's3n://data.s3ndemo.hive/kv';"
I got the error
FAILED: Error in metadata: MetaException(message:java.lang.IllegalArgumentException: AWS Access Key ID and Secret Access Key must be specified as the username or password (respectively) of a s3n URL, or by setting the fs.s3n.awsAccessKeyId or fs.s3n.awsSecretAccessKey properties (respectively).)
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
hive>
Please help.
Thanks,
Shouvanik
________________________________
This message is for the designated recipient only and may contain privileged, proprietary, or otherwise confidential information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of the e-mail by you is prohibited.
Where allowed by local law, electronic communications with Accenture and its affiliates, including e-mail and instant messaging (including content), may be scanned by our systems for the purposes of information security and assessment of internal compliance with Accenture policy.
______________________________________________________________________________________
www.accenture.com<http://www.accenture.com/>
--
Nitin Pawar
--
Nitin Pawar
--
Nitin Pawar
--
Nitin Pawar
--
Nitin Pawar
Re: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
Posted by Nitin Pawar <ni...@gmail.com>.
Ok, so back to basics.
correct me If I am wrong
You are trying to setup YARN using s3 as file system?
Then you want to use hive on top of it.
On Tue, Sep 10, 2013 at 4:48 PM, <sh...@accenture.com> wrote:
> The issue is if I paste it in core-site.xml of Hadoop on the node in
> question, then will starting datanode would do only? Actually,. I am a bit
> confused what all daemons to start. I have a YARN architecture running 1
> separate – Resource Manager, 1 NameNode & 4 datanodes.****
>
> ** **
>
> *From:* Nitin Pawar [mailto:nitinpawar432@gmail.com]
> *Sent:* Tuesday, September 10, 2013 4:43 PM
>
> *To:* user@hive.apache.org
> *Subject:* Re: error executing the tutorial ->
> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
> ****
>
> ** **
>
> There is not more to start. If hive is not starting properly, it will not
> start at all so you are doing ok. ****
>
> ** **
>
> Problem is when you create table, hive tries to create a directory with
> that name if its not existing. Just try to put those two settings inside
> core-site.xml and see if the hdfs can use it to create directory ****
>
> ** **
>
> On Tue, Sep 10, 2013 at 4:06 PM, <sh...@accenture.com> wrote:**
> **
>
> The values are showing when I run “hive> set;” command. But it’s not
> getting identified. I guess I am not starting HIVE properly. Do you know
> what are the steps?****
>
> ****
>
> *From:* Nitin Pawar [mailto:nitinpawar432@gmail.com]
> *Sent:* Tuesday, September 10, 2013 4:04 PM****
>
>
> *To:* user@hive.apache.org
> *Subject:* Re: error executing the tutorial ->
> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
> ****
>
> ****
>
> From the config file it looks ok. Not really sure why it is not picking up
> when you run it via hive cli. ****
>
> ****
>
> Can you try putting these in core-site.xml of hdfs. ****
>
> ****
>
> ****
>
> <property>****
>
> <name>fs.s3n.awsAccessKeyId</name>****
>
> <value>ID</value>****
>
> </property>****
>
> <property>****
>
> <name>fs.s3n.awsSecretAccessKey</name>****
>
> <value>SECRET</value>****
>
> </property>****
>
> ****
>
> On Tue, Sep 10, 2013 at 3:11 PM, <sh...@accenture.com> wrote:**
> **
>
> Hi Nitin,****
>
> ****
>
> PFA hive-site.xml. I wanted to start Hortonworks HIVE. Can you please help.
> ****
>
> ****
>
> Thanks,****
>
> Shouvanik****
>
> ****
>
> *From:* Nitin Pawar [mailto:nitinpawar432@gmail.com]
> *Sent:* Tuesday, September 10, 2013 11:15 AM****
>
>
> *To:* user@hive.apache.org
> *Subject:* Re: error executing the tutorial ->
> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
> ****
>
> ****
>
> if you have set these values inside hive-site.xml and restarted hive
> service. It should work. ****
>
> ****
>
> Share your hive-site.xml (please remove values for s3naccess key and
> secret ) ****
>
> ****
>
> On Tue, Sep 10, 2013 at 3:17 AM, <sh...@accenture.com> wrote:**
> **
>
> Please help. I am badly stuck on this. Please help.****
>
> ****
>
> *From:* Haldar, Shouvanik
> *Sent:* Tuesday, September 10, 2013 1:03 AM****
>
>
> *To:* 'user@hive.apache.org'
> *Subject:* RE: error executing the tutorial ->
> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
> ****
>
> ****
>
> When I set values inside hive-site.xml & test whether values are set by
> typing hive> set; I see that the values are set.****
>
> ****
>
> Yet, when I run “CREATE EXTERNAL TABLE…” command, I get the same error****
>
> ****
>
> FAILED: Error in metadata:
> MetaException(message:java.lang.IllegalArgumentException: AWS Access Key ID
> and Secret Access Key must be specified as the username or password
> (respectively) of a s3n URL, or by setting the fs.s3n.awsAccessKeyId or
> fs.s3n.awsSecretAccessKey properties (respectively).)****
>
> FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.DDLTask****
>
> ****
>
> Please help.****
>
> ****
>
> Thanks,****
>
> Shouvanik****
>
> ****
>
> *From:* Haldar, Shouvanik ****
>
> *Sent:* Tuesday, September 10, 2013 12:39 AM****
>
> *To:* 'user@hive.apache.org'
> *Subject:* RE: error executing the tutorial ->
> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
> ****
>
> ****
>
> Hi Nitin,****
>
> ****
>
> Please help me with the below?****
>
> ****
>
> Thanks,****
>
> Shouvanik****
>
> ****
>
> *From:* Haldar, Shouvanik ****
>
> *Sent:* Saturday, September 07, 2013 2:30 AM****
>
> *To:* user@hive.apache.org
> *Subject:* RE: error executing the tutorial ->
> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
> ****
>
> ****
>
> Hi ,****
>
> ****
>
> I tried to set the parameters inside hive-site.xml on the node from where
> I am running hive. Even “hive>set;” command showed me that the parameters
> were successfully set. Yet, when running hive “CREATE EXTERNAL TABLE…”, I
> am getting the following error****
>
> ****
>
> FAILED: Error in metadata:
> MetaException(message:java.lang.IllegalArgumentException: AWS Access Key ID
> and Secret Access Key must be specified as the username or password
> (respectively) of a s3n URL, or by setting the fs.s3n.awsAccessKeyId or
> fs.s3n.awsSecretAccessKey properties (respectively).)****
>
> FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.DDLTask ****
>
> ****
>
> ****
>
> Please help.****
>
> ****
>
> Thanks,****
>
> Shouvanik****
>
> ****
>
> *From:* Nitin Pawar [mailto:nitinpawar432@gmail.com<ni...@gmail.com>]
> ****
>
> *Sent:* Saturday, September 07, 2013 2:25 AM****
>
> *To:* user@hive.apache.org
> *Subject:* Re: error executing the tutorial ->
> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
> ****
>
> ****
>
> did you try setting them in hive-site.xml file ? ****
>
> ****
>
> On Sat, Sep 7, 2013 at 2:18 AM, <sh...@accenture.com> wrote:***
> *
>
> Hi,****
>
> ****
>
> Can anybody please reply to this below? I am stuck up very bad.****
>
> ****
>
> Regards,****
>
> Shouvanik****
>
> ****
>
> *From:* Haldar, Shouvanik
> *Sent:* Friday, September 06, 2013 11:22 PM
> *To:* user@hive.apache.org
> *Subject:* RE: error executing the tutorial ->
> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
> ****
>
> ****
>
> Hi,****
>
> ****
>
> Is it possible to permanently set the below parameters? Please help.****
>
> ****
>
> Thanks,****
>
> SHouvanik****
>
> ****
>
> *From:* Jitendra Yadav [mailto:jeetuyadav200890@gmail.com<je...@gmail.com>]
>
> *Sent:* Thursday, September 05, 2013 6:02 PM****
>
>
> *To:* user@hive.apache.org
> *Subject:* Re: error executing the tutorial ->
> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
> ****
>
> ****
>
> OK got your point, As an alternative you can set these variables in hive
> environment before creating the table****
>
> ****
>
> Hive > set fs.s3n.awsSecretAccessKey='value'
> Hive > set fs.s3n.awsAccessKeyId='Value'****
>
> ****
>
> Hope this will help you.****
>
> ****
>
> Thanks****
>
> Jitendra
> On Thu, Sep 5, 2013 at 5:19 PM, <sh...@accenture.com> wrote:***
> *
>
> Hi,****
>
> ****
>
> Thanks for the reply.****
>
> ****
>
> I had seen in some posts, that you could mention these keys on the Hive
> CLI. If things work, then I am later planning to include it in
> core-site.xml of Hadoop config. Presently, I am not allowed to restart
> Hadoop services. ****
>
> ****
>
> “create external table ….****
>
> …..****
>
> LOCATION 's3n:// <fs.s3n.awsAccessKeyId: fs.s3n>.<awsSecretAccessKey>@ap-hw-dev/service-names';
> ****
>
> ****
>
> Please help me.****
>
> ****
>
> Thanks,****
>
> Shouvanik****
>
> ****
>
> *From:* Jitendra Yadav [mailto:jeetuyadav200890@gmail.com]
> *Sent:* Thursday, September 05, 2013 5:11 PM
> *To:* user@hive.apache.org
> *Subject:* Re: error executing the tutorial ->
> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
> ****
>
> ****
>
> Hi,****
>
> ****
>
> Did you configured fs.s3n.awsAccessKeyId or fs.s3n.awsSecretAccessKey
> properties****
>
> in your core-site.xml file?****
>
> ****
>
> Regards****
>
> Jitendra****
>
> On Thu, Sep 5, 2013 at 3:52 PM, <sh...@accenture.com> wrote:***
> *
>
> Hi,****
>
> ****
>
> I am executing the example, but got initial roadblock while trying out
> “Setting up tables (DDL Statements)” section.****
>
> ****
>
> When I executed the command “hive> create external table kv (key int,
> values string) location 's3n://data.s3ndemo.hive/kv';”****
>
> ****
>
> I got the error****
>
> ****
>
> FAILED: Error in metadata:
> MetaException(message:java.lang.IllegalArgumentException: AWS Access Key ID
> and Secret Access Key must be specified as the username or password
> (respectively) of a s3n URL, or by setting the fs.s3n.awsAccessKeyId or
> fs.s3n.awsSecretAccessKey properties (respectively).)****
>
> FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.DDLTask****
>
> hive>****
>
> ****
>
> Please help.****
>
> ****
>
> ****
>
> Thanks,****
>
> Shouvanik****
>
> ****
> ------------------------------
>
> This message is for the designated recipient only and may contain
> privileged, proprietary, or otherwise confidential information. If you have
> received it in error, please notify the sender immediately and delete the
> original. Any other use of the e-mail by you is prohibited.
>
> Where allowed by local law, electronic communications with Accenture and
> its affiliates, including e-mail and instant messaging (including content),
> may be scanned by our systems for the purposes of information security and
> assessment of internal compliance with Accenture policy.
>
>
> ______________________________________________________________________________________
>
> www.accenture.com****
>
> ****
>
> ****
>
>
>
> ****
>
> ****
>
> --
> Nitin Pawar****
>
>
>
> ****
>
> ****
>
> --
> Nitin Pawar****
>
>
>
> ****
>
> ****
>
> --
> Nitin Pawar****
>
>
>
> ****
>
> ** **
>
> --
> Nitin Pawar****
>
--
Nitin Pawar
RE: error executing the tutorial ->
https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
Posted by sh...@accenture.com.
The issue is if I paste it in core-site.xml of Hadoop on the node in question, then will starting datanode would do only? Actually,. I am a bit confused what all daemons to start. I have a YARN architecture running 1 separate - Resource Manager, 1 NameNode & 4 datanodes.
From: Nitin Pawar [mailto:nitinpawar432@gmail.com]
Sent: Tuesday, September 10, 2013 4:43 PM
To: user@hive.apache.org
Subject: Re: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
There is not more to start. If hive is not starting properly, it will not start at all so you are doing ok.
Problem is when you create table, hive tries to create a directory with that name if its not existing. Just try to put those two settings inside core-site.xml and see if the hdfs can use it to create directory
On Tue, Sep 10, 2013 at 4:06 PM, <sh...@accenture.com>> wrote:
The values are showing when I run "hive> set;" command. But it's not getting identified. I guess I am not starting HIVE properly. Do you know what are the steps?
From: Nitin Pawar [mailto:nitinpawar432@gmail.com<ma...@gmail.com>]
Sent: Tuesday, September 10, 2013 4:04 PM
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: Re: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
>From the config file it looks ok. Not really sure why it is not picking up when you run it via hive cli.
Can you try putting these in core-site.xml of hdfs.
<property>
<name>fs.s3n.awsAccessKeyId</name>
<value>ID</value>
</property>
<property>
<name>fs.s3n.awsSecretAccessKey</name>
<value>SECRET</value>
</property>
On Tue, Sep 10, 2013 at 3:11 PM, <sh...@accenture.com>> wrote:
Hi Nitin,
PFA hive-site.xml. I wanted to start Hortonworks HIVE. Can you please help.
Thanks,
Shouvanik
From: Nitin Pawar [mailto:nitinpawar432@gmail.com<ma...@gmail.com>]
Sent: Tuesday, September 10, 2013 11:15 AM
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: Re: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
if you have set these values inside hive-site.xml and restarted hive service. It should work.
Share your hive-site.xml (please remove values for s3naccess key and secret )
On Tue, Sep 10, 2013 at 3:17 AM, <sh...@accenture.com>> wrote:
Please help. I am badly stuck on this. Please help.
From: Haldar, Shouvanik
Sent: Tuesday, September 10, 2013 1:03 AM
To: 'user@hive.apache.org<ma...@hive.apache.org>'
Subject: RE: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
When I set values inside hive-site.xml & test whether values are set by typing hive> set; I see that the values are set.
Yet, when I run "CREATE EXTERNAL TABLE..." command, I get the same error
FAILED: Error in metadata: MetaException(message:java.lang.IllegalArgumentException: AWS Access Key ID and Secret Access Key must be specified as the username or password (respectively) of a s3n URL, or by setting the fs.s3n.awsAccessKeyId or fs.s3n.awsSecretAccessKey properties (respectively).)
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
Please help.
Thanks,
Shouvanik
From: Haldar, Shouvanik
Sent: Tuesday, September 10, 2013 12:39 AM
To: 'user@hive.apache.org<ma...@hive.apache.org>'
Subject: RE: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
Hi Nitin,
Please help me with the below?
Thanks,
Shouvanik
From: Haldar, Shouvanik
Sent: Saturday, September 07, 2013 2:30 AM
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: RE: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
Hi ,
I tried to set the parameters inside hive-site.xml on the node from where I am running hive. Even "hive>set;" command showed me that the parameters were successfully set. Yet, when running hive "CREATE EXTERNAL TABLE...", I am getting the following error
FAILED: Error in metadata: MetaException(message:java.lang.IllegalArgumentException: AWS Access Key ID and Secret Access Key must be specified as the username or password (respectively) of a s3n URL, or by setting the fs.s3n.awsAccessKeyId or fs.s3n.awsSecretAccessKey properties (respectively).)
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
Please help.
Thanks,
Shouvanik
From: Nitin Pawar [mailto:nitinpawar432@gmail.com]
Sent: Saturday, September 07, 2013 2:25 AM
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: Re: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
did you try setting them in hive-site.xml file ?
On Sat, Sep 7, 2013 at 2:18 AM, <sh...@accenture.com>> wrote:
Hi,
Can anybody please reply to this below? I am stuck up very bad.
Regards,
Shouvanik
From: Haldar, Shouvanik
Sent: Friday, September 06, 2013 11:22 PM
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: RE: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
Hi,
Is it possible to permanently set the below parameters? Please help.
Thanks,
SHouvanik
From: Jitendra Yadav [mailto:jeetuyadav200890@gmail.com]
Sent: Thursday, September 05, 2013 6:02 PM
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: Re: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
OK got your point, As an alternative you can set these variables in hive environment before creating the table
Hive > set fs.s3n.awsSecretAccessKey='value'
Hive > set fs.s3n.awsAccessKeyId='Value'
Hope this will help you.
Thanks
Jitendra
On Thu, Sep 5, 2013 at 5:19 PM, <sh...@accenture.com>> wrote:
Hi,
Thanks for the reply.
I had seen in some posts, that you could mention these keys on the Hive CLI. If things work, then I am later planning to include it in core-site.xml of Hadoop config. Presently, I am not allowed to restart Hadoop services.
"create external table ....
.....
LOCATION 's3n:// <fs.s3n.awsAccessKeyId: fs.s3n>.<awsSecretAccessKey >@ap-hw-dev/service-names';
Please help me.
Thanks,
Shouvanik
From: Jitendra Yadav [mailto:jeetuyadav200890@gmail.com<ma...@gmail.com>]
Sent: Thursday, September 05, 2013 5:11 PM
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: Re: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
Hi,
Did you configured fs.s3n.awsAccessKeyId or fs.s3n.awsSecretAccessKey properties
in your core-site.xml file?
Regards
Jitendra
On Thu, Sep 5, 2013 at 3:52 PM, <sh...@accenture.com>> wrote:
Hi,
I am executing the example, but got initial roadblock while trying out "Setting up tables (DDL Statements)" section.
When I executed the command "hive> create external table kv (key int, values string) location 's3n://data.s3ndemo.hive/kv';"
I got the error
FAILED: Error in metadata: MetaException(message:java.lang.IllegalArgumentException: AWS Access Key ID and Secret Access Key must be specified as the username or password (respectively) of a s3n URL, or by setting the fs.s3n.awsAccessKeyId or fs.s3n.awsSecretAccessKey properties (respectively).)
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
hive>
Please help.
Thanks,
Shouvanik
________________________________
This message is for the designated recipient only and may contain privileged, proprietary, or otherwise confidential information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of the e-mail by you is prohibited.
Where allowed by local law, electronic communications with Accenture and its affiliates, including e-mail and instant messaging (including content), may be scanned by our systems for the purposes of information security and assessment of internal compliance with Accenture policy.
______________________________________________________________________________________
www.accenture.com<http://www.accenture.com/>
--
Nitin Pawar
--
Nitin Pawar
--
Nitin Pawar
--
Nitin Pawar
Re: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
Posted by Nitin Pawar <ni...@gmail.com>.
There is not more to start. If hive is not starting properly, it will not
start at all so you are doing ok.
Problem is when you create table, hive tries to create a directory with
that name if its not existing. Just try to put those two settings inside
core-site.xml and see if the hdfs can use it to create directory
On Tue, Sep 10, 2013 at 4:06 PM, <sh...@accenture.com> wrote:
> The values are showing when I run “hive> set;” command. But it’s not
> getting identified. I guess I am not starting HIVE properly. Do you know
> what are the steps?****
>
> ** **
>
> *From:* Nitin Pawar [mailto:nitinpawar432@gmail.com]
> *Sent:* Tuesday, September 10, 2013 4:04 PM
>
> *To:* user@hive.apache.org
> *Subject:* Re: error executing the tutorial ->
> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
> ****
>
> ** **
>
> From the config file it looks ok. Not really sure why it is not picking up
> when you run it via hive cli. ****
>
> ** **
>
> Can you try putting these in core-site.xml of hdfs. ****
>
> ** **
>
> ** **
>
> <property>****
>
> <name>fs.s3n.awsAccessKeyId</name>****
>
> <value>ID</value>****
>
> </property>****
>
> <property>****
>
> <name>fs.s3n.awsSecretAccessKey</name>****
>
> <value>SECRET</value>****
>
> </property>****
>
> ** **
>
> On Tue, Sep 10, 2013 at 3:11 PM, <sh...@accenture.com> wrote:**
> **
>
> Hi Nitin,****
>
> ****
>
> PFA hive-site.xml. I wanted to start Hortonworks HIVE. Can you please help.
> ****
>
> ****
>
> Thanks,****
>
> Shouvanik****
>
> ****
>
> *From:* Nitin Pawar [mailto:nitinpawar432@gmail.com]
> *Sent:* Tuesday, September 10, 2013 11:15 AM****
>
>
> *To:* user@hive.apache.org
> *Subject:* Re: error executing the tutorial ->
> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
> ****
>
> ****
>
> if you have set these values inside hive-site.xml and restarted hive
> service. It should work. ****
>
> ****
>
> Share your hive-site.xml (please remove values for s3naccess key and
> secret ) ****
>
> ****
>
> On Tue, Sep 10, 2013 at 3:17 AM, <sh...@accenture.com> wrote:**
> **
>
> Please help. I am badly stuck on this. Please help.****
>
> ****
>
> *From:* Haldar, Shouvanik
> *Sent:* Tuesday, September 10, 2013 1:03 AM****
>
>
> *To:* 'user@hive.apache.org'
> *Subject:* RE: error executing the tutorial ->
> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
> ****
>
> ****
>
> When I set values inside hive-site.xml & test whether values are set by
> typing hive> set; I see that the values are set.****
>
> ****
>
> Yet, when I run “CREATE EXTERNAL TABLE…” command, I get the same error****
>
> ****
>
> FAILED: Error in metadata:
> MetaException(message:java.lang.IllegalArgumentException: AWS Access Key ID
> and Secret Access Key must be specified as the username or password
> (respectively) of a s3n URL, or by setting the fs.s3n.awsAccessKeyId or
> fs.s3n.awsSecretAccessKey properties (respectively).)****
>
> FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.DDLTask****
>
> ****
>
> Please help.****
>
> ****
>
> Thanks,****
>
> Shouvanik****
>
> ****
>
> *From:* Haldar, Shouvanik ****
>
> *Sent:* Tuesday, September 10, 2013 12:39 AM****
>
> *To:* 'user@hive.apache.org'
> *Subject:* RE: error executing the tutorial ->
> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
> ****
>
> ****
>
> Hi Nitin,****
>
> ****
>
> Please help me with the below?****
>
> ****
>
> Thanks,****
>
> Shouvanik****
>
> ****
>
> *From:* Haldar, Shouvanik ****
>
> *Sent:* Saturday, September 07, 2013 2:30 AM****
>
> *To:* user@hive.apache.org
> *Subject:* RE: error executing the tutorial ->
> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
> ****
>
> ****
>
> Hi ,****
>
> ****
>
> I tried to set the parameters inside hive-site.xml on the node from where
> I am running hive. Even “hive>set;” command showed me that the parameters
> were successfully set. Yet, when running hive “CREATE EXTERNAL TABLE…”, I
> am getting the following error****
>
> ****
>
> FAILED: Error in metadata:
> MetaException(message:java.lang.IllegalArgumentException: AWS Access Key ID
> and Secret Access Key must be specified as the username or password
> (respectively) of a s3n URL, or by setting the fs.s3n.awsAccessKeyId or
> fs.s3n.awsSecretAccessKey properties (respectively).)****
>
> FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.DDLTask ****
>
> ****
>
> ****
>
> Please help.****
>
> ****
>
> Thanks,****
>
> Shouvanik****
>
> ****
>
> *From:* Nitin Pawar [mailto:nitinpawar432@gmail.com<ni...@gmail.com>]
> ****
>
> *Sent:* Saturday, September 07, 2013 2:25 AM****
>
> *To:* user@hive.apache.org
> *Subject:* Re: error executing the tutorial ->
> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
> ****
>
> ****
>
> did you try setting them in hive-site.xml file ? ****
>
> ****
>
> On Sat, Sep 7, 2013 at 2:18 AM, <sh...@accenture.com> wrote:***
> *
>
> Hi,****
>
> ****
>
> Can anybody please reply to this below? I am stuck up very bad.****
>
> ****
>
> Regards,****
>
> Shouvanik****
>
> ****
>
> *From:* Haldar, Shouvanik
> *Sent:* Friday, September 06, 2013 11:22 PM
> *To:* user@hive.apache.org
> *Subject:* RE: error executing the tutorial ->
> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
> ****
>
> ****
>
> Hi,****
>
> ****
>
> Is it possible to permanently set the below parameters? Please help.****
>
> ****
>
> Thanks,****
>
> SHouvanik****
>
> ****
>
> *From:* Jitendra Yadav [mailto:jeetuyadav200890@gmail.com<je...@gmail.com>]
>
> *Sent:* Thursday, September 05, 2013 6:02 PM****
>
>
> *To:* user@hive.apache.org
> *Subject:* Re: error executing the tutorial ->
> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
> ****
>
> ****
>
> OK got your point, As an alternative you can set these variables in hive
> environment before creating the table****
>
> ****
>
> Hive > set fs.s3n.awsSecretAccessKey='value'
> Hive > set fs.s3n.awsAccessKeyId='Value'****
>
> ****
>
> Hope this will help you.****
>
> ****
>
> Thanks****
>
> Jitendra
> On Thu, Sep 5, 2013 at 5:19 PM, <sh...@accenture.com> wrote:***
> *
>
> Hi,****
>
> ****
>
> Thanks for the reply.****
>
> ****
>
> I had seen in some posts, that you could mention these keys on the Hive
> CLI. If things work, then I am later planning to include it in
> core-site.xml of Hadoop config. Presently, I am not allowed to restart
> Hadoop services. ****
>
> ****
>
> “create external table ….****
>
> …..****
>
> LOCATION 's3n:// <fs.s3n.awsAccessKeyId: fs.s3n>.<awsSecretAccessKey>@ap-hw-dev/service-names';
> ****
>
> ****
>
> Please help me.****
>
> ****
>
> Thanks,****
>
> Shouvanik****
>
> ****
>
> *From:* Jitendra Yadav [mailto:jeetuyadav200890@gmail.com]
> *Sent:* Thursday, September 05, 2013 5:11 PM
> *To:* user@hive.apache.org
> *Subject:* Re: error executing the tutorial ->
> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
> ****
>
> ****
>
> Hi,****
>
> ****
>
> Did you configured fs.s3n.awsAccessKeyId or fs.s3n.awsSecretAccessKey
> properties****
>
> in your core-site.xml file?****
>
> ****
>
> Regards****
>
> Jitendra****
>
> On Thu, Sep 5, 2013 at 3:52 PM, <sh...@accenture.com> wrote:***
> *
>
> Hi,****
>
> ****
>
> I am executing the example, but got initial roadblock while trying out
> “Setting up tables (DDL Statements)” section.****
>
> ****
>
> When I executed the command “hive> create external table kv (key int,
> values string) location 's3n://data.s3ndemo.hive/kv';”****
>
> ****
>
> I got the error****
>
> ****
>
> FAILED: Error in metadata:
> MetaException(message:java.lang.IllegalArgumentException: AWS Access Key ID
> and Secret Access Key must be specified as the username or password
> (respectively) of a s3n URL, or by setting the fs.s3n.awsAccessKeyId or
> fs.s3n.awsSecretAccessKey properties (respectively).)****
>
> FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.DDLTask****
>
> hive>****
>
> ****
>
> Please help.****
>
> ****
>
> ****
>
> Thanks,****
>
> Shouvanik****
>
> ****
> ------------------------------
>
> This message is for the designated recipient only and may contain
> privileged, proprietary, or otherwise confidential information. If you have
> received it in error, please notify the sender immediately and delete the
> original. Any other use of the e-mail by you is prohibited.
>
> Where allowed by local law, electronic communications with Accenture and
> its affiliates, including e-mail and instant messaging (including content),
> may be scanned by our systems for the purposes of information security and
> assessment of internal compliance with Accenture policy.
>
>
> ______________________________________________________________________________________
>
> www.accenture.com****
>
> ****
>
> ****
>
>
>
> ****
>
> ****
>
> --
> Nitin Pawar****
>
>
>
> ****
>
> ****
>
> --
> Nitin Pawar****
>
>
>
> ****
>
> ** **
>
> --
> Nitin Pawar****
>
--
Nitin Pawar
RE: error executing the tutorial ->
https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
Posted by sh...@accenture.com.
The values are showing when I run "hive> set;" command. But it's not getting identified. I guess I am not starting HIVE properly. Do you know what are the steps?
From: Nitin Pawar [mailto:nitinpawar432@gmail.com]
Sent: Tuesday, September 10, 2013 4:04 PM
To: user@hive.apache.org
Subject: Re: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
>From the config file it looks ok. Not really sure why it is not picking up when you run it via hive cli.
Can you try putting these in core-site.xml of hdfs.
<property>
<name>fs.s3n.awsAccessKeyId</name>
<value>ID</value>
</property>
<property>
<name>fs.s3n.awsSecretAccessKey</name>
<value>SECRET</value>
</property>
On Tue, Sep 10, 2013 at 3:11 PM, <sh...@accenture.com>> wrote:
Hi Nitin,
PFA hive-site.xml. I wanted to start Hortonworks HIVE. Can you please help.
Thanks,
Shouvanik
From: Nitin Pawar [mailto:nitinpawar432@gmail.com<ma...@gmail.com>]
Sent: Tuesday, September 10, 2013 11:15 AM
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: Re: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
if you have set these values inside hive-site.xml and restarted hive service. It should work.
Share your hive-site.xml (please remove values for s3naccess key and secret )
On Tue, Sep 10, 2013 at 3:17 AM, <sh...@accenture.com>> wrote:
Please help. I am badly stuck on this. Please help.
From: Haldar, Shouvanik
Sent: Tuesday, September 10, 2013 1:03 AM
To: 'user@hive.apache.org<ma...@hive.apache.org>'
Subject: RE: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
When I set values inside hive-site.xml & test whether values are set by typing hive> set; I see that the values are set.
Yet, when I run "CREATE EXTERNAL TABLE..." command, I get the same error
FAILED: Error in metadata: MetaException(message:java.lang.IllegalArgumentException: AWS Access Key ID and Secret Access Key must be specified as the username or password (respectively) of a s3n URL, or by setting the fs.s3n.awsAccessKeyId or fs.s3n.awsSecretAccessKey properties (respectively).)
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
Please help.
Thanks,
Shouvanik
From: Haldar, Shouvanik
Sent: Tuesday, September 10, 2013 12:39 AM
To: 'user@hive.apache.org<ma...@hive.apache.org>'
Subject: RE: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
Hi Nitin,
Please help me with the below?
Thanks,
Shouvanik
From: Haldar, Shouvanik
Sent: Saturday, September 07, 2013 2:30 AM
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: RE: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
Hi ,
I tried to set the parameters inside hive-site.xml on the node from where I am running hive. Even "hive>set;" command showed me that the parameters were successfully set. Yet, when running hive "CREATE EXTERNAL TABLE...", I am getting the following error
FAILED: Error in metadata: MetaException(message:java.lang.IllegalArgumentException: AWS Access Key ID and Secret Access Key must be specified as the username or password (respectively) of a s3n URL, or by setting the fs.s3n.awsAccessKeyId or fs.s3n.awsSecretAccessKey properties (respectively).)
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
Please help.
Thanks,
Shouvanik
From: Nitin Pawar [mailto:nitinpawar432@gmail.com]
Sent: Saturday, September 07, 2013 2:25 AM
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: Re: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
did you try setting them in hive-site.xml file ?
On Sat, Sep 7, 2013 at 2:18 AM, <sh...@accenture.com>> wrote:
Hi,
Can anybody please reply to this below? I am stuck up very bad.
Regards,
Shouvanik
From: Haldar, Shouvanik
Sent: Friday, September 06, 2013 11:22 PM
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: RE: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
Hi,
Is it possible to permanently set the below parameters? Please help.
Thanks,
SHouvanik
From: Jitendra Yadav [mailto:jeetuyadav200890@gmail.com]
Sent: Thursday, September 05, 2013 6:02 PM
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: Re: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
OK got your point, As an alternative you can set these variables in hive environment before creating the table
Hive > set fs.s3n.awsSecretAccessKey='value'
Hive > set fs.s3n.awsAccessKeyId='Value'
Hope this will help you.
Thanks
Jitendra
On Thu, Sep 5, 2013 at 5:19 PM, <sh...@accenture.com>> wrote:
Hi,
Thanks for the reply.
I had seen in some posts, that you could mention these keys on the Hive CLI. If things work, then I am later planning to include it in core-site.xml of Hadoop config. Presently, I am not allowed to restart Hadoop services.
"create external table ....
.....
LOCATION 's3n:// <fs.s3n.awsAccessKeyId: fs.s3n>.<awsSecretAccessKey >@ap-hw-dev/service-names';
Please help me.
Thanks,
Shouvanik
From: Jitendra Yadav [mailto:jeetuyadav200890@gmail.com<ma...@gmail.com>]
Sent: Thursday, September 05, 2013 5:11 PM
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: Re: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
Hi,
Did you configured fs.s3n.awsAccessKeyId or fs.s3n.awsSecretAccessKey properties
in your core-site.xml file?
Regards
Jitendra
On Thu, Sep 5, 2013 at 3:52 PM, <sh...@accenture.com>> wrote:
Hi,
I am executing the example, but got initial roadblock while trying out "Setting up tables (DDL Statements)" section.
When I executed the command "hive> create external table kv (key int, values string) location 's3n://data.s3ndemo.hive/kv';"
I got the error
FAILED: Error in metadata: MetaException(message:java.lang.IllegalArgumentException: AWS Access Key ID and Secret Access Key must be specified as the username or password (respectively) of a s3n URL, or by setting the fs.s3n.awsAccessKeyId or fs.s3n.awsSecretAccessKey properties (respectively).)
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
hive>
Please help.
Thanks,
Shouvanik
________________________________
This message is for the designated recipient only and may contain privileged, proprietary, or otherwise confidential information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of the e-mail by you is prohibited.
Where allowed by local law, electronic communications with Accenture and its affiliates, including e-mail and instant messaging (including content), may be scanned by our systems for the purposes of information security and assessment of internal compliance with Accenture policy.
______________________________________________________________________________________
www.accenture.com<http://www.accenture.com/>
--
Nitin Pawar
--
Nitin Pawar
--
Nitin Pawar
Re: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
Posted by Nitin Pawar <ni...@gmail.com>.
>From the config file it looks ok. Not really sure why it is not picking up
when you run it via hive cli.
Can you try putting these in core-site.xml of hdfs.
<property>
<name>fs.s3n.awsAccessKeyId</name>
<value>ID</value>
</property>
<property>
<name>fs.s3n.awsSecretAccessKey</name>
<value>SECRET</value>
</property>
On Tue, Sep 10, 2013 at 3:11 PM, <sh...@accenture.com> wrote:
> Hi Nitin,****
>
> ** **
>
> PFA hive-site.xml. I wanted to start Hortonworks HIVE. Can you please help.
> ****
>
> ** **
>
> Thanks,****
>
> Shouvanik****
>
> ** **
>
> *From:* Nitin Pawar [mailto:nitinpawar432@gmail.com]
> *Sent:* Tuesday, September 10, 2013 11:15 AM
>
> *To:* user@hive.apache.org
> *Subject:* Re: error executing the tutorial ->
> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
> ****
>
> ** **
>
> if you have set these values inside hive-site.xml and restarted hive
> service. It should work. ****
>
> ** **
>
> Share your hive-site.xml (please remove values for s3naccess key and
> secret ) ****
>
> ** **
>
> On Tue, Sep 10, 2013 at 3:17 AM, <sh...@accenture.com> wrote:**
> **
>
> Please help. I am badly stuck on this. Please help.****
>
> ****
>
> *From:* Haldar, Shouvanik
> *Sent:* Tuesday, September 10, 2013 1:03 AM****
>
>
> *To:* 'user@hive.apache.org'
> *Subject:* RE: error executing the tutorial ->
> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
> ****
>
> ****
>
> When I set values inside hive-site.xml & test whether values are set by
> typing hive> set; I see that the values are set.****
>
> ****
>
> Yet, when I run “CREATE EXTERNAL TABLE…” command, I get the same error****
>
> ****
>
> FAILED: Error in metadata:
> MetaException(message:java.lang.IllegalArgumentException: AWS Access Key ID
> and Secret Access Key must be specified as the username or password
> (respectively) of a s3n URL, or by setting the fs.s3n.awsAccessKeyId or
> fs.s3n.awsSecretAccessKey properties (respectively).)****
>
> FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.DDLTask****
>
> ****
>
> Please help.****
>
> ****
>
> Thanks,****
>
> Shouvanik****
>
> ****
>
> *From:* Haldar, Shouvanik ****
>
> *Sent:* Tuesday, September 10, 2013 12:39 AM****
>
> *To:* 'user@hive.apache.org'
> *Subject:* RE: error executing the tutorial ->
> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
> ****
>
> ****
>
> Hi Nitin,****
>
> ****
>
> Please help me with the below?****
>
> ****
>
> Thanks,****
>
> Shouvanik****
>
> ****
>
> *From:* Haldar, Shouvanik ****
>
> *Sent:* Saturday, September 07, 2013 2:30 AM****
>
> *To:* user@hive.apache.org
> *Subject:* RE: error executing the tutorial ->
> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
> ****
>
> ****
>
> Hi ,****
>
> ****
>
> I tried to set the parameters inside hive-site.xml on the node from where
> I am running hive. Even “hive>set;” command showed me that the parameters
> were successfully set. Yet, when running hive “CREATE EXTERNAL TABLE…”, I
> am getting the following error****
>
> ****
>
> FAILED: Error in metadata:
> MetaException(message:java.lang.IllegalArgumentException: AWS Access Key ID
> and Secret Access Key must be specified as the username or password
> (respectively) of a s3n URL, or by setting the fs.s3n.awsAccessKeyId or
> fs.s3n.awsSecretAccessKey properties (respectively).)****
>
> FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.DDLTask ****
>
> ****
>
> ****
>
> Please help.****
>
> ****
>
> Thanks,****
>
> Shouvanik****
>
> ****
>
> *From:* Nitin Pawar [mailto:nitinpawar432@gmail.com<ni...@gmail.com>]
> ****
>
> *Sent:* Saturday, September 07, 2013 2:25 AM****
>
> *To:* user@hive.apache.org
> *Subject:* Re: error executing the tutorial ->
> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
> ****
>
> ****
>
> did you try setting them in hive-site.xml file ? ****
>
> ****
>
> On Sat, Sep 7, 2013 at 2:18 AM, <sh...@accenture.com> wrote:***
> *
>
> Hi,****
>
> ****
>
> Can anybody please reply to this below? I am stuck up very bad.****
>
> ****
>
> Regards,****
>
> Shouvanik****
>
> ****
>
> *From:* Haldar, Shouvanik
> *Sent:* Friday, September 06, 2013 11:22 PM
> *To:* user@hive.apache.org
> *Subject:* RE: error executing the tutorial ->
> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
> ****
>
> ****
>
> Hi,****
>
> ****
>
> Is it possible to permanently set the below parameters? Please help.****
>
> ****
>
> Thanks,****
>
> SHouvanik****
>
> ****
>
> *From:* Jitendra Yadav [mailto:jeetuyadav200890@gmail.com<je...@gmail.com>]
>
> *Sent:* Thursday, September 05, 2013 6:02 PM****
>
>
> *To:* user@hive.apache.org
> *Subject:* Re: error executing the tutorial ->
> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
> ****
>
> ****
>
> OK got your point, As an alternative you can set these variables in hive
> environment before creating the table****
>
> ****
>
> Hive > set fs.s3n.awsSecretAccessKey='value'
> Hive > set fs.s3n.awsAccessKeyId='Value'****
>
> ****
>
> Hope this will help you.****
>
> ****
>
> Thanks****
>
> Jitendra
> On Thu, Sep 5, 2013 at 5:19 PM, <sh...@accenture.com> wrote:***
> *
>
> Hi,****
>
> ****
>
> Thanks for the reply.****
>
> ****
>
> I had seen in some posts, that you could mention these keys on the Hive
> CLI. If things work, then I am later planning to include it in
> core-site.xml of Hadoop config. Presently, I am not allowed to restart
> Hadoop services. ****
>
> ****
>
> “create external table ….****
>
> …..****
>
> LOCATION 's3n:// <fs.s3n.awsAccessKeyId: fs.s3n>.<awsSecretAccessKey>@ap-hw-dev/service-names';
> ****
>
> ****
>
> Please help me.****
>
> ****
>
> Thanks,****
>
> Shouvanik****
>
> ****
>
> *From:* Jitendra Yadav [mailto:jeetuyadav200890@gmail.com]
> *Sent:* Thursday, September 05, 2013 5:11 PM
> *To:* user@hive.apache.org
> *Subject:* Re: error executing the tutorial ->
> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
> ****
>
> ****
>
> Hi,****
>
> ****
>
> Did you configured fs.s3n.awsAccessKeyId or fs.s3n.awsSecretAccessKey
> properties****
>
> in your core-site.xml file?****
>
> ****
>
> Regards****
>
> Jitendra****
>
> On Thu, Sep 5, 2013 at 3:52 PM, <sh...@accenture.com> wrote:***
> *
>
> Hi,****
>
> ****
>
> I am executing the example, but got initial roadblock while trying out
> “Setting up tables (DDL Statements)” section.****
>
> ****
>
> When I executed the command “hive> create external table kv (key int,
> values string) location 's3n://data.s3ndemo.hive/kv';”****
>
> ****
>
> I got the error****
>
> ****
>
> FAILED: Error in metadata:
> MetaException(message:java.lang.IllegalArgumentException: AWS Access Key ID
> and Secret Access Key must be specified as the username or password
> (respectively) of a s3n URL, or by setting the fs.s3n.awsAccessKeyId or
> fs.s3n.awsSecretAccessKey properties (respectively).)****
>
> FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.DDLTask****
>
> hive>****
>
> ****
>
> Please help.****
>
> ****
>
> ****
>
> Thanks,****
>
> Shouvanik****
>
> ****
> ------------------------------
>
> This message is for the designated recipient only and may contain
> privileged, proprietary, or otherwise confidential information. If you have
> received it in error, please notify the sender immediately and delete the
> original. Any other use of the e-mail by you is prohibited.
>
> Where allowed by local law, electronic communications with Accenture and
> its affiliates, including e-mail and instant messaging (including content),
> may be scanned by our systems for the purposes of information security and
> assessment of internal compliance with Accenture policy.
>
>
> ______________________________________________________________________________________
>
> www.accenture.com****
>
> ****
>
> ****
>
>
>
> ****
>
> ****
>
> --
> Nitin Pawar****
>
>
>
> ****
>
> ** **
>
> --
> Nitin Pawar****
>
--
Nitin Pawar
RE: error executing the tutorial ->
https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
Posted by sh...@accenture.com.
Hi Nitin,
PFA hive-site.xml. I wanted to start Hortonworks HIVE. Can you please help.
Thanks,
Shouvanik
From: Nitin Pawar [mailto:nitinpawar432@gmail.com]
Sent: Tuesday, September 10, 2013 11:15 AM
To: user@hive.apache.org
Subject: Re: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
if you have set these values inside hive-site.xml and restarted hive service. It should work.
Share your hive-site.xml (please remove values for s3naccess key and secret )
On Tue, Sep 10, 2013 at 3:17 AM, <sh...@accenture.com>> wrote:
Please help. I am badly stuck on this. Please help.
From: Haldar, Shouvanik
Sent: Tuesday, September 10, 2013 1:03 AM
To: 'user@hive.apache.org<ma...@hive.apache.org>'
Subject: RE: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
When I set values inside hive-site.xml & test whether values are set by typing hive> set; I see that the values are set.
Yet, when I run "CREATE EXTERNAL TABLE..." command, I get the same error
FAILED: Error in metadata: MetaException(message:java.lang.IllegalArgumentException: AWS Access Key ID and Secret Access Key must be specified as the username or password (respectively) of a s3n URL, or by setting the fs.s3n.awsAccessKeyId or fs.s3n.awsSecretAccessKey properties (respectively).)
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
Please help.
Thanks,
Shouvanik
From: Haldar, Shouvanik
Sent: Tuesday, September 10, 2013 12:39 AM
To: 'user@hive.apache.org<ma...@hive.apache.org>'
Subject: RE: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
Hi Nitin,
Please help me with the below?
Thanks,
Shouvanik
From: Haldar, Shouvanik
Sent: Saturday, September 07, 2013 2:30 AM
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: RE: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
Hi ,
I tried to set the parameters inside hive-site.xml on the node from where I am running hive. Even "hive>set;" command showed me that the parameters were successfully set. Yet, when running hive "CREATE EXTERNAL TABLE...", I am getting the following error
FAILED: Error in metadata: MetaException(message:java.lang.IllegalArgumentException: AWS Access Key ID and Secret Access Key must be specified as the username or password (respectively) of a s3n URL, or by setting the fs.s3n.awsAccessKeyId or fs.s3n.awsSecretAccessKey properties (respectively).)
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
Please help.
Thanks,
Shouvanik
From: Nitin Pawar [mailto:nitinpawar432@gmail.com]
Sent: Saturday, September 07, 2013 2:25 AM
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: Re: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
did you try setting them in hive-site.xml file ?
On Sat, Sep 7, 2013 at 2:18 AM, <sh...@accenture.com>> wrote:
Hi,
Can anybody please reply to this below? I am stuck up very bad.
Regards,
Shouvanik
From: Haldar, Shouvanik
Sent: Friday, September 06, 2013 11:22 PM
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: RE: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
Hi,
Is it possible to permanently set the below parameters? Please help.
Thanks,
SHouvanik
From: Jitendra Yadav [mailto:jeetuyadav200890@gmail.com]
Sent: Thursday, September 05, 2013 6:02 PM
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: Re: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
OK got your point, As an alternative you can set these variables in hive environment before creating the table
Hive > set fs.s3n.awsSecretAccessKey='value'
Hive > set fs.s3n.awsAccessKeyId='Value'
Hope this will help you.
Thanks
Jitendra
On Thu, Sep 5, 2013 at 5:19 PM, <sh...@accenture.com>> wrote:
Hi,
Thanks for the reply.
I had seen in some posts, that you could mention these keys on the Hive CLI. If things work, then I am later planning to include it in core-site.xml of Hadoop config. Presently, I am not allowed to restart Hadoop services.
"create external table ....
.....
LOCATION 's3n:// <fs.s3n.awsAccessKeyId: fs.s3n>.<awsSecretAccessKey >@ap-hw-dev/service-names';
Please help me.
Thanks,
Shouvanik
From: Jitendra Yadav [mailto:jeetuyadav200890@gmail.com<ma...@gmail.com>]
Sent: Thursday, September 05, 2013 5:11 PM
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: Re: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
Hi,
Did you configured fs.s3n.awsAccessKeyId or fs.s3n.awsSecretAccessKey properties
in your core-site.xml file?
Regards
Jitendra
On Thu, Sep 5, 2013 at 3:52 PM, <sh...@accenture.com>> wrote:
Hi,
I am executing the example, but got initial roadblock while trying out "Setting up tables (DDL Statements)" section.
When I executed the command "hive> create external table kv (key int, values string) location 's3n://data.s3ndemo.hive/kv';"
I got the error
FAILED: Error in metadata: MetaException(message:java.lang.IllegalArgumentException: AWS Access Key ID and Secret Access Key must be specified as the username or password (respectively) of a s3n URL, or by setting the fs.s3n.awsAccessKeyId or fs.s3n.awsSecretAccessKey properties (respectively).)
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
hive>
Please help.
Thanks,
Shouvanik
________________________________
This message is for the designated recipient only and may contain privileged, proprietary, or otherwise confidential information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of the e-mail by you is prohibited.
Where allowed by local law, electronic communications with Accenture and its affiliates, including e-mail and instant messaging (including content), may be scanned by our systems for the purposes of information security and assessment of internal compliance with Accenture policy.
______________________________________________________________________________________
www.accenture.com<http://www.accenture.com/>
--
Nitin Pawar
--
Nitin Pawar
Re: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
Posted by Nitin Pawar <ni...@gmail.com>.
if you have set these values inside hive-site.xml and restarted hive
service. It should work.
Share your hive-site.xml (please remove values for s3naccess key and secret
)
On Tue, Sep 10, 2013 at 3:17 AM, <sh...@accenture.com> wrote:
> Please help. I am badly stuck on this. Please help.****
>
> ** **
>
> *From:* Haldar, Shouvanik
> *Sent:* Tuesday, September 10, 2013 1:03 AM
>
> *To:* 'user@hive.apache.org'
> *Subject:* RE: error executing the tutorial ->
> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
> ****
>
> ** **
>
> When I set values inside hive-site.xml & test whether values are set by
> typing hive> set; I see that the values are set.****
>
> ** **
>
> Yet, when I run “CREATE EXTERNAL TABLE…” command, I get the same error****
>
> ** **
>
> FAILED: Error in metadata:
> MetaException(message:java.lang.IllegalArgumentException: AWS Access Key ID
> and Secret Access Key must be specified as the username or password
> (respectively) of a s3n URL, or by setting the fs.s3n.awsAccessKeyId or
> fs.s3n.awsSecretAccessKey properties (respectively).)****
>
> FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.DDLTask****
>
> ** **
>
> Please help.****
>
> ** **
>
> Thanks,****
>
> Shouvanik****
>
> ** **
>
> *From:* Haldar, Shouvanik
> *Sent:* Tuesday, September 10, 2013 12:39 AM
> *To:* 'user@hive.apache.org'
> *Subject:* RE: error executing the tutorial ->
> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
> ****
>
> ** **
>
> Hi Nitin,****
>
> ** **
>
> Please help me with the below?****
>
> ** **
>
> Thanks,****
>
> Shouvanik****
>
> ** **
>
> *From:* Haldar, Shouvanik
> *Sent:* Saturday, September 07, 2013 2:30 AM
> *To:* user@hive.apache.org
> *Subject:* RE: error executing the tutorial ->
> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
> ****
>
> ** **
>
> Hi ,****
>
> ** **
>
> I tried to set the parameters inside hive-site.xml on the node from where
> I am running hive. Even “hive>set;” command showed me that the parameters
> were successfully set. Yet, when running hive “CREATE EXTERNAL TABLE…”, I
> am getting the following error****
>
> ** **
>
> FAILED: Error in metadata:
> MetaException(message:java.lang.IllegalArgumentException: AWS Access Key ID
> and Secret Access Key must be specified as the username or password
> (respectively) of a s3n URL, or by setting the fs.s3n.awsAccessKeyId or
> fs.s3n.awsSecretAccessKey properties (respectively).)****
>
> FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.DDLTask ****
>
> ** **
>
> ** **
>
> Please help.****
>
> ** **
>
> Thanks,****
>
> Shouvanik****
>
> ** **
>
> *From:* Nitin Pawar [mailto:nitinpawar432@gmail.com<ni...@gmail.com>]
>
> *Sent:* Saturday, September 07, 2013 2:25 AM
> *To:* user@hive.apache.org
> *Subject:* Re: error executing the tutorial ->
> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
> ****
>
> ** **
>
> did you try setting them in hive-site.xml file ? ****
>
> ** **
>
> On Sat, Sep 7, 2013 at 2:18 AM, <sh...@accenture.com> wrote:***
> *
>
> Hi,****
>
> ****
>
> Can anybody please reply to this below? I am stuck up very bad.****
>
> ****
>
> Regards,****
>
> Shouvanik****
>
> ****
>
> *From:* Haldar, Shouvanik
> *Sent:* Friday, September 06, 2013 11:22 PM
> *To:* user@hive.apache.org
> *Subject:* RE: error executing the tutorial ->
> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
> ****
>
> ****
>
> Hi,****
>
> ****
>
> Is it possible to permanently set the below parameters? Please help.****
>
> ****
>
> Thanks,****
>
> SHouvanik****
>
> ****
>
> *From:* Jitendra Yadav [mailto:jeetuyadav200890@gmail.com<je...@gmail.com>]
>
> *Sent:* Thursday, September 05, 2013 6:02 PM****
>
>
> *To:* user@hive.apache.org
> *Subject:* Re: error executing the tutorial ->
> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
> ****
>
> ****
>
> OK got your point, As an alternative you can set these variables in hive
> environment before creating the table****
>
> ****
>
> Hive > set fs.s3n.awsSecretAccessKey='value'
> Hive > set fs.s3n.awsAccessKeyId='Value'****
>
> ****
>
> Hope this will help you.****
>
> ****
>
> Thanks****
>
> Jitendra
> On Thu, Sep 5, 2013 at 5:19 PM, <sh...@accenture.com> wrote:***
> *
>
> Hi,****
>
> ****
>
> Thanks for the reply.****
>
> ****
>
> I had seen in some posts, that you could mention these keys on the Hive
> CLI. If things work, then I am later planning to include it in
> core-site.xml of Hadoop config. Presently, I am not allowed to restart
> Hadoop services. ****
>
> ****
>
> “create external table ….****
>
> …..****
>
> LOCATION 's3n:// <fs.s3n.awsAccessKeyId: fs.s3n>.<awsSecretAccessKey>@ap-hw-dev/service-names';
> ****
>
> ****
>
> Please help me.****
>
> ****
>
> Thanks,****
>
> Shouvanik****
>
> ****
>
> *From:* Jitendra Yadav [mailto:jeetuyadav200890@gmail.com]
> *Sent:* Thursday, September 05, 2013 5:11 PM
> *To:* user@hive.apache.org
> *Subject:* Re: error executing the tutorial ->
> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
> ****
>
> ****
>
> Hi,****
>
> ****
>
> Did you configured fs.s3n.awsAccessKeyId or fs.s3n.awsSecretAccessKey
> properties****
>
> in your core-site.xml file?****
>
> ****
>
> Regards****
>
> Jitendra****
>
> On Thu, Sep 5, 2013 at 3:52 PM, <sh...@accenture.com> wrote:***
> *
>
> Hi,****
>
> ****
>
> I am executing the example, but got initial roadblock while trying out
> “Setting up tables (DDL Statements)” section.****
>
> ****
>
> When I executed the command “hive> create external table kv (key int,
> values string) location 's3n://data.s3ndemo.hive/kv';”****
>
> ****
>
> I got the error****
>
> ****
>
> FAILED: Error in metadata:
> MetaException(message:java.lang.IllegalArgumentException: AWS Access Key ID
> and Secret Access Key must be specified as the username or password
> (respectively) of a s3n URL, or by setting the fs.s3n.awsAccessKeyId or
> fs.s3n.awsSecretAccessKey properties (respectively).)****
>
> FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.DDLTask****
>
> hive>****
>
> ****
>
> Please help.****
>
> ****
>
> ****
>
> Thanks,****
>
> Shouvanik****
>
> ****
> ------------------------------
>
> This message is for the designated recipient only and may contain
> privileged, proprietary, or otherwise confidential information. If you have
> received it in error, please notify the sender immediately and delete the
> original. Any other use of the e-mail by you is prohibited.
>
> Where allowed by local law, electronic communications with Accenture and
> its affiliates, including e-mail and instant messaging (including content),
> may be scanned by our systems for the purposes of information security and
> assessment of internal compliance with Accenture policy.
>
>
> ______________________________________________________________________________________
>
> www.accenture.com****
>
> ****
>
> ****
>
>
>
> ****
>
> ** **
>
> --
> Nitin Pawar****
>
--
Nitin Pawar
RE: error executing the tutorial ->
https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
Posted by sh...@accenture.com.
Please help. I am badly stuck on this. Please help.
From: Haldar, Shouvanik
Sent: Tuesday, September 10, 2013 1:03 AM
To: 'user@hive.apache.org'
Subject: RE: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
When I set values inside hive-site.xml & test whether values are set by typing hive> set; I see that the values are set.
Yet, when I run "CREATE EXTERNAL TABLE..." command, I get the same error
FAILED: Error in metadata: MetaException(message:java.lang.IllegalArgumentException: AWS Access Key ID and Secret Access Key must be specified as the username or password (respectively) of a s3n URL, or by setting the fs.s3n.awsAccessKeyId or fs.s3n.awsSecretAccessKey properties (respectively).)
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
Please help.
Thanks,
Shouvanik
From: Haldar, Shouvanik
Sent: Tuesday, September 10, 2013 12:39 AM
To: 'user@hive.apache.org'
Subject: RE: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
Hi Nitin,
Please help me with the below?
Thanks,
Shouvanik
From: Haldar, Shouvanik
Sent: Saturday, September 07, 2013 2:30 AM
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: RE: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
Hi ,
I tried to set the parameters inside hive-site.xml on the node from where I am running hive. Even "hive>set;" command showed me that the parameters were successfully set. Yet, when running hive "CREATE EXTERNAL TABLE...", I am getting the following error
FAILED: Error in metadata: MetaException(message:java.lang.IllegalArgumentException: AWS Access Key ID and Secret Access Key must be specified as the username or password (respectively) of a s3n URL, or by setting the fs.s3n.awsAccessKeyId or fs.s3n.awsSecretAccessKey properties (respectively).)
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
Please help.
Thanks,
Shouvanik
From: Nitin Pawar [mailto:nitinpawar432@gmail.com]
Sent: Saturday, September 07, 2013 2:25 AM
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: Re: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
did you try setting them in hive-site.xml file ?
On Sat, Sep 7, 2013 at 2:18 AM, <sh...@accenture.com>> wrote:
Hi,
Can anybody please reply to this below? I am stuck up very bad.
Regards,
Shouvanik
From: Haldar, Shouvanik
Sent: Friday, September 06, 2013 11:22 PM
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: RE: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
Hi,
Is it possible to permanently set the below parameters? Please help.
Thanks,
SHouvanik
From: Jitendra Yadav [mailto:jeetuyadav200890@gmail.com]
Sent: Thursday, September 05, 2013 6:02 PM
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: Re: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
OK got your point, As an alternative you can set these variables in hive environment before creating the table
Hive > set fs.s3n.awsSecretAccessKey='value'
Hive > set fs.s3n.awsAccessKeyId='Value'
Hope this will help you.
Thanks
Jitendra
On Thu, Sep 5, 2013 at 5:19 PM, <sh...@accenture.com>> wrote:
Hi,
Thanks for the reply.
I had seen in some posts, that you could mention these keys on the Hive CLI. If things work, then I am later planning to include it in core-site.xml of Hadoop config. Presently, I am not allowed to restart Hadoop services.
"create external table ....
.....
LOCATION 's3n:// <fs.s3n.awsAccessKeyId: fs.s3n>.<awsSecretAccessKey >@ap-hw-dev/service-names';
Please help me.
Thanks,
Shouvanik
From: Jitendra Yadav [mailto:jeetuyadav200890@gmail.com<ma...@gmail.com>]
Sent: Thursday, September 05, 2013 5:11 PM
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: Re: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
Hi,
Did you configured fs.s3n.awsAccessKeyId or fs.s3n.awsSecretAccessKey properties
in your core-site.xml file?
Regards
Jitendra
On Thu, Sep 5, 2013 at 3:52 PM, <sh...@accenture.com>> wrote:
Hi,
I am executing the example, but got initial roadblock while trying out "Setting up tables (DDL Statements)" section.
When I executed the command "hive> create external table kv (key int, values string) location 's3n://data.s3ndemo.hive/kv';"
I got the error
FAILED: Error in metadata: MetaException(message:java.lang.IllegalArgumentException: AWS Access Key ID and Secret Access Key must be specified as the username or password (respectively) of a s3n URL, or by setting the fs.s3n.awsAccessKeyId or fs.s3n.awsSecretAccessKey properties (respectively).)
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
hive>
Please help.
Thanks,
Shouvanik
________________________________
This message is for the designated recipient only and may contain privileged, proprietary, or otherwise confidential information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of the e-mail by you is prohibited.
Where allowed by local law, electronic communications with Accenture and its affiliates, including e-mail and instant messaging (including content), may be scanned by our systems for the purposes of information security and assessment of internal compliance with Accenture policy.
______________________________________________________________________________________
www.accenture.com<http://www.accenture.com/>
--
Nitin Pawar
RE: error executing the tutorial ->
https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
Posted by sh...@accenture.com.
Hi Nitin,
Please help me with the below?
Thanks,
Shouvanik
From: Haldar, Shouvanik
Sent: Saturday, September 07, 2013 2:30 AM
To: user@hive.apache.org
Subject: RE: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
Hi ,
I tried to set the parameters inside hive-site.xml on the node from where I am running hive. Even "hive>set;" command showed me that the parameters were successfully set. Yet, when running hive "CREATE EXTERNAL TABLE...", I am getting the following error
FAILED: Error in metadata: MetaException(message:java.lang.IllegalArgumentException: AWS Access Key ID and Secret Access Key must be specified as the username or password (respectively) of a s3n URL, or by setting the fs.s3n.awsAccessKeyId or fs.s3n.awsSecretAccessKey properties (respectively).)
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
Please help.
Thanks,
Shouvanik
From: Nitin Pawar [mailto:nitinpawar432@gmail.com]
Sent: Saturday, September 07, 2013 2:25 AM
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: Re: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
did you try setting them in hive-site.xml file ?
On Sat, Sep 7, 2013 at 2:18 AM, <sh...@accenture.com>> wrote:
Hi,
Can anybody please reply to this below? I am stuck up very bad.
Regards,
Shouvanik
From: Haldar, Shouvanik
Sent: Friday, September 06, 2013 11:22 PM
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: RE: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
Hi,
Is it possible to permanently set the below parameters? Please help.
Thanks,
SHouvanik
From: Jitendra Yadav [mailto:jeetuyadav200890@gmail.com]
Sent: Thursday, September 05, 2013 6:02 PM
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: Re: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
OK got your point, As an alternative you can set these variables in hive environment before creating the table
Hive > set fs.s3n.awsSecretAccessKey='value'
Hive > set fs.s3n.awsAccessKeyId='Value'
Hope this will help you.
Thanks
Jitendra
On Thu, Sep 5, 2013 at 5:19 PM, <sh...@accenture.com>> wrote:
Hi,
Thanks for the reply.
I had seen in some posts, that you could mention these keys on the Hive CLI. If things work, then I am later planning to include it in core-site.xml of Hadoop config. Presently, I am not allowed to restart Hadoop services.
"create external table ....
.....
LOCATION 's3n:// <fs.s3n.awsAccessKeyId: fs.s3n>.<awsSecretAccessKey >@ap-hw-dev/service-names';
Please help me.
Thanks,
Shouvanik
From: Jitendra Yadav [mailto:jeetuyadav200890@gmail.com<ma...@gmail.com>]
Sent: Thursday, September 05, 2013 5:11 PM
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: Re: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
Hi,
Did you configured fs.s3n.awsAccessKeyId or fs.s3n.awsSecretAccessKey properties
in your core-site.xml file?
Regards
Jitendra
On Thu, Sep 5, 2013 at 3:52 PM, <sh...@accenture.com>> wrote:
Hi,
I am executing the example, but got initial roadblock while trying out "Setting up tables (DDL Statements)" section.
When I executed the command "hive> create external table kv (key int, values string) location 's3n://data.s3ndemo.hive/kv';"
I got the error
FAILED: Error in metadata: MetaException(message:java.lang.IllegalArgumentException: AWS Access Key ID and Secret Access Key must be specified as the username or password (respectively) of a s3n URL, or by setting the fs.s3n.awsAccessKeyId or fs.s3n.awsSecretAccessKey properties (respectively).)
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
hive>
Please help.
Thanks,
Shouvanik
________________________________
This message is for the designated recipient only and may contain privileged, proprietary, or otherwise confidential information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of the e-mail by you is prohibited.
Where allowed by local law, electronic communications with Accenture and its affiliates, including e-mail and instant messaging (including content), may be scanned by our systems for the purposes of information security and assessment of internal compliance with Accenture policy.
______________________________________________________________________________________
www.accenture.com<http://www.accenture.com/>
--
Nitin Pawar
RE: error executing the tutorial ->
https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
Posted by sh...@accenture.com.
Hi ,
I tried to set the parameters inside hive-site.xml on the node from where I am running hive. Even "hive>set;" command showed me that the parameters were successfully set. Yet, when running hive "CREATE EXTERNAL TABLE...", I am getting the following error
FAILED: Error in metadata: MetaException(message:java.lang.IllegalArgumentException: AWS Access Key ID and Secret Access Key must be specified as the username or password (respectively) of a s3n URL, or by setting the fs.s3n.awsAccessKeyId or fs.s3n.awsSecretAccessKey properties (respectively).)
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
Please help.
Thanks,
Shouvanik
From: Nitin Pawar [mailto:nitinpawar432@gmail.com]
Sent: Saturday, September 07, 2013 2:25 AM
To: user@hive.apache.org
Subject: Re: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
did you try setting them in hive-site.xml file ?
On Sat, Sep 7, 2013 at 2:18 AM, <sh...@accenture.com>> wrote:
Hi,
Can anybody please reply to this below? I am stuck up very bad.
Regards,
Shouvanik
From: Haldar, Shouvanik
Sent: Friday, September 06, 2013 11:22 PM
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: RE: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
Hi,
Is it possible to permanently set the below parameters? Please help.
Thanks,
SHouvanik
From: Jitendra Yadav [mailto:jeetuyadav200890@gmail.com]
Sent: Thursday, September 05, 2013 6:02 PM
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: Re: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
OK got your point, As an alternative you can set these variables in hive environment before creating the table
Hive > set fs.s3n.awsSecretAccessKey='value'
Hive > set fs.s3n.awsAccessKeyId='Value'
Hope this will help you.
Thanks
Jitendra
On Thu, Sep 5, 2013 at 5:19 PM, <sh...@accenture.com>> wrote:
Hi,
Thanks for the reply.
I had seen in some posts, that you could mention these keys on the Hive CLI. If things work, then I am later planning to include it in core-site.xml of Hadoop config. Presently, I am not allowed to restart Hadoop services.
"create external table ....
.....
LOCATION 's3n:// <fs.s3n.awsAccessKeyId: fs.s3n>.<awsSecretAccessKey >@ap-hw-dev/service-names';
Please help me.
Thanks,
Shouvanik
From: Jitendra Yadav [mailto:jeetuyadav200890@gmail.com<ma...@gmail.com>]
Sent: Thursday, September 05, 2013 5:11 PM
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: Re: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
Hi,
Did you configured fs.s3n.awsAccessKeyId or fs.s3n.awsSecretAccessKey properties
in your core-site.xml file?
Regards
Jitendra
On Thu, Sep 5, 2013 at 3:52 PM, <sh...@accenture.com>> wrote:
Hi,
I am executing the example, but got initial roadblock while trying out "Setting up tables (DDL Statements)" section.
When I executed the command "hive> create external table kv (key int, values string) location 's3n://data.s3ndemo.hive/kv';"
I got the error
FAILED: Error in metadata: MetaException(message:java.lang.IllegalArgumentException: AWS Access Key ID and Secret Access Key must be specified as the username or password (respectively) of a s3n URL, or by setting the fs.s3n.awsAccessKeyId or fs.s3n.awsSecretAccessKey properties (respectively).)
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
hive>
Please help.
Thanks,
Shouvanik
________________________________
This message is for the designated recipient only and may contain privileged, proprietary, or otherwise confidential information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of the e-mail by you is prohibited.
Where allowed by local law, electronic communications with Accenture and its affiliates, including e-mail and instant messaging (including content), may be scanned by our systems for the purposes of information security and assessment of internal compliance with Accenture policy.
______________________________________________________________________________________
www.accenture.com<http://www.accenture.com/>
--
Nitin Pawar
RE: error executing the tutorial ->
https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
Posted by sh...@accenture.com.
When I set values inside hive-site.xml & test whether values are set by typing hive> set; I see that the values are set.
Yet, when I run "CREATE EXTERNAL TABLE..." command, I get the same error
FAILED: Error in metadata: MetaException(message:java.lang.IllegalArgumentException: AWS Access Key ID and Secret Access Key must be specified as the username or password (respectively) of a s3n URL, or by setting the fs.s3n.awsAccessKeyId or fs.s3n.awsSecretAccessKey properties (respectively).)
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
Please help.
Thanks,
Shouvanik
From: Haldar, Shouvanik
Sent: Tuesday, September 10, 2013 12:39 AM
To: 'user@hive.apache.org'
Subject: RE: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
Hi Nitin,
Please help me with the below?
Thanks,
Shouvanik
From: Haldar, Shouvanik
Sent: Saturday, September 07, 2013 2:30 AM
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: RE: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
Hi ,
I tried to set the parameters inside hive-site.xml on the node from where I am running hive. Even "hive>set;" command showed me that the parameters were successfully set. Yet, when running hive "CREATE EXTERNAL TABLE...", I am getting the following error
FAILED: Error in metadata: MetaException(message:java.lang.IllegalArgumentException: AWS Access Key ID and Secret Access Key must be specified as the username or password (respectively) of a s3n URL, or by setting the fs.s3n.awsAccessKeyId or fs.s3n.awsSecretAccessKey properties (respectively).)
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
Please help.
Thanks,
Shouvanik
From: Nitin Pawar [mailto:nitinpawar432@gmail.com]
Sent: Saturday, September 07, 2013 2:25 AM
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: Re: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
did you try setting them in hive-site.xml file ?
On Sat, Sep 7, 2013 at 2:18 AM, <sh...@accenture.com>> wrote:
Hi,
Can anybody please reply to this below? I am stuck up very bad.
Regards,
Shouvanik
From: Haldar, Shouvanik
Sent: Friday, September 06, 2013 11:22 PM
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: RE: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
Hi,
Is it possible to permanently set the below parameters? Please help.
Thanks,
SHouvanik
From: Jitendra Yadav [mailto:jeetuyadav200890@gmail.com]
Sent: Thursday, September 05, 2013 6:02 PM
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: Re: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
OK got your point, As an alternative you can set these variables in hive environment before creating the table
Hive > set fs.s3n.awsSecretAccessKey='value'
Hive > set fs.s3n.awsAccessKeyId='Value'
Hope this will help you.
Thanks
Jitendra
On Thu, Sep 5, 2013 at 5:19 PM, <sh...@accenture.com>> wrote:
Hi,
Thanks for the reply.
I had seen in some posts, that you could mention these keys on the Hive CLI. If things work, then I am later planning to include it in core-site.xml of Hadoop config. Presently, I am not allowed to restart Hadoop services.
"create external table ....
.....
LOCATION 's3n:// <fs.s3n.awsAccessKeyId: fs.s3n>.<awsSecretAccessKey >@ap-hw-dev/service-names';
Please help me.
Thanks,
Shouvanik
From: Jitendra Yadav [mailto:jeetuyadav200890@gmail.com<ma...@gmail.com>]
Sent: Thursday, September 05, 2013 5:11 PM
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: Re: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
Hi,
Did you configured fs.s3n.awsAccessKeyId or fs.s3n.awsSecretAccessKey properties
in your core-site.xml file?
Regards
Jitendra
On Thu, Sep 5, 2013 at 3:52 PM, <sh...@accenture.com>> wrote:
Hi,
I am executing the example, but got initial roadblock while trying out "Setting up tables (DDL Statements)" section.
When I executed the command "hive> create external table kv (key int, values string) location 's3n://data.s3ndemo.hive/kv';"
I got the error
FAILED: Error in metadata: MetaException(message:java.lang.IllegalArgumentException: AWS Access Key ID and Secret Access Key must be specified as the username or password (respectively) of a s3n URL, or by setting the fs.s3n.awsAccessKeyId or fs.s3n.awsSecretAccessKey properties (respectively).)
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
hive>
Please help.
Thanks,
Shouvanik
________________________________
This message is for the designated recipient only and may contain privileged, proprietary, or otherwise confidential information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of the e-mail by you is prohibited.
Where allowed by local law, electronic communications with Accenture and its affiliates, including e-mail and instant messaging (including content), may be scanned by our systems for the purposes of information security and assessment of internal compliance with Accenture policy.
______________________________________________________________________________________
www.accenture.com<http://www.accenture.com/>
--
Nitin Pawar
Re: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
Posted by Nitin Pawar <ni...@gmail.com>.
did you try setting them in hive-site.xml file ?
On Sat, Sep 7, 2013 at 2:18 AM, <sh...@accenture.com> wrote:
> Hi,****
>
> ** **
>
> Can anybody please reply to this below? I am stuck up very bad.****
>
> ** **
>
> Regards,****
>
> Shouvanik****
>
> ** **
>
> *From:* Haldar, Shouvanik
> *Sent:* Friday, September 06, 2013 11:22 PM
> *To:* user@hive.apache.org
> *Subject:* RE: error executing the tutorial ->
> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
> ****
>
> ** **
>
> Hi,****
>
> ** **
>
> Is it possible to permanently set the below parameters? Please help.****
>
> ** **
>
> Thanks,****
>
> SHouvanik****
>
> ** **
>
> *From:* Jitendra Yadav [mailto:jeetuyadav200890@gmail.com<je...@gmail.com>]
>
> *Sent:* Thursday, September 05, 2013 6:02 PM
>
> *To:* user@hive.apache.org
> *Subject:* Re: error executing the tutorial ->
> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
> ****
>
> ** **
>
> OK got your point, As an alternative you can set these variables in hive
> environment before creating the table****
>
> ****
>
> Hive > set fs.s3n.awsSecretAccessKey='value'
> Hive > set fs.s3n.awsAccessKeyId='Value'****
>
> ****
>
> Hope this will help you.****
>
> ****
>
> Thanks****
>
> Jitendra
> On Thu, Sep 5, 2013 at 5:19 PM, <sh...@accenture.com> wrote:***
> *
>
> Hi,****
>
> ****
>
> Thanks for the reply.****
>
> ****
>
> I had seen in some posts, that you could mention these keys on the Hive
> CLI. If things work, then I am later planning to include it in
> core-site.xml of Hadoop config. Presently, I am not allowed to restart
> Hadoop services. ****
>
> ****
>
> “create external table ….****
>
> …..****
>
> LOCATION 's3n:// <fs.s3n.awsAccessKeyId: fs.s3n>.<awsSecretAccessKey>@ap-hw-dev/service-names';
> ****
>
> ****
>
> Please help me.****
>
> ****
>
> Thanks,****
>
> Shouvanik****
>
> ****
>
> *From:* Jitendra Yadav [mailto:jeetuyadav200890@gmail.com]
> *Sent:* Thursday, September 05, 2013 5:11 PM
> *To:* user@hive.apache.org
> *Subject:* Re: error executing the tutorial ->
> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
> ****
>
> ****
>
> Hi,****
>
> ****
>
> Did you configured fs.s3n.awsAccessKeyId or fs.s3n.awsSecretAccessKey
> properties****
>
> in your core-site.xml file?****
>
> ****
>
> Regards****
>
> Jitendra****
>
> On Thu, Sep 5, 2013 at 3:52 PM, <sh...@accenture.com> wrote:***
> *
>
> Hi,****
>
> ****
>
> I am executing the example, but got initial roadblock while trying out
> “Setting up tables (DDL Statements)” section.****
>
> ****
>
> When I executed the command “hive> create external table kv (key int,
> values string) location 's3n://data.s3ndemo.hive/kv';”****
>
> ****
>
> I got the error****
>
> ****
>
> FAILED: Error in metadata:
> MetaException(message:java.lang.IllegalArgumentException: AWS Access Key ID
> and Secret Access Key must be specified as the username or password
> (respectively) of a s3n URL, or by setting the fs.s3n.awsAccessKeyId or
> fs.s3n.awsSecretAccessKey properties (respectively).)****
>
> FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.DDLTask****
>
> hive>****
>
> ****
>
> Please help.****
>
> ****
>
> ****
>
> Thanks,****
>
> Shouvanik****
>
> ****
> ------------------------------
>
> This message is for the designated recipient only and may contain
> privileged, proprietary, or otherwise confidential information. If you have
> received it in error, please notify the sender immediately and delete the
> original. Any other use of the e-mail by you is prohibited.
>
> Where allowed by local law, electronic communications with Accenture and
> its affiliates, including e-mail and instant messaging (including content),
> may be scanned by our systems for the purposes of information security and
> assessment of internal compliance with Accenture policy.
>
>
> ______________________________________________________________________________________
>
> www.accenture.com****
>
> ****
>
> ** **
>
--
Nitin Pawar
RE: error executing the tutorial ->
https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
Posted by sh...@accenture.com.
Hi,
Can anybody please reply to this below? I am stuck up very bad.
Regards,
Shouvanik
From: Haldar, Shouvanik
Sent: Friday, September 06, 2013 11:22 PM
To: user@hive.apache.org
Subject: RE: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
Hi,
Is it possible to permanently set the below parameters? Please help.
Thanks,
SHouvanik
From: Jitendra Yadav [mailto:jeetuyadav200890@gmail.com]
Sent: Thursday, September 05, 2013 6:02 PM
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: Re: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
OK got your point, As an alternative you can set these variables in hive environment before creating the table
Hive > set fs.s3n.awsSecretAccessKey='value'
Hive > set fs.s3n.awsAccessKeyId='Value'
Hope this will help you.
Thanks
Jitendra
On Thu, Sep 5, 2013 at 5:19 PM, <sh...@accenture.com>> wrote:
Hi,
Thanks for the reply.
I had seen in some posts, that you could mention these keys on the Hive CLI. If things work, then I am later planning to include it in core-site.xml of Hadoop config. Presently, I am not allowed to restart Hadoop services.
"create external table ....
.....
LOCATION 's3n:// <fs.s3n.awsAccessKeyId: fs.s3n>.<awsSecretAccessKey >@ap-hw-dev/service-names';
Please help me.
Thanks,
Shouvanik
From: Jitendra Yadav [mailto:jeetuyadav200890@gmail.com<ma...@gmail.com>]
Sent: Thursday, September 05, 2013 5:11 PM
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: Re: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
Hi,
Did you configured fs.s3n.awsAccessKeyId or fs.s3n.awsSecretAccessKey properties
in your core-site.xml file?
Regards
Jitendra
On Thu, Sep 5, 2013 at 3:52 PM, <sh...@accenture.com>> wrote:
Hi,
I am executing the example, but got initial roadblock while trying out "Setting up tables (DDL Statements)" section.
When I executed the command "hive> create external table kv (key int, values string) location 's3n://data.s3ndemo.hive/kv';"
I got the error
FAILED: Error in metadata: MetaException(message:java.lang.IllegalArgumentException: AWS Access Key ID and Secret Access Key must be specified as the username or password (respectively) of a s3n URL, or by setting the fs.s3n.awsAccessKeyId or fs.s3n.awsSecretAccessKey properties (respectively).)
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
hive>
Please help.
Thanks,
Shouvanik
________________________________
This message is for the designated recipient only and may contain privileged, proprietary, or otherwise confidential information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of the e-mail by you is prohibited.
Where allowed by local law, electronic communications with Accenture and its affiliates, including e-mail and instant messaging (including content), may be scanned by our systems for the purposes of information security and assessment of internal compliance with Accenture policy.
______________________________________________________________________________________
www.accenture.com<http://www.accenture.com/>
RE: error executing the tutorial ->
https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
Posted by sh...@accenture.com.
Hi,
Is it possible to permanently set the below parameters? Please help.
Thanks,
SHouvanik
From: Jitendra Yadav [mailto:jeetuyadav200890@gmail.com]
Sent: Thursday, September 05, 2013 6:02 PM
To: user@hive.apache.org
Subject: Re: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
OK got your point, As an alternative you can set these variables in hive environment before creating the table
Hive > set fs.s3n.awsSecretAccessKey='value'
Hive > set fs.s3n.awsAccessKeyId='Value'
Hope this will help you.
Thanks
Jitendra
On Thu, Sep 5, 2013 at 5:19 PM, <sh...@accenture.com>> wrote:
Hi,
Thanks for the reply.
I had seen in some posts, that you could mention these keys on the Hive CLI. If things work, then I am later planning to include it in core-site.xml of Hadoop config. Presently, I am not allowed to restart Hadoop services.
"create external table ....
.....
LOCATION 's3n:// <fs.s3n.awsAccessKeyId: fs.s3n>.<awsSecretAccessKey >@ap-hw-dev/service-names';
Please help me.
Thanks,
Shouvanik
From: Jitendra Yadav [mailto:jeetuyadav200890@gmail.com<ma...@gmail.com>]
Sent: Thursday, September 05, 2013 5:11 PM
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: Re: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
Hi,
Did you configured fs.s3n.awsAccessKeyId or fs.s3n.awsSecretAccessKey properties
in your core-site.xml file?
Regards
Jitendra
On Thu, Sep 5, 2013 at 3:52 PM, <sh...@accenture.com>> wrote:
Hi,
I am executing the example, but got initial roadblock while trying out "Setting up tables (DDL Statements)" section.
When I executed the command "hive> create external table kv (key int, values string) location 's3n://data.s3ndemo.hive/kv';"
I got the error
FAILED: Error in metadata: MetaException(message:java.lang.IllegalArgumentException: AWS Access Key ID and Secret Access Key must be specified as the username or password (respectively) of a s3n URL, or by setting the fs.s3n.awsAccessKeyId or fs.s3n.awsSecretAccessKey properties (respectively).)
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
hive>
Please help.
Thanks,
Shouvanik
________________________________
This message is for the designated recipient only and may contain privileged, proprietary, or otherwise confidential information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of the e-mail by you is prohibited.
Where allowed by local law, electronic communications with Accenture and its affiliates, including e-mail and instant messaging (including content), may be scanned by our systems for the purposes of information security and assessment of internal compliance with Accenture policy.
______________________________________________________________________________________
www.accenture.com<http://www.accenture.com/>
Re: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
Posted by Jitendra Yadav <je...@gmail.com>.
OK got your point, As an alternative you can set these variables in hive
environment before creating the table
Hive > set fs.s3n.awsSecretAccessKey='value'
Hive > set fs.s3n.awsAccessKeyId='Value'
Hope this will help you.
Thanks
Jitendra
On Thu, Sep 5, 2013 at 5:19 PM, <sh...@accenture.com> wrote:
> Hi,****
>
> ** **
>
> Thanks for the reply.****
>
> ** **
>
> I had seen in some posts, that you could mention these keys on the Hive
> CLI. If things work, then I am later planning to include it in
> core-site.xml of Hadoop config. Presently, I am not allowed to restart
> Hadoop services. ****
>
> ** **
>
> “create external table ….****
>
> …..****
>
> LOCATION 's3n:// <fs.s3n.awsAccessKeyId: fs.s3n>.<awsSecretAccessKey>@ap-hw-dev/service-names';
> ****
>
> ** **
>
> Please help me.****
>
> ** **
>
> Thanks,****
>
> Shouvanik****
>
> ** **
>
> *From:* Jitendra Yadav [mailto:jeetuyadav200890@gmail.com]
> *Sent:* Thursday, September 05, 2013 5:11 PM
> *To:* user@hive.apache.org
> *Subject:* Re: error executing the tutorial ->
> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
> ****
>
> ** **
>
> Hi,****
>
> ****
>
> Did you configured fs.s3n.awsAccessKeyId or fs.s3n.awsSecretAccessKey
> properties****
>
> in your core-site.xml file?****
>
> ****
>
> Regards****
>
> Jitendra****
>
> On Thu, Sep 5, 2013 at 3:52 PM, <sh...@accenture.com> wrote:***
> *
>
> Hi,****
>
> ****
>
> I am executing the example, but got initial roadblock while trying out
> “Setting up tables (DDL Statements)” section.****
>
> ****
>
> When I executed the command “hive> create external table kv (key int,
> values string) location 's3n://data.s3ndemo.hive/kv';”****
>
> ****
>
> I got the error****
>
> ****
>
> FAILED: Error in metadata:
> MetaException(message:java.lang.IllegalArgumentException: AWS Access Key ID
> and Secret Access Key must be specified as the username or password
> (respectively) of a s3n URL, or by setting the fs.s3n.awsAccessKeyId or
> fs.s3n.awsSecretAccessKey properties (respectively).)****
>
> FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.DDLTask****
>
> hive>****
>
> ****
>
> Please help.****
>
> ****
>
> ****
>
> Thanks,****
>
> Shouvanik****
>
> ** **
> ------------------------------
>
> This message is for the designated recipient only and may contain
> privileged, proprietary, or otherwise confidential information. If you have
> received it in error, please notify the sender immediately and delete the
> original. Any other use of the e-mail by you is prohibited.
>
> Where allowed by local law, electronic communications with Accenture and
> its affiliates, including e-mail and instant messaging (including content),
> may be scanned by our systems for the purposes of information security and
> assessment of internal compliance with Accenture policy.
>
>
> ______________________________________________________________________________________
>
> www.accenture.com****
>
> ** **
>
RE: error executing the tutorial ->
https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
Posted by sh...@accenture.com.
Hi,
Thanks for the reply.
I had seen in some posts, that you could mention these keys on the Hive CLI. If things work, then I am later planning to include it in core-site.xml of Hadoop config. Presently, I am not allowed to restart Hadoop services.
"create external table ....
.....
LOCATION 's3n:// <fs.s3n.awsAccessKeyId: fs.s3n>.<awsSecretAccessKey >@ap-hw-dev/service-names';
Please help me.
Thanks,
Shouvanik
From: Jitendra Yadav [mailto:jeetuyadav200890@gmail.com]
Sent: Thursday, September 05, 2013 5:11 PM
To: user@hive.apache.org
Subject: Re: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
Hi,
Did you configured fs.s3n.awsAccessKeyId or fs.s3n.awsSecretAccessKey properties
in your core-site.xml file?
Regards
Jitendra
On Thu, Sep 5, 2013 at 3:52 PM, <sh...@accenture.com>> wrote:
Hi,
I am executing the example, but got initial roadblock while trying out "Setting up tables (DDL Statements)" section.
When I executed the command "hive> create external table kv (key int, values string) location 's3n://data.s3ndemo.hive/kv';"
I got the error
FAILED: Error in metadata: MetaException(message:java.lang.IllegalArgumentException: AWS Access Key ID and Secret Access Key must be specified as the username or password (respectively) of a s3n URL, or by setting the fs.s3n.awsAccessKeyId or fs.s3n.awsSecretAccessKey properties (respectively).)
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
hive>
Please help.
Thanks,
Shouvanik
________________________________
This message is for the designated recipient only and may contain privileged, proprietary, or otherwise confidential information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of the e-mail by you is prohibited.
Where allowed by local law, electronic communications with Accenture and its affiliates, including e-mail and instant messaging (including content), may be scanned by our systems for the purposes of information security and assessment of internal compliance with Accenture policy.
______________________________________________________________________________________
www.accenture.com<http://www.accenture.com/>
Re: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
Posted by Jitendra Yadav <je...@gmail.com>.
Hi,
Did you configured fs.s3n.awsAccessKeyId or fs.s3n.awsSecretAccessKey
properties
in your core-site.xml file?
Regards
Jitendra
On Thu, Sep 5, 2013 at 3:52 PM, <sh...@accenture.com> wrote:
> Hi,****
>
> ** **
>
> I am executing the example, but got initial roadblock while trying out
> “Setting up tables (DDL Statements)” section.****
>
> ** **
>
> When I executed the command “hive> create external table kv (key int,
> values string) location 's3n://data.s3ndemo.hive/kv';”****
>
> ** **
>
> I got the error****
>
> ** **
>
> FAILED: Error in metadata:
> MetaException(message:java.lang.IllegalArgumentException: AWS Access Key ID
> and Secret Access Key must be specified as the username or password
> (respectively) of a s3n URL, or by setting the fs.s3n.awsAccessKeyId or
> fs.s3n.awsSecretAccessKey properties (respectively).)****
>
> FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.DDLTask****
>
> hive>****
>
> ** **
>
> Please help.****
>
> ** **
>
> ** **
>
> Thanks,****
>
> Shouvanik****
>
> ------------------------------
> This message is for the designated recipient only and may contain
> privileged, proprietary, or otherwise confidential information. If you have
> received it in error, please notify the sender immediately and delete the
> original. Any other use of the e-mail by you is prohibited.
>
> Where allowed by local law, electronic communications with Accenture and
> its affiliates, including e-mail and instant messaging (including content),
> may be scanned by our systems for the purposes of information security and
> assessment of internal compliance with Accenture policy.
>
>
> ______________________________________________________________________________________
>
> www.accenture.com
>