You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Juan Carlos <ju...@gmail.com> on 2014/11/05 09:45:56 UTC
Hive-server2 trying to execute local mapreduce
I have a secured and HA hdfs cluster, and I have been trying to execute a
join operation with beeline CLI.
My issue is that it try to execute mapreduce localy instead by yarn. I set
parameters
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
<property>
<name>mapred.job.tracker</name>
<value>anything</value>
</property>
I'm using hive 0.13 and hadoop 2.2.0
In logs I see this:
ERROR [pool-2-thread-2]: mr.MapredLocalTask
(MapredLocalTask.java:execute(282)) - Exception: Cannot run program
"/usr/lib/hadoop/bin/hadoop" (in directory "/root"): error=13, Permission
denied
2014-11-05 09:31:33,368 ERROR [pool-2-thread-2]: ql.Driver
(SessionState.java:printError(545)) - FAILED: Execution Error, return code
1 from org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask
2014-11-05 09:31:33,368 INFO [pool-2-thread-2]: log.PerfLogger
(PerfLogger.java:PerfLogEnd(135)) - </PERFLOG method=Driver.execute
start=1415176292766 end=1415176293368 duration=602
from=org.apache.hadoop.hive.ql.Driver>
2014-11-05 09:31:33,368 INFO [pool-2-thread-2]: log.PerfLogger
(PerfLogger.java:PerfLogBegin(108)) - <PERFLOG method=releaseLocks
from=org.apache.hadoop.hive.ql.Driver>
2014-11-05 09:31:33,368 INFO [pool-2-thread-2]: ZooKeeperHiveLockManager
(ZooKeeperHiveLockManager.java:releaseLocks(254)) - about to release lock
for default/sample_08
2014-11-05 09:31:33,429 INFO [pool-2-thread-2]: ZooKeeperHiveLockManager
(ZooKeeperHiveLockManager.java:releaseLocks(254)) - about to release lock
for default/sample_07
2014-11-05 09:31:33,514 INFO [pool-2-thread-2]: ZooKeeperHiveLockManager
(ZooKeeperHiveLockManager.java:releaseLocks(254)) - about to release lock
for default
2014-11-05 09:31:33,675 INFO [pool-2-thread-2]: log.PerfLogger
(PerfLogger.java:PerfLogEnd(135)) - </PERFLOG method=releaseLocks
start=1415176293368 end=1415176293675 duration=307
from=org.apache.hadoop.hive.ql.Driver>
2014-11-05 09:31:33,815 ERROR [pool-2-thread-2]: operation.Operation
(SQLOperation.java:run(202)) - Error running hive query:
org.apache.hive.service.cli.HiveSQLException: Error while processing
statement: FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask
at
org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:146)
at
org.apache.hive.service.cli.operation.SQLOperation.access$000(SQLOperation.java:68)
at
org.apache.hive.service.cli.operation.SQLOperation$1$1.run(SQLOperation.java:199)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
at
org.apache.hadoop.hive.shims.HadoopShimsSecure.doAs(HadoopShimsSecure.java:493)
at
org.apache.hive.service.cli.operation.SQLOperation$1.run(SQLOperation.java:208)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Any idea? Anyone could point me what else to check?
Regards
Re: Hive-server2 trying to execute local mapreduce
Posted by Juan Carlos <ju...@gmail.com>.
My problem was that I configured in hive-site.xml this:
<property>
<name>hive.exec.mode.local.auto</name>
<value>true</value>
</property>
After setting it to false it works fine
2014-11-06 12:21 GMT+01:00 Juan Carlos <ju...@gmail.com>:
> Thanks Matouk,
> I checked what you said, and finally I noticed that it's not the same
> running "/etc/init.d/hive-server2 start" than "service hive-server2 start".
> In the first case the CWD are being used by the script and it was causing
> my issue.
>
> Now I'm having another issue, after submiting the join query, I find in
> logs this:
>
> ERROR [main]: security.UserGroupInformation
> (UserGroupInformation.java:doAs(1494)) - PriviledgedActionException as:hive/
> huesec8.dtardon.cediant.es@DTARDON.CEDIANT.ES (auth:KERBEROS)
> cause:ENOENT: No such file or directory
> 2014-11-06 11:53:47,165 ERROR [main]: security.UserGroupInformation
> (UserGroupInformation.java:doAs(1494)) - PriviledgedActionException as:hive/
> huesec8.dtardon.cediant.es@DTARDON.CEDIANT.ES (auth:KERBEROS)
> cause:ENOENT: No such file or directory
> 2014-11-06 11:53:47,176 ERROR [main]: mr.ExecDriver
> (SessionState.java:printError(545)) - Job Submission failed with exception
> 'org.apache.hadoop.io.nativeio.NativeIOException(No such file or directory)'
> ENOENT: No such file or directory
> at org.apache.hadoop.io.nativeio.NativeIO$POSIX.chmodImpl(Native
> Method)
> at
> org.apache.hadoop.io.nativeio.NativeIO$POSIX.chmod(NativeIO.java:158)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:635)
> at
> org.apache.hadoop.fs.FilterFileSystem.setPermission(FilterFileSystem.java:468)
> at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:596)
> at
> org.apache.hadoop.mapreduce.JobSubmitter.copyAndConfigureFiles(JobSubmitter.java:178)
> at
> org.apache.hadoop.mapreduce.JobSubmitter.copyAndConfigureFiles(JobSubmitter.java:300)
> at
> org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:387)
> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1268)
> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1265)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
> at org.apache.hadoop.mapreduce.Job.submit(Job.java:1265)
> at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:562)
> at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:557)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
> at
> org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:557)
> at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:548)
> at
> org.apache.hadoop.hive.ql.exec.mr.ExecDriver.execute(ExecDriver.java:420)
> at
> org.apache.hadoop.hive.ql.exec.mr.ExecDriver.main(ExecDriver.java:740)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
>
> I already tried to send a job to yarn with hive user and it run
> successfully.
> Any idea?
>
>
>
>
>
>
> 2014-11-05 10:28 GMT+01:00 Matouk IFTISSEN <ma...@ysance.com>:
>
>> Hello Juan,
>> As you see the problem is come from the permissions roles, I had have
>> like this error before and pass it.
>> check and compare :
>>
>> 1. your hadoop installation is done as 'root' or an other user (if
>> this is the suoer user)?
>> 2. your hive execution (who -'user'- run hive script)
>> 3. the users in the 'container-executor.cfg' as you are in yarn mode
>>
>> Hope this helps you ;)
>>
>> 2014-11-05 9:45 GMT+01:00 Juan Carlos <ju...@gmail.com>:
>>
>>> I have a secured and HA hdfs cluster, and I have been trying to execute
>>> a join operation with beeline CLI.
>>>
>>> My issue is that it try to execute mapreduce localy instead by yarn. I
>>> set parameters
>>>
>>> <property>
>>> <name>mapreduce.framework.name</name>
>>> <value>yarn</value>
>>> </property>
>>>
>>> <property>
>>> <name>mapred.job.tracker</name>
>>> <value>anything</value>
>>> </property>
>>>
>>> I'm using hive 0.13 and hadoop 2.2.0
>>>
>>> In logs I see this:
>>> ERROR [pool-2-thread-2]: mr.MapredLocalTask
>>> (MapredLocalTask.java:execute(282)) - Exception: Cannot run program
>>> "/usr/lib/hadoop/bin/hadoop" (in directory "/root"): error=13, Permission
>>> denied
>>> 2014-11-05 09:31:33,368 ERROR [pool-2-thread-2]: ql.Driver
>>> (SessionState.java:printError(545)) - FAILED: Execution Error, return code
>>> 1 from org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask
>>> 2014-11-05 09:31:33,368 INFO [pool-2-thread-2]: log.PerfLogger
>>> (PerfLogger.java:PerfLogEnd(135)) - </PERFLOG method=Driver.execute
>>> start=1415176292766 end=1415176293368 duration=602
>>> from=org.apache.hadoop.hive.ql.Driver>
>>> 2014-11-05 09:31:33,368 INFO [pool-2-thread-2]: log.PerfLogger
>>> (PerfLogger.java:PerfLogBegin(108)) - <PERFLOG method=releaseLocks
>>> from=org.apache.hadoop.hive.ql.Driver>
>>> 2014-11-05 09:31:33,368 INFO [pool-2-thread-2]:
>>> ZooKeeperHiveLockManager (ZooKeeperHiveLockManager.java:releaseLocks(254))
>>> - about to release lock for default/sample_08
>>> 2014-11-05 09:31:33,429 INFO [pool-2-thread-2]:
>>> ZooKeeperHiveLockManager (ZooKeeperHiveLockManager.java:releaseLocks(254))
>>> - about to release lock for default/sample_07
>>> 2014-11-05 09:31:33,514 INFO [pool-2-thread-2]:
>>> ZooKeeperHiveLockManager (ZooKeeperHiveLockManager.java:releaseLocks(254))
>>> - about to release lock for default
>>> 2014-11-05 09:31:33,675 INFO [pool-2-thread-2]: log.PerfLogger
>>> (PerfLogger.java:PerfLogEnd(135)) - </PERFLOG method=releaseLocks
>>> start=1415176293368 end=1415176293675 duration=307
>>> from=org.apache.hadoop.hive.ql.Driver>
>>> 2014-11-05 09:31:33,815 ERROR [pool-2-thread-2]: operation.Operation
>>> (SQLOperation.java:run(202)) - Error running hive query:
>>> org.apache.hive.service.cli.HiveSQLException: Error while processing
>>> statement: FAILED: Execution Error, return code 1 from
>>> org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask
>>> at
>>> org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:146)
>>> at
>>> org.apache.hive.service.cli.operation.SQLOperation.access$000(SQLOperation.java:68)
>>> at
>>> org.apache.hive.service.cli.operation.SQLOperation$1$1.run(SQLOperation.java:199)
>>> at java.security.AccessController.doPrivileged(Native Method)
>>> at javax.security.auth.Subject.doAs(Subject.java:415)
>>> at
>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
>>> at
>>> org.apache.hadoop.hive.shims.HadoopShimsSecure.doAs(HadoopShimsSecure.java:493)
>>> at
>>> org.apache.hive.service.cli.operation.SQLOperation$1.run(SQLOperation.java:208)
>>> at
>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>>> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>>> at
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>>> at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>>> at java.lang.Thread.run(Thread.java:744)
>>>
>>>
>>> Any idea? Anyone could point me what else to check?
>>>
>>> Regards
>>>
>>
>>
>>
>> --
>> ---------------
>> Life and Relations are not binary
>>
>> *Matouk IFTISSEN | Consultant BI & Big Data[image:
>> http://www.ysance.com] *
>> 24 rue du sentier - 75002 Paris - www.ysance.com <http://www.ysance.com/>
>> Fax : +33 1 73 72 97 26
>> *Ysance sur* :*Twitter* <http://twitter.com/ysance>* | Facebook
>> <https://www.facebook.com/pages/Ysance/131036788697> | Google+
>> <https://plus.google.com/u/0/b/115710923959357341736/115710923959357341736/posts> | LinkedIn
>> <http://www.linkedin.com/company/ysance> | Newsletter
>> <http://www.ysance.com/nous-contacter.html>*
>> *Nos autres sites* : *ys4you* <http://wwww.ys4you.com/>* | labdecisionnel
>> <http://www.labdecisionnel.com/> | decrypt <http://decrypt.ysance.com/>*
>>
>
>
Re: Hive-server2 trying to execute local mapreduce
Posted by Juan Carlos <ju...@gmail.com>.
Thanks Matouk,
I checked what you said, and finally I noticed that it's not the same
running "/etc/init.d/hive-server2 start" than "service hive-server2 start".
In the first case the CWD are being used by the script and it was causing
my issue.
Now I'm having another issue, after submiting the join query, I find in
logs this:
ERROR [main]: security.UserGroupInformation
(UserGroupInformation.java:doAs(1494)) - PriviledgedActionException as:hive/
huesec8.dtardon.cediant.es@DTARDON.CEDIANT.ES (auth:KERBEROS) cause:ENOENT:
No such file or directory
2014-11-06 11:53:47,165 ERROR [main]: security.UserGroupInformation
(UserGroupInformation.java:doAs(1494)) - PriviledgedActionException as:hive/
huesec8.dtardon.cediant.es@DTARDON.CEDIANT.ES (auth:KERBEROS) cause:ENOENT:
No such file or directory
2014-11-06 11:53:47,176 ERROR [main]: mr.ExecDriver
(SessionState.java:printError(545)) - Job Submission failed with exception
'org.apache.hadoop.io.nativeio.NativeIOException(No such file or directory)'
ENOENT: No such file or directory
at org.apache.hadoop.io.nativeio.NativeIO$POSIX.chmodImpl(Native Method)
at org.apache.hadoop.io.nativeio.NativeIO$POSIX.chmod(NativeIO.java:158)
at
org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:635)
at
org.apache.hadoop.fs.FilterFileSystem.setPermission(FilterFileSystem.java:468)
at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:596)
at
org.apache.hadoop.mapreduce.JobSubmitter.copyAndConfigureFiles(JobSubmitter.java:178)
at
org.apache.hadoop.mapreduce.JobSubmitter.copyAndConfigureFiles(JobSubmitter.java:300)
at
org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:387)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1268)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1265)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1265)
at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:562)
at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:557)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
at
org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:557)
at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:548)
at
org.apache.hadoop.hive.ql.exec.mr.ExecDriver.execute(ExecDriver.java:420)
at
org.apache.hadoop.hive.ql.exec.mr.ExecDriver.main(ExecDriver.java:740)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
I already tried to send a job to yarn with hive user and it run
successfully.
Any idea?
2014-11-05 10:28 GMT+01:00 Matouk IFTISSEN <ma...@ysance.com>:
> Hello Juan,
> As you see the problem is come from the permissions roles, I had have like
> this error before and pass it.
> check and compare :
>
> 1. your hadoop installation is done as 'root' or an other user (if
> this is the suoer user)?
> 2. your hive execution (who -'user'- run hive script)
> 3. the users in the 'container-executor.cfg' as you are in yarn mode
>
> Hope this helps you ;)
>
> 2014-11-05 9:45 GMT+01:00 Juan Carlos <ju...@gmail.com>:
>
>> I have a secured and HA hdfs cluster, and I have been trying to execute a
>> join operation with beeline CLI.
>>
>> My issue is that it try to execute mapreduce localy instead by yarn. I
>> set parameters
>>
>> <property>
>> <name>mapreduce.framework.name</name>
>> <value>yarn</value>
>> </property>
>>
>> <property>
>> <name>mapred.job.tracker</name>
>> <value>anything</value>
>> </property>
>>
>> I'm using hive 0.13 and hadoop 2.2.0
>>
>> In logs I see this:
>> ERROR [pool-2-thread-2]: mr.MapredLocalTask
>> (MapredLocalTask.java:execute(282)) - Exception: Cannot run program
>> "/usr/lib/hadoop/bin/hadoop" (in directory "/root"): error=13, Permission
>> denied
>> 2014-11-05 09:31:33,368 ERROR [pool-2-thread-2]: ql.Driver
>> (SessionState.java:printError(545)) - FAILED: Execution Error, return code
>> 1 from org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask
>> 2014-11-05 09:31:33,368 INFO [pool-2-thread-2]: log.PerfLogger
>> (PerfLogger.java:PerfLogEnd(135)) - </PERFLOG method=Driver.execute
>> start=1415176292766 end=1415176293368 duration=602
>> from=org.apache.hadoop.hive.ql.Driver>
>> 2014-11-05 09:31:33,368 INFO [pool-2-thread-2]: log.PerfLogger
>> (PerfLogger.java:PerfLogBegin(108)) - <PERFLOG method=releaseLocks
>> from=org.apache.hadoop.hive.ql.Driver>
>> 2014-11-05 09:31:33,368 INFO [pool-2-thread-2]: ZooKeeperHiveLockManager
>> (ZooKeeperHiveLockManager.java:releaseLocks(254)) - about to release lock
>> for default/sample_08
>> 2014-11-05 09:31:33,429 INFO [pool-2-thread-2]: ZooKeeperHiveLockManager
>> (ZooKeeperHiveLockManager.java:releaseLocks(254)) - about to release lock
>> for default/sample_07
>> 2014-11-05 09:31:33,514 INFO [pool-2-thread-2]: ZooKeeperHiveLockManager
>> (ZooKeeperHiveLockManager.java:releaseLocks(254)) - about to release lock
>> for default
>> 2014-11-05 09:31:33,675 INFO [pool-2-thread-2]: log.PerfLogger
>> (PerfLogger.java:PerfLogEnd(135)) - </PERFLOG method=releaseLocks
>> start=1415176293368 end=1415176293675 duration=307
>> from=org.apache.hadoop.hive.ql.Driver>
>> 2014-11-05 09:31:33,815 ERROR [pool-2-thread-2]: operation.Operation
>> (SQLOperation.java:run(202)) - Error running hive query:
>> org.apache.hive.service.cli.HiveSQLException: Error while processing
>> statement: FAILED: Execution Error, return code 1 from
>> org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask
>> at
>> org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:146)
>> at
>> org.apache.hive.service.cli.operation.SQLOperation.access$000(SQLOperation.java:68)
>> at
>> org.apache.hive.service.cli.operation.SQLOperation$1$1.run(SQLOperation.java:199)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at javax.security.auth.Subject.doAs(Subject.java:415)
>> at
>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
>> at
>> org.apache.hadoop.hive.shims.HadoopShimsSecure.doAs(HadoopShimsSecure.java:493)
>> at
>> org.apache.hive.service.cli.operation.SQLOperation$1.run(SQLOperation.java:208)
>> at
>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>> at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>> at java.lang.Thread.run(Thread.java:744)
>>
>>
>> Any idea? Anyone could point me what else to check?
>>
>> Regards
>>
>
>
>
> --
> ---------------
> Life and Relations are not binary
>
> *Matouk IFTISSEN | Consultant BI & Big Data[image: http://www.ysance.com] *
> 24 rue du sentier - 75002 Paris - www.ysance.com <http://www.ysance.com/>
> Fax : +33 1 73 72 97 26
> *Ysance sur* :*Twitter* <http://twitter.com/ysance>* | Facebook
> <https://www.facebook.com/pages/Ysance/131036788697> | Google+
> <https://plus.google.com/u/0/b/115710923959357341736/115710923959357341736/posts> | LinkedIn
> <http://www.linkedin.com/company/ysance> | Newsletter
> <http://www.ysance.com/nous-contacter.html>*
> *Nos autres sites* : *ys4you* <http://wwww.ys4you.com/>* | labdecisionnel
> <http://www.labdecisionnel.com/> | decrypt <http://decrypt.ysance.com/>*
>
Re: Hive-server2 trying to execute local mapreduce
Posted by Matouk IFTISSEN <ma...@ysance.com>.
Hello Juan,
As you see the problem is come from the permissions roles, I had have like
this error before and pass it.
check and compare :
1. your hadoop installation is done as 'root' or an other user (if this
is the suoer user)?
2. your hive execution (who -'user'- run hive script)
3. the users in the 'container-executor.cfg' as you are in yarn mode
Hope this helps you ;)
2014-11-05 9:45 GMT+01:00 Juan Carlos <ju...@gmail.com>:
> I have a secured and HA hdfs cluster, and I have been trying to execute a
> join operation with beeline CLI.
>
> My issue is that it try to execute mapreduce localy instead by yarn. I set
> parameters
>
> <property>
> <name>mapreduce.framework.name</name>
> <value>yarn</value>
> </property>
>
> <property>
> <name>mapred.job.tracker</name>
> <value>anything</value>
> </property>
>
> I'm using hive 0.13 and hadoop 2.2.0
>
> In logs I see this:
> ERROR [pool-2-thread-2]: mr.MapredLocalTask
> (MapredLocalTask.java:execute(282)) - Exception: Cannot run program
> "/usr/lib/hadoop/bin/hadoop" (in directory "/root"): error=13, Permission
> denied
> 2014-11-05 09:31:33,368 ERROR [pool-2-thread-2]: ql.Driver
> (SessionState.java:printError(545)) - FAILED: Execution Error, return code
> 1 from org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask
> 2014-11-05 09:31:33,368 INFO [pool-2-thread-2]: log.PerfLogger
> (PerfLogger.java:PerfLogEnd(135)) - </PERFLOG method=Driver.execute
> start=1415176292766 end=1415176293368 duration=602
> from=org.apache.hadoop.hive.ql.Driver>
> 2014-11-05 09:31:33,368 INFO [pool-2-thread-2]: log.PerfLogger
> (PerfLogger.java:PerfLogBegin(108)) - <PERFLOG method=releaseLocks
> from=org.apache.hadoop.hive.ql.Driver>
> 2014-11-05 09:31:33,368 INFO [pool-2-thread-2]: ZooKeeperHiveLockManager
> (ZooKeeperHiveLockManager.java:releaseLocks(254)) - about to release lock
> for default/sample_08
> 2014-11-05 09:31:33,429 INFO [pool-2-thread-2]: ZooKeeperHiveLockManager
> (ZooKeeperHiveLockManager.java:releaseLocks(254)) - about to release lock
> for default/sample_07
> 2014-11-05 09:31:33,514 INFO [pool-2-thread-2]: ZooKeeperHiveLockManager
> (ZooKeeperHiveLockManager.java:releaseLocks(254)) - about to release lock
> for default
> 2014-11-05 09:31:33,675 INFO [pool-2-thread-2]: log.PerfLogger
> (PerfLogger.java:PerfLogEnd(135)) - </PERFLOG method=releaseLocks
> start=1415176293368 end=1415176293675 duration=307
> from=org.apache.hadoop.hive.ql.Driver>
> 2014-11-05 09:31:33,815 ERROR [pool-2-thread-2]: operation.Operation
> (SQLOperation.java:run(202)) - Error running hive query:
> org.apache.hive.service.cli.HiveSQLException: Error while processing
> statement: FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask
> at
> org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:146)
> at
> org.apache.hive.service.cli.operation.SQLOperation.access$000(SQLOperation.java:68)
> at
> org.apache.hive.service.cli.operation.SQLOperation$1$1.run(SQLOperation.java:199)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
> at
> org.apache.hadoop.hive.shims.HadoopShimsSecure.doAs(HadoopShimsSecure.java:493)
> at
> org.apache.hive.service.cli.operation.SQLOperation$1.run(SQLOperation.java:208)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:744)
>
>
> Any idea? Anyone could point me what else to check?
>
> Regards
>
--
---------------
Life and Relations are not binary
*Matouk IFTISSEN | Consultant BI & Big Data[image: http://www.ysance.com] *
24 rue du sentier - 75002 Paris - www.ysance.com <http://www.ysance.com/>
Fax : +33 1 73 72 97 26
*Ysance sur* :*Twitter* <http://twitter.com/ysance>* | Facebook
<https://www.facebook.com/pages/Ysance/131036788697> | Google+
<https://plus.google.com/u/0/b/115710923959357341736/115710923959357341736/posts>
| LinkedIn
<http://www.linkedin.com/company/ysance> | Newsletter
<http://www.ysance.com/nous-contacter.html>*
*Nos autres sites* : *ys4you* <http://wwww.ys4you.com/>* | labdecisionnel
<http://www.labdecisionnel.com/> | decrypt <http://decrypt.ysance.com/>*