答复: hadoop: command not found

[liujs3 <li...@asiainfo.com>]

I set hadoop_home hadoop_classpath to path in /etc/profile, this is all user use it?

It work!

Is appConfig.json set hbase user?
Like site.hbase-site.hbase.superuser:ocean?
But I try this ,it no work.



-----邮件原件-----
发件人: Steve Loughran [mailto:stevel@hortonworks.com] 
发送时间: 2014年12月9日 18:13
收件人: Slider Dev
主题: Re: hadoop: command not found

On 9 December 2014 at 08:22, liujs3 <li...@asiainfo.com> wrote:

> 1.in our product-line,we used hadoop2.2,no hadoop2.4 ,It mean we cant 
> use slider?
>

The latest (0.60) branch requires hadoop 2.6. That may seem very incompatible with shipping releases, but a lot of work in 2.6 went in to supporting long-lived applications like Slider —we are one of the first users of those features.


>
> 2.when I install slider 0.4.0 I found this log in hamster container.it 
> doesn’t work.
>   my config ---slider-client.xml  yarn.application.classpath is right, 
> it work in slider0.6.0.
>   my system config profile hadoop_home hadoop_classpath 
> hadoop_conf_dir is right ,
>   I cant find any config about hadoop. This problem found in slider-0.30.
>   In my system shell run hadoop is right.
>   I found a file slider-env.sh after slider-0.4.0,why the slider-0.4.0 
> no use it.
>
>
>

> Fail: Execution of 'hadoop fs -mkdir `rpm -q hadoop | grep -q "hadoop-1"
> || echo "-p"` 
> || hdfs://OCNoSQLBJ/user/ocean/.slider/cluster/s48/database/data
> hdfs://OCNoSQLBJ/user/ocean/.slider/cluster/s48/database/staging && 
> hadoop fs -chmod  711 
> hdfs://OCNoSQLBJ/user/ocean/.slider/cluster/s48/database/staging' 
> returned 127. /bin/bash: hadoop: command not found
>
>
It looks like the 'hadoop' command isn't on the PATH. This is surprising as the the node manager would need to have the yarn command on its PATH to start, and bin/hadoop is usually alongside this.

If you log in to the machine, what does "hadoop -version" say?

--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.

Re: 答复: hadoop: command not found

[Steve Loughran <st...@hortonworks.com>]

On 11 December 2014 at 10:56, liujs3 <li...@asiainfo.com> wrote:

> Is appConfig.json set hbase user?
> Like site.hbase-site.hbase.superuser:ocean?
>

the user that an app runs on is the user that YARN starts applications on.

1. on an insecure cluster, it will start as a user with the name "mapred"
or "yarn" (usually, if you install ASF hadoop yourself, it runs as whoever
starts the YARN node managers. We pass an environment variable down,
HADOOP_USER, which is automatically picked up by the standard Hadoop HDFS
client to set the username of the user talking to HDFS.

This means that although it is running as a user called "yarn", it is
talking to HDFS as the user who started the job.

2. On an secure cluster, YARN requires a cluster account (i.e. unix or
windows) for every user submitting a job. The client authenticates using
Kerberos, includes the kerberos keytab needed to allow the deployed service
to talk to HDFS and other services (Zookeeper, for example). Slider and its
deployed containers run as you. The keytab is also used by HBase to
authenticate with HDFS.

Summary: if you want isolated per-user applications, you need to run on a
secure cluster. On an insecure cluster, anyone has access to the hbase data
anyway, because HDFS is readable by all.

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.