You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-user@hadoop.apache.org by Artem Ervits <ar...@nyp.org> on 2012/08/08 20:34:23 UTC
Setting up HTTP authentication
Hello all,
I followed the 1.0.3 docs to setup http simple authentication. I'd like to pass a username to open the web interface. My new settings for core-site.xml are below:
<property>
<name>hadoop.http.filter.initializers</name>
<value>org.apache.hadoop.http.lib.StaticUserWebFilter,org.apache.hadoop.security.AuthenticationFilterInitializer</value>
<description>HTTP Authentication document in hadoop tar file</description>
</property>
<property>
<name>hadoop.http.authentication.type</name>
<value>simple</value>
<description>authentication type for web UI</description>
</property>
<property>
<name>hadoop.http.authentication.token.validity</name>
<value>36000</value>
<description>how long authentication token is valid before it needs to be renewed</description>
</property>
<property>
<name>hadoop.http.authentication.signature.secret</name>
<value>test</value>
<description>signature secret for signing authentication tokens</description>
</property>
<property>
<name>hadoop.http.authentication.cookie.domain</name>
<value></value>
<description>domain to use for the http cookie that stores authentication token</description>
</property>
<property>
<name>hadoop.http.authentication.simple.anonymous.allowed</name>
<value>false</value>
<description>anonymous web UI requests enabled or disabled</description>
</property>
Unfortunately, I don't have a 1.0.3 hadoop cluster handy and am testing this on 0.20.203.0. When I start the cluster, jobtracker doesn't start. Looking at the logs, I see the following:
tasktracker log:
2012-08-08 13:40:26,178 WARN org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Source name ugi already exists!
and
2012-08-08 13:40:26,445 ERROR org.apache.hadoop.mapred.TaskTracker: Can not start task tracker because java.lang.RuntimeException: java.lang.ClassNotFoundException: org.apache.hadoop.security.AuthenticationFilterInitializer
Namenode log shows:
2012-08-08 14:18:17,839 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: master/10.0.0.101:54310. Already tried 9 time(s).
2012-08-08 14:18:17,839 INFO org.apache.hadoop.ipc.RPC: Server at master/10.0.0.101:54310 not available yet, Zzzzz...
2012-08-08 14:18:19,841 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: master/10.0.0.101:54310. Already tried 0 time(s).
My question is, is the org.apache.hadoop.security.AuthenticationFilterInitiazlier only available in 1.* releases or the problem is somewhere else?
Thank you.
Artem Ervits
Data Analyst
New York Presbyterian Hospital
________________________________
This electronic message is intended to be for the use only of the named recipient, and may contain information that is confidential or privileged. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited. If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message. Thank you.
--------------------
This electronic message is intended to be for the use only of the named recipient, and may contain information that is confidential or privileged. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited. If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message. Thank you.
--------------------
This electronic message is intended to be for the use only of the named recipient, and may contain information that is confidential or privileged. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited. If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message. Thank you.
RE: Setting up HTTP authentication
Posted by Artem Ervits <ar...@nyp.org>.
This was a version dependency issue. The class is not in 0.20.203.0.
From: Artem Ervits [mailto:are9004@nyp.org]
Sent: Wednesday, August 08, 2012 2:34 PM
To: user@hadoop.apache.org
Subject: Setting up HTTP authentication
Hello all,
I followed the 1.0.3 docs to setup http simple authentication. I'd like to pass a username to open the web interface. My new settings for core-site.xml are below:
<property>
<name>hadoop.http.filter.initializers</name>
<value>org.apache.hadoop.http.lib.StaticUserWebFilter,org.apache.hadoop.security.AuthenticationFilterInitializer</value>
<description>HTTP Authentication document in hadoop tar file</description>
</property>
<property>
<name>hadoop.http.authentication.type</name>
<value>simple</value>
<description>authentication type for web UI</description>
</property>
<property>
<name>hadoop.http.authentication.token.validity</name>
<value>36000</value>
<description>how long authentication token is valid before it needs to be renewed</description>
</property>
<property>
<name>hadoop.http.authentication.signature.secret</name>
<value>test</value>
<description>signature secret for signing authentication tokens</description>
</property>
<property>
<name>hadoop.http.authentication.cookie.domain</name>
<value></value>
<description>domain to use for the http cookie that stores authentication token</description>
</property>
<property>
<name>hadoop.http.authentication.simple.anonymous.allowed</name>
<value>false</value>
<description>anonymous web UI requests enabled or disabled</description>
</property>
Unfortunately, I don't have a 1.0.3 hadoop cluster handy and am testing this on 0.20.203.0. When I start the cluster, jobtracker doesn't start. Looking at the logs, I see the following:
tasktracker log:
2012-08-08 13:40:26,178 WARN org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Source name ugi already exists!
and
2012-08-08 13:40:26,445 ERROR org.apache.hadoop.mapred.TaskTracker: Can not start task tracker because java.lang.RuntimeException: java.lang.ClassNotFoundException: org.apache.hadoop.security.AuthenticationFilterInitializer
Namenode log shows:
2012-08-08 14:18:17,839 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: master/10.0.0.101:54310. Already tried 9 time(s).
2012-08-08 14:18:17,839 INFO org.apache.hadoop.ipc.RPC: Server at master/10.0.0.101:54310 not available yet, Zzzzz...
2012-08-08 14:18:19,841 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: master/10.0.0.101:54310. Already tried 0 time(s).
My question is, is the org.apache.hadoop.security.AuthenticationFilterInitiazlier only available in 1.* releases or the problem is somewhere else?
Thank you.
Artem Ervits
Data Analyst
New York Presbyterian Hospital
________________________________
This electronic message is intended to be for the use only of the named recipient, and may contain information that is confidential or privileged. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited. If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message. Thank you.
--------------------
This electronic message is intended to be for the use only of the named recipient, and may contain information that is confidential or privileged. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited. If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message. Thank you.
--------------------
This electronic message is intended to be for the use only of the named recipient, and may contain information that is confidential or privileged. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited. If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message. Thank you.
________________________________
Confidential Information subject to NYP's (and its affiliates') information management and security policies (http://infonet.nyp.org/QA/HospitalManual).
--------------------
This electronic message is intended to be for the use only of the named recipient, and may contain information that is confidential or privileged. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited. If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message. Thank you.
--------------------
This electronic message is intended to be for the use only of the named recipient, and may contain information that is confidential or privileged. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited. If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message. Thank you.
RE: Setting up HTTP authentication
Posted by Artem Ervits <ar...@nyp.org>.
This was a version dependency issue. The class is not in 0.20.203.0.
From: Artem Ervits [mailto:are9004@nyp.org]
Sent: Wednesday, August 08, 2012 2:34 PM
To: user@hadoop.apache.org
Subject: Setting up HTTP authentication
Hello all,
I followed the 1.0.3 docs to setup http simple authentication. I'd like to pass a username to open the web interface. My new settings for core-site.xml are below:
<property>
<name>hadoop.http.filter.initializers</name>
<value>org.apache.hadoop.http.lib.StaticUserWebFilter,org.apache.hadoop.security.AuthenticationFilterInitializer</value>
<description>HTTP Authentication document in hadoop tar file</description>
</property>
<property>
<name>hadoop.http.authentication.type</name>
<value>simple</value>
<description>authentication type for web UI</description>
</property>
<property>
<name>hadoop.http.authentication.token.validity</name>
<value>36000</value>
<description>how long authentication token is valid before it needs to be renewed</description>
</property>
<property>
<name>hadoop.http.authentication.signature.secret</name>
<value>test</value>
<description>signature secret for signing authentication tokens</description>
</property>
<property>
<name>hadoop.http.authentication.cookie.domain</name>
<value></value>
<description>domain to use for the http cookie that stores authentication token</description>
</property>
<property>
<name>hadoop.http.authentication.simple.anonymous.allowed</name>
<value>false</value>
<description>anonymous web UI requests enabled or disabled</description>
</property>
Unfortunately, I don't have a 1.0.3 hadoop cluster handy and am testing this on 0.20.203.0. When I start the cluster, jobtracker doesn't start. Looking at the logs, I see the following:
tasktracker log:
2012-08-08 13:40:26,178 WARN org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Source name ugi already exists!
and
2012-08-08 13:40:26,445 ERROR org.apache.hadoop.mapred.TaskTracker: Can not start task tracker because java.lang.RuntimeException: java.lang.ClassNotFoundException: org.apache.hadoop.security.AuthenticationFilterInitializer
Namenode log shows:
2012-08-08 14:18:17,839 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: master/10.0.0.101:54310. Already tried 9 time(s).
2012-08-08 14:18:17,839 INFO org.apache.hadoop.ipc.RPC: Server at master/10.0.0.101:54310 not available yet, Zzzzz...
2012-08-08 14:18:19,841 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: master/10.0.0.101:54310. Already tried 0 time(s).
My question is, is the org.apache.hadoop.security.AuthenticationFilterInitiazlier only available in 1.* releases or the problem is somewhere else?
Thank you.
Artem Ervits
Data Analyst
New York Presbyterian Hospital
________________________________
This electronic message is intended to be for the use only of the named recipient, and may contain information that is confidential or privileged. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited. If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message. Thank you.
--------------------
This electronic message is intended to be for the use only of the named recipient, and may contain information that is confidential or privileged. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited. If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message. Thank you.
--------------------
This electronic message is intended to be for the use only of the named recipient, and may contain information that is confidential or privileged. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited. If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message. Thank you.
________________________________
Confidential Information subject to NYP's (and its affiliates') information management and security policies (http://infonet.nyp.org/QA/HospitalManual).
--------------------
This electronic message is intended to be for the use only of the named recipient, and may contain information that is confidential or privileged. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited. If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message. Thank you.
--------------------
This electronic message is intended to be for the use only of the named recipient, and may contain information that is confidential or privileged. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited. If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message. Thank you.
RE: Setting up HTTP authentication
Posted by Artem Ervits <ar...@nyp.org>.
This was a version dependency issue. The class is not in 0.20.203.0.
From: Artem Ervits [mailto:are9004@nyp.org]
Sent: Wednesday, August 08, 2012 2:34 PM
To: user@hadoop.apache.org
Subject: Setting up HTTP authentication
Hello all,
I followed the 1.0.3 docs to setup http simple authentication. I'd like to pass a username to open the web interface. My new settings for core-site.xml are below:
<property>
<name>hadoop.http.filter.initializers</name>
<value>org.apache.hadoop.http.lib.StaticUserWebFilter,org.apache.hadoop.security.AuthenticationFilterInitializer</value>
<description>HTTP Authentication document in hadoop tar file</description>
</property>
<property>
<name>hadoop.http.authentication.type</name>
<value>simple</value>
<description>authentication type for web UI</description>
</property>
<property>
<name>hadoop.http.authentication.token.validity</name>
<value>36000</value>
<description>how long authentication token is valid before it needs to be renewed</description>
</property>
<property>
<name>hadoop.http.authentication.signature.secret</name>
<value>test</value>
<description>signature secret for signing authentication tokens</description>
</property>
<property>
<name>hadoop.http.authentication.cookie.domain</name>
<value></value>
<description>domain to use for the http cookie that stores authentication token</description>
</property>
<property>
<name>hadoop.http.authentication.simple.anonymous.allowed</name>
<value>false</value>
<description>anonymous web UI requests enabled or disabled</description>
</property>
Unfortunately, I don't have a 1.0.3 hadoop cluster handy and am testing this on 0.20.203.0. When I start the cluster, jobtracker doesn't start. Looking at the logs, I see the following:
tasktracker log:
2012-08-08 13:40:26,178 WARN org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Source name ugi already exists!
and
2012-08-08 13:40:26,445 ERROR org.apache.hadoop.mapred.TaskTracker: Can not start task tracker because java.lang.RuntimeException: java.lang.ClassNotFoundException: org.apache.hadoop.security.AuthenticationFilterInitializer
Namenode log shows:
2012-08-08 14:18:17,839 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: master/10.0.0.101:54310. Already tried 9 time(s).
2012-08-08 14:18:17,839 INFO org.apache.hadoop.ipc.RPC: Server at master/10.0.0.101:54310 not available yet, Zzzzz...
2012-08-08 14:18:19,841 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: master/10.0.0.101:54310. Already tried 0 time(s).
My question is, is the org.apache.hadoop.security.AuthenticationFilterInitiazlier only available in 1.* releases or the problem is somewhere else?
Thank you.
Artem Ervits
Data Analyst
New York Presbyterian Hospital
________________________________
This electronic message is intended to be for the use only of the named recipient, and may contain information that is confidential or privileged. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited. If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message. Thank you.
--------------------
This electronic message is intended to be for the use only of the named recipient, and may contain information that is confidential or privileged. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited. If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message. Thank you.
--------------------
This electronic message is intended to be for the use only of the named recipient, and may contain information that is confidential or privileged. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited. If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message. Thank you.
________________________________
Confidential Information subject to NYP's (and its affiliates') information management and security policies (http://infonet.nyp.org/QA/HospitalManual).
--------------------
This electronic message is intended to be for the use only of the named recipient, and may contain information that is confidential or privileged. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited. If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message. Thank you.
--------------------
This electronic message is intended to be for the use only of the named recipient, and may contain information that is confidential or privileged. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited. If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message. Thank you.
RE: Setting up HTTP authentication
Posted by Artem Ervits <ar...@nyp.org>.
This was a version dependency issue. The class is not in 0.20.203.0.
From: Artem Ervits [mailto:are9004@nyp.org]
Sent: Wednesday, August 08, 2012 2:34 PM
To: user@hadoop.apache.org
Subject: Setting up HTTP authentication
Hello all,
I followed the 1.0.3 docs to setup http simple authentication. I'd like to pass a username to open the web interface. My new settings for core-site.xml are below:
<property>
<name>hadoop.http.filter.initializers</name>
<value>org.apache.hadoop.http.lib.StaticUserWebFilter,org.apache.hadoop.security.AuthenticationFilterInitializer</value>
<description>HTTP Authentication document in hadoop tar file</description>
</property>
<property>
<name>hadoop.http.authentication.type</name>
<value>simple</value>
<description>authentication type for web UI</description>
</property>
<property>
<name>hadoop.http.authentication.token.validity</name>
<value>36000</value>
<description>how long authentication token is valid before it needs to be renewed</description>
</property>
<property>
<name>hadoop.http.authentication.signature.secret</name>
<value>test</value>
<description>signature secret for signing authentication tokens</description>
</property>
<property>
<name>hadoop.http.authentication.cookie.domain</name>
<value></value>
<description>domain to use for the http cookie that stores authentication token</description>
</property>
<property>
<name>hadoop.http.authentication.simple.anonymous.allowed</name>
<value>false</value>
<description>anonymous web UI requests enabled or disabled</description>
</property>
Unfortunately, I don't have a 1.0.3 hadoop cluster handy and am testing this on 0.20.203.0. When I start the cluster, jobtracker doesn't start. Looking at the logs, I see the following:
tasktracker log:
2012-08-08 13:40:26,178 WARN org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Source name ugi already exists!
and
2012-08-08 13:40:26,445 ERROR org.apache.hadoop.mapred.TaskTracker: Can not start task tracker because java.lang.RuntimeException: java.lang.ClassNotFoundException: org.apache.hadoop.security.AuthenticationFilterInitializer
Namenode log shows:
2012-08-08 14:18:17,839 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: master/10.0.0.101:54310. Already tried 9 time(s).
2012-08-08 14:18:17,839 INFO org.apache.hadoop.ipc.RPC: Server at master/10.0.0.101:54310 not available yet, Zzzzz...
2012-08-08 14:18:19,841 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: master/10.0.0.101:54310. Already tried 0 time(s).
My question is, is the org.apache.hadoop.security.AuthenticationFilterInitiazlier only available in 1.* releases or the problem is somewhere else?
Thank you.
Artem Ervits
Data Analyst
New York Presbyterian Hospital
________________________________
This electronic message is intended to be for the use only of the named recipient, and may contain information that is confidential or privileged. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited. If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message. Thank you.
--------------------
This electronic message is intended to be for the use only of the named recipient, and may contain information that is confidential or privileged. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited. If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message. Thank you.
--------------------
This electronic message is intended to be for the use only of the named recipient, and may contain information that is confidential or privileged. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited. If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message. Thank you.
________________________________
Confidential Information subject to NYP's (and its affiliates') information management and security policies (http://infonet.nyp.org/QA/HospitalManual).
--------------------
This electronic message is intended to be for the use only of the named recipient, and may contain information that is confidential or privileged. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited. If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message. Thank you.
--------------------
This electronic message is intended to be for the use only of the named recipient, and may contain information that is confidential or privileged. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited. If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message. Thank you.