You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Eric Yang (JIRA)" <ji...@apache.org> on 2011/08/31 01:06:09 UTC
[jira] [Created] (HADOOP-7596) Enable jsvc to work with Hadoop RPM
package
Enable jsvc to work with Hadoop RPM package
-------------------------------------------
Key: HADOOP-7596
URL: https://issues.apache.org/jira/browse/HADOOP-7596
Project: Hadoop Common
Issue Type: Bug
Components: build
Affects Versions: 0.20.204.0
Environment: Java 6, RedHat EL 5.6
Reporter: Eric Yang
Assignee: Eric Yang
Fix For: 0.20.205.0
For secure Hadoop 0.20.2xx cluster, datanode can only run with 32 bit jvm because Hadoop only packages 32 bit jsvc. The build process should download proper jsvc versions base on the build architecture. In addition, the shell script should be enhanced to locate hadoop jar files in the proper location.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7596) Enable jsvc to work with Hadoop
RPM package
Posted by "Eric Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13095476#comment-13095476 ]
Eric Yang commented on HADOOP-7596:
-----------------------------------
For Kerberos secured Hadoop, Sun Java is required. Java definition fix is to ensure that we limit the compatibility to Sun Java for now until Hadoop has been verified to work with other vendor's Java.
> Enable jsvc to work with Hadoop RPM package
> -------------------------------------------
>
> Key: HADOOP-7596
> URL: https://issues.apache.org/jira/browse/HADOOP-7596
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
> Affects Versions: 0.20.204.0
> Environment: Java 6, RedHat EL 5.6
> Reporter: Eric Yang
> Assignee: Eric Yang
> Fix For: 0.20.205.0
>
> Attachments: HADOOP-7596.patch
>
>
> For secure Hadoop 0.20.2xx cluster, datanode can only run with 32 bit jvm because Hadoop only packages 32 bit jsvc. The build process should download proper jsvc versions base on the build architecture. In addition, the shell script should be enhanced to locate hadoop jar files in the proper location.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7596) Enable jsvc to work with Hadoop
RPM package
Posted by "Bruno Mahé (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13096560#comment-13096560 ]
Bruno Mahé commented on HADOOP-7596:
------------------------------------
{noformat}
Bruno, Debian system uses .pid file to check for process aliveness. The filename generated by hadoop-daemon.sh can have discrepancy between the starting user and effective running user. This is why this fix only applies to Debian family. For redhat, it is using /var/lock/subsys to track process aliveness. Hence, in Redhat, there is no discrepancy for the pid file name verse the user that started the process because the lock is a fixed name.
{noformat}
I am not sure to follow what you mean by "track process aliveness". Lock files and pid files have different purposes.
And as far as I can tell from https://github.com/apache/hadoop-common/blob/branch-0.20-security-205/src/packages/rpm/init.d/hadoop-datanode you use $PIDFILE to check aliveness where PIDFILE="${HADOOP_PID_DIR}/hadoop-hdfs-datanode.pid".
Does that mean a secure cluster on redhat does not have all the issues debian platform have? (dropping privileges and changing user...)
> Enable jsvc to work with Hadoop RPM package
> -------------------------------------------
>
> Key: HADOOP-7596
> URL: https://issues.apache.org/jira/browse/HADOOP-7596
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
> Affects Versions: 0.20.204.0
> Environment: Java 6, RedHat EL 5.6
> Reporter: Eric Yang
> Assignee: Eric Yang
> Fix For: 0.20.205.0
>
> Attachments: HADOOP-7596-2.patch, HADOOP-7596-3.patch, HADOOP-7596.patch
>
>
> For secure Hadoop 0.20.2xx cluster, datanode can only run with 32 bit jvm because Hadoop only packages 32 bit jsvc. The build process should download proper jsvc versions base on the build architecture. In addition, the shell script should be enhanced to locate hadoop jar files in the proper location.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7596) Enable jsvc to work with Hadoop
RPM package
Posted by "Eric Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13095548#comment-13095548 ]
Eric Yang commented on HADOOP-7596:
-----------------------------------
Dependency on openjdk was a mistake on my part. HDFS-2192 and MAPREDUCE-2728 have patch to address this problem for trunk.
> Enable jsvc to work with Hadoop RPM package
> -------------------------------------------
>
> Key: HADOOP-7596
> URL: https://issues.apache.org/jira/browse/HADOOP-7596
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
> Affects Versions: 0.20.204.0
> Environment: Java 6, RedHat EL 5.6
> Reporter: Eric Yang
> Assignee: Eric Yang
> Fix For: 0.20.205.0
>
> Attachments: HADOOP-7596-2.patch, HADOOP-7596.patch
>
>
> For secure Hadoop 0.20.2xx cluster, datanode can only run with 32 bit jvm because Hadoop only packages 32 bit jsvc. The build process should download proper jsvc versions base on the build architecture. In addition, the shell script should be enhanced to locate hadoop jar files in the proper location.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7596) Enable jsvc to work with Hadoop
RPM package
Posted by "Bruno Mahé (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13095528#comment-13095528 ]
Bruno Mahé commented on HADOOP-7596:
------------------------------------
{quote}
Sun Java is required
{quote}
So why do we have the following line:
https://github.com/apache/hadoop-common/blob/trunk/hadoop-common-project/hadoop-common/src/main/packages/deb/hadoop.control/control
{quote}
Depends: openjdk-6-jre-headless
{quote}
I am not that much familiar with packaging in hadoop itself, so I am sorry if I have missed some bits of infofmation.
> Enable jsvc to work with Hadoop RPM package
> -------------------------------------------
>
> Key: HADOOP-7596
> URL: https://issues.apache.org/jira/browse/HADOOP-7596
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
> Affects Versions: 0.20.204.0
> Environment: Java 6, RedHat EL 5.6
> Reporter: Eric Yang
> Assignee: Eric Yang
> Fix For: 0.20.205.0
>
> Attachments: HADOOP-7596-2.patch, HADOOP-7596.patch
>
>
> For secure Hadoop 0.20.2xx cluster, datanode can only run with 32 bit jvm because Hadoop only packages 32 bit jsvc. The build process should download proper jsvc versions base on the build architecture. In addition, the shell script should be enhanced to locate hadoop jar files in the proper location.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7596) Enable jsvc to work with Hadoop
RPM package
Posted by "Eric Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13095369#comment-13095369 ]
Eric Yang commented on HADOOP-7596:
-----------------------------------
bq. /usr/lib/jvm/java-6-sun is not valid on my machine (I have the JDK package installed). So why this path?
Using update-alternatives prompts user for selection of java, if there are multiple java installed. This can cause the installer to hang. Hadoop only works with Sun Java. Therefore, this change was made in trunk to set to /usr/lib/jvm/java-6-sun for debian system. Part of this patch brings 0.20.2xx to be in sync with trunk code.
bq. + DN_USER="-c hdfs"
Good catch, I made a mistake in my patch. Glad the mistake was caught. Thank you.
> Enable jsvc to work with Hadoop RPM package
> -------------------------------------------
>
> Key: HADOOP-7596
> URL: https://issues.apache.org/jira/browse/HADOOP-7596
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
> Affects Versions: 0.20.204.0
> Environment: Java 6, RedHat EL 5.6
> Reporter: Eric Yang
> Assignee: Eric Yang
> Fix For: 0.20.205.0
>
> Attachments: HADOOP-7596.patch
>
>
> For secure Hadoop 0.20.2xx cluster, datanode can only run with 32 bit jvm because Hadoop only packages 32 bit jsvc. The build process should download proper jsvc versions base on the build architecture. In addition, the shell script should be enhanced to locate hadoop jar files in the proper location.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7596) Enable jsvc to work with Hadoop
RPM package
Posted by "Eric Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13096557#comment-13096557 ]
Eric Yang commented on HADOOP-7596:
-----------------------------------
bq. Does this mean I'll have to adduser hdfs to run a non-secure cluster? If so I'd much rather not.
Ravi, hdfs is added as part of the rpm/deb package installation. This is the common behavior for rpm installed by packages. The headless user is preconfigured as part of package installation process.
bq. I would be expecting an identical behavior in both init scripts and use DN_USER/IDENT_USER for both cases
Bruno, Debian system uses .pid file to check for process aliveness. The filename generated by hadoop-daemon.sh can have discrepancy between the starting user and effective running user. This is why this fix only applies to Debian family. For redhat, it is using /var/lock/subsys to track process aliveness. Hence, in Redhat, there is no discrepancy for the pid file name verse the user that started the process because the lock is a fixed name.
> Enable jsvc to work with Hadoop RPM package
> -------------------------------------------
>
> Key: HADOOP-7596
> URL: https://issues.apache.org/jira/browse/HADOOP-7596
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
> Affects Versions: 0.20.204.0
> Environment: Java 6, RedHat EL 5.6
> Reporter: Eric Yang
> Assignee: Eric Yang
> Fix For: 0.20.205.0
>
> Attachments: HADOOP-7596-2.patch, HADOOP-7596-3.patch, HADOOP-7596.patch
>
>
> For secure Hadoop 0.20.2xx cluster, datanode can only run with 32 bit jvm because Hadoop only packages 32 bit jsvc. The build process should download proper jsvc versions base on the build architecture. In addition, the shell script should be enhanced to locate hadoop jar files in the proper location.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-7596) Enable jsvc to work with Hadoop RPM
package
Posted by "Eric Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eric Yang updated HADOOP-7596:
------------------------------
Attachment: HADOOP-7596.patch
- Bundle both 32bit and 64bit jsvc in release tar ball.
- Change scripts to construct class path properly.
- Bug fix for start up script to utilize proper jsvc.
> Enable jsvc to work with Hadoop RPM package
> -------------------------------------------
>
> Key: HADOOP-7596
> URL: https://issues.apache.org/jira/browse/HADOOP-7596
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
> Affects Versions: 0.20.204.0
> Environment: Java 6, RedHat EL 5.6
> Reporter: Eric Yang
> Assignee: Eric Yang
> Fix For: 0.20.205.0
>
> Attachments: HADOOP-7596.patch
>
>
> For secure Hadoop 0.20.2xx cluster, datanode can only run with 32 bit jvm because Hadoop only packages 32 bit jsvc. The build process should download proper jsvc versions base on the build architecture. In addition, the shell script should be enhanced to locate hadoop jar files in the proper location.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-7596) Enable jsvc to work with Hadoop RPM
package
Posted by "Eric Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eric Yang updated HADOOP-7596:
------------------------------
Status: Patch Available (was: Open)
> Enable jsvc to work with Hadoop RPM package
> -------------------------------------------
>
> Key: HADOOP-7596
> URL: https://issues.apache.org/jira/browse/HADOOP-7596
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
> Affects Versions: 0.20.204.0
> Environment: Java 6, RedHat EL 5.6
> Reporter: Eric Yang
> Assignee: Eric Yang
> Fix For: 0.20.205.0
>
> Attachments: HADOOP-7596.patch
>
>
> For secure Hadoop 0.20.2xx cluster, datanode can only run with 32 bit jvm because Hadoop only packages 32 bit jsvc. The build process should download proper jsvc versions base on the build architecture. In addition, the shell script should be enhanced to locate hadoop jar files in the proper location.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7596) Enable jsvc to work with Hadoop
RPM package
Posted by "Bruno Mahé (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13096563#comment-13096563 ]
Bruno Mahé commented on HADOOP-7596:
------------------------------------
Anyway, this is going off-topic. I just meant to replace in your patch:
{noformat}
- daemon --user hdfs ${HADOOP_PREFIX}/sbin/hadoop-daemon.sh --config "${HADOOP_CONF_DIR}" stop datanode
+ if [ -n "$HADOOP_SECURE_DN_USER" ]; then
+ daemon ${HADOOP_PREFIX}/sbin/hadoop-daemon.sh --config "${HADOOP_CONF_DIR}" stop datanode
+ else
+ daemon --user hdfs ${HADOOP_PREFIX}/sbin/hadoop-daemon.sh --config "${HADOOP_CONF_DIR}" stop datanode
+ fi
{noformat}
by something similar to what you did in the debian side. Which would look like something like:
{noformat}
+if [ -n "$HADOOP_SECURE_DN_USER" ]; then
+ DN_USER="root"
+else
+ DN_USER="hdfs"
+fi
{noformat}
And then later on:
{noformat}
- daemon --user hdfs ${HADOOP_PREFIX}/sbin/hadoop-daemon.sh --config "${HADOOP_CONF_DIR}" stop datanode
+ daemon --user $DN_USER ${HADOOP_PREFIX}/sbin/hadoop-daemon.sh --config "${HADOOP_CONF_DIR}" stop datanode
{noformat}
So you end up with a similar structure as the deb side and less repetition.
> Enable jsvc to work with Hadoop RPM package
> -------------------------------------------
>
> Key: HADOOP-7596
> URL: https://issues.apache.org/jira/browse/HADOOP-7596
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
> Affects Versions: 0.20.204.0
> Environment: Java 6, RedHat EL 5.6
> Reporter: Eric Yang
> Assignee: Eric Yang
> Fix For: 0.20.205.0
>
> Attachments: HADOOP-7596-2.patch, HADOOP-7596-3.patch, HADOOP-7596.patch
>
>
> For secure Hadoop 0.20.2xx cluster, datanode can only run with 32 bit jvm because Hadoop only packages 32 bit jsvc. The build process should download proper jsvc versions base on the build architecture. In addition, the shell script should be enhanced to locate hadoop jar files in the proper location.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7596) Enable jsvc to work with Hadoop
RPM package
Posted by "Ravi Prakash (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13096551#comment-13096551 ]
Ravi Prakash commented on HADOOP-7596:
--------------------------------------
bq. For non-secure cluster, datanode process is hard coded to hdfs user.
Does this mean I'll have to adduser hdfs to run a non-secure cluster? If so I'd much rather not.
> Enable jsvc to work with Hadoop RPM package
> -------------------------------------------
>
> Key: HADOOP-7596
> URL: https://issues.apache.org/jira/browse/HADOOP-7596
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
> Affects Versions: 0.20.204.0
> Environment: Java 6, RedHat EL 5.6
> Reporter: Eric Yang
> Assignee: Eric Yang
> Fix For: 0.20.205.0
>
> Attachments: HADOOP-7596-2.patch, HADOOP-7596-3.patch, HADOOP-7596.patch
>
>
> For secure Hadoop 0.20.2xx cluster, datanode can only run with 32 bit jvm because Hadoop only packages 32 bit jsvc. The build process should download proper jsvc versions base on the build architecture. In addition, the shell script should be enhanced to locate hadoop jar files in the proper location.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7596) Enable jsvc to work with Hadoop
RPM package
Posted by "Bruno Mahé (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13096589#comment-13096589 ]
Bruno Mahé commented on HADOOP-7596:
------------------------------------
Thanks a lot for your reply. I understand the motivation now.
But imho, this is negligible comparing to:
* Everything else in the hadoop stack.
* The advantages that consistency and readability would bring.
* the efforts to maintain such micro-optimization across all the GNU/Linux distributions(from which LSB implementation wildly differ and in some case is partly broken)
Thank you again very much for your patience.
> Enable jsvc to work with Hadoop RPM package
> -------------------------------------------
>
> Key: HADOOP-7596
> URL: https://issues.apache.org/jira/browse/HADOOP-7596
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
> Affects Versions: 0.20.204.0
> Environment: Java 6, RedHat EL 5.6
> Reporter: Eric Yang
> Assignee: Eric Yang
> Fix For: 0.20.205.0
>
> Attachments: HADOOP-7596-2.patch, HADOOP-7596-3.patch, HADOOP-7596.patch
>
>
> For secure Hadoop 0.20.2xx cluster, datanode can only run with 32 bit jvm because Hadoop only packages 32 bit jsvc. The build process should download proper jsvc versions base on the build architecture. In addition, the shell script should be enhanced to locate hadoop jar files in the proper location.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7596) Enable jsvc to work with Hadoop
RPM package
Posted by "Bruno Mahé (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13095085#comment-13095085 ]
Bruno Mahé commented on HADOOP-7596:
------------------------------------
I am confused.
{noformat}
- JAVA_HOME=`update-alternatives --config java | grep java | cut -f2 -d':' | cut -f2 -d' ' | sed -e 's/\/bin\/java//'`
+ JAVA_HOME=/usr/lib/jvm/java-6-sun
{noformat}
/usr/lib/jvm/java-6-sun is not valid on my machine (I have the JDK package installed). So why this path?
And also I see
{noformat}
+if [ -n "$HADOOP_SECURE_DN_USER" ]; then
+ DN_USER=""
+else
+ DN_USER="-c hdfs"
+fi
+
{noformat}
but then it is used later on in paths. Here is one example:
{noformat}
- if start-stop-daemon --start --quiet --oknodo --pidfile ${HADOOP_PID_DIR}/hadoop-hdfs-datanode.pid -c hdfs -x ${HADOOP_PREFIX}/sbin/hadoop-daemon.sh -- --config ${HADOOP_CONF_DIR} start datanode; then
+ if start-stop-daemon --start --quiet --oknodo --pidfile ${HADOOP_PID_DIR}/hadoop-${DN_USER}-datanode.pid -c ${DN_USER} -x ${HADOOP_PREFIX}/sbin/hadoop-daemon.sh -- --config ${HADOOP_CONF_DIR} start datanode; then
{noformat}
So
{noformat}
--pidfile ${HADOOP_PID_DIR}/hadoop-${DN_USER}-datanode.pid
{noformat}
would be expanded to
{noformat}
--pidfile ${HADOOP_PID_DIR}/hadoop-c hdfs-datanode.pid
{noformat}
> Enable jsvc to work with Hadoop RPM package
> -------------------------------------------
>
> Key: HADOOP-7596
> URL: https://issues.apache.org/jira/browse/HADOOP-7596
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
> Affects Versions: 0.20.204.0
> Environment: Java 6, RedHat EL 5.6
> Reporter: Eric Yang
> Assignee: Eric Yang
> Fix For: 0.20.205.0
>
> Attachments: HADOOP-7596.patch
>
>
> For secure Hadoop 0.20.2xx cluster, datanode can only run with 32 bit jvm because Hadoop only packages 32 bit jsvc. The build process should download proper jsvc versions base on the build architecture. In addition, the shell script should be enhanced to locate hadoop jar files in the proper location.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-7596) Enable jsvc to work with Hadoop RPM
package
Posted by "Eric Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eric Yang updated HADOOP-7596:
------------------------------
Status: Open (was: Patch Available)
Cancel patch because classpath should be setup differently between tar layout and binary layout
> Enable jsvc to work with Hadoop RPM package
> -------------------------------------------
>
> Key: HADOOP-7596
> URL: https://issues.apache.org/jira/browse/HADOOP-7596
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
> Affects Versions: 0.20.204.0
> Environment: Java 6, RedHat EL 5.6
> Reporter: Eric Yang
> Assignee: Eric Yang
> Fix For: 0.20.205.0
>
> Attachments: HADOOP-7596.patch
>
>
> For secure Hadoop 0.20.2xx cluster, datanode can only run with 32 bit jvm because Hadoop only packages 32 bit jsvc. The build process should download proper jsvc versions base on the build architecture. In addition, the shell script should be enhanced to locate hadoop jar files in the proper location.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7596) Enable jsvc to work with Hadoop
RPM package
Posted by "Bruno Mahé (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13096554#comment-13096554 ]
Bruno Mahé commented on HADOOP-7596:
------------------------------------
Thank you very much for your reply.
But I am still confused.
I see
{noformat}
+if [ -n "$HADOOP_SECURE_DN_USER" ]; then
+ DN_USER="root"
+ IDENT_USER=${HADOOP_SECURE_DN_USER}
+else
+ DN_USER="hdfs"
+ IDENT_USER=${DN_USER}
+fi
{noformat}
defined in src/packages/deb/init.d/hadoop-datanode but not in src/packages/rpm/init.d/hadoop-datanode ? (I am looking at https://github.com/apache/hadoop-common/blob/branch-0.20-security-205/src/packages/rpm/init.d/hadoop-datanode so hopefully I am on the right branch)
I would be expecting an identical behavior in both init scripts and use DN_USER/IDENT_USER for both cases
> Enable jsvc to work with Hadoop RPM package
> -------------------------------------------
>
> Key: HADOOP-7596
> URL: https://issues.apache.org/jira/browse/HADOOP-7596
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
> Affects Versions: 0.20.204.0
> Environment: Java 6, RedHat EL 5.6
> Reporter: Eric Yang
> Assignee: Eric Yang
> Fix For: 0.20.205.0
>
> Attachments: HADOOP-7596-2.patch, HADOOP-7596-3.patch, HADOOP-7596.patch
>
>
> For secure Hadoop 0.20.2xx cluster, datanode can only run with 32 bit jvm because Hadoop only packages 32 bit jsvc. The build process should download proper jsvc versions base on the build architecture. In addition, the shell script should be enhanced to locate hadoop jar files in the proper location.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7596) Enable jsvc to work with Hadoop
RPM package
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13095154#comment-13095154 ]
Hadoop QA commented on HADOOP-7596:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12492542/HADOOP-7596.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
-1 patch. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/116//console
This message is automatically generated.
> Enable jsvc to work with Hadoop RPM package
> -------------------------------------------
>
> Key: HADOOP-7596
> URL: https://issues.apache.org/jira/browse/HADOOP-7596
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
> Affects Versions: 0.20.204.0
> Environment: Java 6, RedHat EL 5.6
> Reporter: Eric Yang
> Assignee: Eric Yang
> Fix For: 0.20.205.0
>
> Attachments: HADOOP-7596.patch
>
>
> For secure Hadoop 0.20.2xx cluster, datanode can only run with 32 bit jvm because Hadoop only packages 32 bit jsvc. The build process should download proper jsvc versions base on the build architecture. In addition, the shell script should be enhanced to locate hadoop jar files in the proper location.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7596) Enable jsvc to work with Hadoop
RPM package
Posted by "Eric Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13094218#comment-13094218 ]
Eric Yang commented on HADOOP-7596:
-----------------------------------
Jakob, yes, but there is only one release tar ball which contains both i386 and x86_64 libraries. There is no unique location for jsvc which bundled in Hadoop. Hence, the release tarball is effectively 32 bit jsvc only.
Ideally, there should be one tarball/rpm/deb per architecture build. However, having separated tar ball per architecture build prevents us from having a single signed hadoop-core*.jar file to be push to maven repository.
Therefore, to solve this problem, we probably should retain the single tar ball release, but bundle both jsvc in the release tar ball. The only change would be the shell script to do the late binding to select the right jsvc to use at runtime.
RPM/DEB packages should only package jsvc base on the building architecture.
Does this sound reasonable?
> Enable jsvc to work with Hadoop RPM package
> -------------------------------------------
>
> Key: HADOOP-7596
> URL: https://issues.apache.org/jira/browse/HADOOP-7596
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
> Affects Versions: 0.20.204.0
> Environment: Java 6, RedHat EL 5.6
> Reporter: Eric Yang
> Assignee: Eric Yang
> Fix For: 0.20.205.0
>
>
> For secure Hadoop 0.20.2xx cluster, datanode can only run with 32 bit jvm because Hadoop only packages 32 bit jsvc. The build process should download proper jsvc versions base on the build architecture. In addition, the shell script should be enhanced to locate hadoop jar files in the proper location.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7596) Enable jsvc to work with Hadoop
RPM package
Posted by "Eric Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13096549#comment-13096549 ]
Eric Yang commented on HADOOP-7596:
-----------------------------------
bq. Out of curiosity, what is the rationale of having a ${IDENT_USER} and ${DN_USER} ?
For secure datanode, the script should be started with root user then drop privileges to HADOOP_SECURE_DN_USER. In the secure deployment context, IDENT_USER=hdfs, and DN_USER=root.
bq. hdfs user is still hardcoded in several locations
For non-secure cluster, datanode process is hard coded to hdfs user.
bq. Isn't it a dangerous behaviour if a user customize this directory? (and by mistake or knowingly set it for instance to /var/log)
chmod code was legacy code. The test log dir is to make it less destructive while maintain backward compatibility.
> Enable jsvc to work with Hadoop RPM package
> -------------------------------------------
>
> Key: HADOOP-7596
> URL: https://issues.apache.org/jira/browse/HADOOP-7596
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
> Affects Versions: 0.20.204.0
> Environment: Java 6, RedHat EL 5.6
> Reporter: Eric Yang
> Assignee: Eric Yang
> Fix For: 0.20.205.0
>
> Attachments: HADOOP-7596-2.patch, HADOOP-7596-3.patch, HADOOP-7596.patch
>
>
> For secure Hadoop 0.20.2xx cluster, datanode can only run with 32 bit jvm because Hadoop only packages 32 bit jsvc. The build process should download proper jsvc versions base on the build architecture. In addition, the shell script should be enhanced to locate hadoop jar files in the proper location.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7596) Enable jsvc to work with Hadoop
RPM package
Posted by "Eric Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13094267#comment-13094267 ]
Eric Yang commented on HADOOP-7596:
-----------------------------------
This jira is only valid for 0.20.205 and 2xx branch only. For 0.23+, HDFS-2289 should be applied.
> Enable jsvc to work with Hadoop RPM package
> -------------------------------------------
>
> Key: HADOOP-7596
> URL: https://issues.apache.org/jira/browse/HADOOP-7596
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
> Affects Versions: 0.20.204.0
> Environment: Java 6, RedHat EL 5.6
> Reporter: Eric Yang
> Assignee: Eric Yang
> Fix For: 0.20.205.0
>
>
> For secure Hadoop 0.20.2xx cluster, datanode can only run with 32 bit jvm because Hadoop only packages 32 bit jsvc. The build process should download proper jsvc versions base on the build architecture. In addition, the shell script should be enhanced to locate hadoop jar files in the proper location.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-7596) Enable jsvc to work with Hadoop RPM
package
Posted by "Eric Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eric Yang updated HADOOP-7596:
------------------------------
Status: Open (was: Patch Available)
Data node user is grabbled in the patch.
> Enable jsvc to work with Hadoop RPM package
> -------------------------------------------
>
> Key: HADOOP-7596
> URL: https://issues.apache.org/jira/browse/HADOOP-7596
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
> Affects Versions: 0.20.204.0
> Environment: Java 6, RedHat EL 5.6
> Reporter: Eric Yang
> Assignee: Eric Yang
> Fix For: 0.20.205.0
>
> Attachments: HADOOP-7596.patch
>
>
> For secure Hadoop 0.20.2xx cluster, datanode can only run with 32 bit jvm because Hadoop only packages 32 bit jsvc. The build process should download proper jsvc versions base on the build architecture. In addition, the shell script should be enhanced to locate hadoop jar files in the proper location.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-7596) Enable jsvc to work with Hadoop RPM
package
Posted by "Eric Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eric Yang updated HADOOP-7596:
------------------------------
Attachment: HADOOP-7596-3.patch
- Removed uid/gid from this patch.
- Removed legacy code for setting HADOOP_HOME to $HADOOP_PREFIX/share/hadoop/bin.
> Enable jsvc to work with Hadoop RPM package
> -------------------------------------------
>
> Key: HADOOP-7596
> URL: https://issues.apache.org/jira/browse/HADOOP-7596
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
> Affects Versions: 0.20.204.0
> Environment: Java 6, RedHat EL 5.6
> Reporter: Eric Yang
> Assignee: Eric Yang
> Fix For: 0.20.205.0
>
> Attachments: HADOOP-7596-2.patch, HADOOP-7596-3.patch, HADOOP-7596.patch
>
>
> For secure Hadoop 0.20.2xx cluster, datanode can only run with 32 bit jvm because Hadoop only packages 32 bit jsvc. The build process should download proper jsvc versions base on the build architecture. In addition, the shell script should be enhanced to locate hadoop jar files in the proper location.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7596) Enable jsvc to work with Hadoop
RPM package
Posted by "Eric Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13096535#comment-13096535 ]
Eric Yang commented on HADOOP-7596:
-----------------------------------
Also fixed init.d script for debian where the startup user was set incorrectly in HADOOP-7596-2.patch.
> Enable jsvc to work with Hadoop RPM package
> -------------------------------------------
>
> Key: HADOOP-7596
> URL: https://issues.apache.org/jira/browse/HADOOP-7596
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
> Affects Versions: 0.20.204.0
> Environment: Java 6, RedHat EL 5.6
> Reporter: Eric Yang
> Assignee: Eric Yang
> Fix For: 0.20.205.0
>
> Attachments: HADOOP-7596-2.patch, HADOOP-7596-3.patch, HADOOP-7596.patch
>
>
> For secure Hadoop 0.20.2xx cluster, datanode can only run with 32 bit jvm because Hadoop only packages 32 bit jsvc. The build process should download proper jsvc versions base on the build architecture. In addition, the shell script should be enhanced to locate hadoop jar files in the proper location.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7596) Enable jsvc to work with Hadoop
RPM package
Posted by "Roman Shaposhnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13094248#comment-13094248 ]
Roman Shaposhnik commented on HADOOP-7596:
------------------------------------------
I thought we've agreed over at HDFS-2289 to recompile jsvc as part of the build process. Is this JIRA somehow different?
> Enable jsvc to work with Hadoop RPM package
> -------------------------------------------
>
> Key: HADOOP-7596
> URL: https://issues.apache.org/jira/browse/HADOOP-7596
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
> Affects Versions: 0.20.204.0
> Environment: Java 6, RedHat EL 5.6
> Reporter: Eric Yang
> Assignee: Eric Yang
> Fix For: 0.20.205.0
>
>
> For secure Hadoop 0.20.2xx cluster, datanode can only run with 32 bit jvm because Hadoop only packages 32 bit jsvc. The build process should download proper jsvc versions base on the build architecture. In addition, the shell script should be enhanced to locate hadoop jar files in the proper location.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7596) Enable jsvc to work with Hadoop
RPM package
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13096536#comment-13096536 ]
Hadoop QA commented on HADOOP-7596:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12492861/HADOOP-7596-3.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
-1 patch. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/130//console
This message is automatically generated.
> Enable jsvc to work with Hadoop RPM package
> -------------------------------------------
>
> Key: HADOOP-7596
> URL: https://issues.apache.org/jira/browse/HADOOP-7596
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
> Affects Versions: 0.20.204.0
> Environment: Java 6, RedHat EL 5.6
> Reporter: Eric Yang
> Assignee: Eric Yang
> Fix For: 0.20.205.0
>
> Attachments: HADOOP-7596-2.patch, HADOOP-7596-3.patch, HADOOP-7596.patch
>
>
> For secure Hadoop 0.20.2xx cluster, datanode can only run with 32 bit jvm because Hadoop only packages 32 bit jsvc. The build process should download proper jsvc versions base on the build architecture. In addition, the shell script should be enhanced to locate hadoop jar files in the proper location.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7596) Enable jsvc to work with Hadoop
RPM package
Posted by "Jakob Homan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13094239#comment-13094239 ]
Jakob Homan commented on HADOOP-7596:
-------------------------------------
ok.
> Enable jsvc to work with Hadoop RPM package
> -------------------------------------------
>
> Key: HADOOP-7596
> URL: https://issues.apache.org/jira/browse/HADOOP-7596
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
> Affects Versions: 0.20.204.0
> Environment: Java 6, RedHat EL 5.6
> Reporter: Eric Yang
> Assignee: Eric Yang
> Fix For: 0.20.205.0
>
>
> For secure Hadoop 0.20.2xx cluster, datanode can only run with 32 bit jvm because Hadoop only packages 32 bit jsvc. The build process should download proper jsvc versions base on the build architecture. In addition, the shell script should be enhanced to locate hadoop jar files in the proper location.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-7596) Enable jsvc to work with Hadoop RPM
package
Posted by "Eric Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eric Yang updated HADOOP-7596:
------------------------------
Status: Patch Available (was: Open)
> Enable jsvc to work with Hadoop RPM package
> -------------------------------------------
>
> Key: HADOOP-7596
> URL: https://issues.apache.org/jira/browse/HADOOP-7596
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
> Affects Versions: 0.20.204.0
> Environment: Java 6, RedHat EL 5.6
> Reporter: Eric Yang
> Assignee: Eric Yang
> Fix For: 0.20.205.0
>
> Attachments: HADOOP-7596-2.patch, HADOOP-7596.patch
>
>
> For secure Hadoop 0.20.2xx cluster, datanode can only run with 32 bit jvm because Hadoop only packages 32 bit jsvc. The build process should download proper jsvc versions base on the build architecture. In addition, the shell script should be enhanced to locate hadoop jar files in the proper location.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7596) Enable jsvc to work with Hadoop
RPM package
Posted by "Bruno Mahé (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13096546#comment-13096546 ]
Bruno Mahé commented on HADOOP-7596:
------------------------------------
{quote}
+ if start-stop-daemon --start --quiet --oknodo --pidfile ${HADOOP_PID_DIR}/hadoop-${IDENT_USER}-datanode.pid -c ${DN_USER} -x ${HADOOP_PREFIX}/sbin/hadoop-daemon.sh -- --config ${HADOOP_CONF_DIR} start datanode; then
{quote}
Out of curiosity, what is the rationale of having a ${IDENT_USER} and ${DN_USER} ?
{quote}
+ daemon --user hdfs ${HADOOP_PREFIX}/sbin/hadoop-daemon.sh --config "${HADOOP_CONF_DIR}" start datanode
{quote}
hdfs user is still hardcoded in several locations
{quote}
${HADOOP_PREFIX}/sbin/hadoop-daemon.sh
{quote}
is used in many locations. It may be useful to use a variable instead.
{quote}
-chown $HADOOP_IDENT_STRING $HADOOP_LOG_DIR
+touch $HADOOP_LOG_DIR/.hadoop_test > /dev/null 2>&1
+TEST_LOG_DIR=$?
+if [ "${TEST_LOG_DIR}" = "0" ]; then
+ rm -f $HADOOP_LOG_DIR/.hadoop_test
+else
+ chown $HADOOP_IDENT_STRING $HADOOP_LOG_DIR
+fi
{quote}
Isn't it a dangerous behaviour if a user customize this directory? (and by mistake or knowingly set it for instance to /var/log)
And also in some places I see the LSB log* functions being used and in some other places I see "echo -n". Although this is out of scope of this ticket.
> Enable jsvc to work with Hadoop RPM package
> -------------------------------------------
>
> Key: HADOOP-7596
> URL: https://issues.apache.org/jira/browse/HADOOP-7596
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
> Affects Versions: 0.20.204.0
> Environment: Java 6, RedHat EL 5.6
> Reporter: Eric Yang
> Assignee: Eric Yang
> Fix For: 0.20.205.0
>
> Attachments: HADOOP-7596-2.patch, HADOOP-7596-3.patch, HADOOP-7596.patch
>
>
> For secure Hadoop 0.20.2xx cluster, datanode can only run with 32 bit jvm because Hadoop only packages 32 bit jsvc. The build process should download proper jsvc versions base on the build architecture. In addition, the shell script should be enhanced to locate hadoop jar files in the proper location.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-7596) Enable jsvc to work with Hadoop RPM
package
Posted by "Eric Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eric Yang updated HADOOP-7596:
------------------------------
Attachment: HADOOP-7596-2.patch
- Fixed secure datanode user.
- Improved layout detection for tar layout vs binary layout.
> Enable jsvc to work with Hadoop RPM package
> -------------------------------------------
>
> Key: HADOOP-7596
> URL: https://issues.apache.org/jira/browse/HADOOP-7596
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
> Affects Versions: 0.20.204.0
> Environment: Java 6, RedHat EL 5.6
> Reporter: Eric Yang
> Assignee: Eric Yang
> Fix For: 0.20.205.0
>
> Attachments: HADOOP-7596-2.patch, HADOOP-7596.patch
>
>
> For secure Hadoop 0.20.2xx cluster, datanode can only run with 32 bit jvm because Hadoop only packages 32 bit jsvc. The build process should download proper jsvc versions base on the build architecture. In addition, the shell script should be enhanced to locate hadoop jar files in the proper location.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7596) Enable jsvc to work with Hadoop
RPM package
Posted by "Allen Wittenauer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13095440#comment-13095440 ]
Allen Wittenauer commented on HADOOP-7596:
------------------------------------------
bq. Hadoop only works with Sun Java.
This isn't true and one of the reasons why attempting to figure out which java to use programmatically is full of pot holes.
> Enable jsvc to work with Hadoop RPM package
> -------------------------------------------
>
> Key: HADOOP-7596
> URL: https://issues.apache.org/jira/browse/HADOOP-7596
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
> Affects Versions: 0.20.204.0
> Environment: Java 6, RedHat EL 5.6
> Reporter: Eric Yang
> Assignee: Eric Yang
> Fix For: 0.20.205.0
>
> Attachments: HADOOP-7596.patch
>
>
> For secure Hadoop 0.20.2xx cluster, datanode can only run with 32 bit jvm because Hadoop only packages 32 bit jsvc. The build process should download proper jsvc versions base on the build architecture. In addition, the shell script should be enhanced to locate hadoop jar files in the proper location.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7596) Enable jsvc to work with Hadoop
RPM package
Posted by "Jakob Homan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13094179#comment-13094179 ]
Jakob Homan commented on HADOOP-7596:
-------------------------------------
And this isn't solved by setting -Djsvc.location={url to 64 bit package}?
> Enable jsvc to work with Hadoop RPM package
> -------------------------------------------
>
> Key: HADOOP-7596
> URL: https://issues.apache.org/jira/browse/HADOOP-7596
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
> Affects Versions: 0.20.204.0
> Environment: Java 6, RedHat EL 5.6
> Reporter: Eric Yang
> Assignee: Eric Yang
> Fix For: 0.20.205.0
>
>
> For secure Hadoop 0.20.2xx cluster, datanode can only run with 32 bit jvm because Hadoop only packages 32 bit jsvc. The build process should download proper jsvc versions base on the build architecture. In addition, the shell script should be enhanced to locate hadoop jar files in the proper location.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7596) Enable jsvc to work with Hadoop
RPM package
Posted by "Eric Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13096580#comment-13096580 ]
Eric Yang commented on HADOOP-7596:
-----------------------------------
bq. So you end up with a similar structure as the deb side and less repetition.
The code is optimized for each linux distro. There is a settle difference for "daemon --user root" and "daemon " in Redhat. In the second case, it does not run "runuser" for performance reason. In Debian case, it always runs start-stop-daemon regardless there is a user specified or not. Due to the performance difference, and generalization code are already in hadoop-daemon.sh. There is no need to put generalization code in vendor specific script.
> Enable jsvc to work with Hadoop RPM package
> -------------------------------------------
>
> Key: HADOOP-7596
> URL: https://issues.apache.org/jira/browse/HADOOP-7596
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
> Affects Versions: 0.20.204.0
> Environment: Java 6, RedHat EL 5.6
> Reporter: Eric Yang
> Assignee: Eric Yang
> Fix For: 0.20.205.0
>
> Attachments: HADOOP-7596-2.patch, HADOOP-7596-3.patch, HADOOP-7596.patch
>
>
> For secure Hadoop 0.20.2xx cluster, datanode can only run with 32 bit jvm because Hadoop only packages 32 bit jsvc. The build process should download proper jsvc versions base on the build architecture. In addition, the shell script should be enhanced to locate hadoop jar files in the proper location.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7596) Enable jsvc to work with Hadoop
RPM package
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13096544#comment-13096544 ]
Devaraj Das commented on HADOOP-7596:
-------------------------------------
+1
> Enable jsvc to work with Hadoop RPM package
> -------------------------------------------
>
> Key: HADOOP-7596
> URL: https://issues.apache.org/jira/browse/HADOOP-7596
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
> Affects Versions: 0.20.204.0
> Environment: Java 6, RedHat EL 5.6
> Reporter: Eric Yang
> Assignee: Eric Yang
> Fix For: 0.20.205.0
>
> Attachments: HADOOP-7596-2.patch, HADOOP-7596-3.patch, HADOOP-7596.patch
>
>
> For secure Hadoop 0.20.2xx cluster, datanode can only run with 32 bit jvm because Hadoop only packages 32 bit jsvc. The build process should download proper jsvc versions base on the build architecture. In addition, the shell script should be enhanced to locate hadoop jar files in the proper location.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7596) Enable jsvc to work with Hadoop
RPM package
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13095819#comment-13095819 ]
Hadoop QA commented on HADOOP-7596:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12492644/HADOOP-7596-2.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
-1 patch. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/124//console
This message is automatically generated.
> Enable jsvc to work with Hadoop RPM package
> -------------------------------------------
>
> Key: HADOOP-7596
> URL: https://issues.apache.org/jira/browse/HADOOP-7596
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
> Affects Versions: 0.20.204.0
> Environment: Java 6, RedHat EL 5.6
> Reporter: Eric Yang
> Assignee: Eric Yang
> Fix For: 0.20.205.0
>
> Attachments: HADOOP-7596-2.patch, HADOOP-7596.patch
>
>
> For secure Hadoop 0.20.2xx cluster, datanode can only run with 32 bit jvm because Hadoop only packages 32 bit jsvc. The build process should download proper jsvc versions base on the build architecture. In addition, the shell script should be enhanced to locate hadoop jar files in the proper location.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-7596) Enable jsvc to work with Hadoop RPM
package
Posted by "Eric Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eric Yang updated HADOOP-7596:
------------------------------
Status: Patch Available (was: Open)
The patch is correct. I made a mistake on local testing.
> Enable jsvc to work with Hadoop RPM package
> -------------------------------------------
>
> Key: HADOOP-7596
> URL: https://issues.apache.org/jira/browse/HADOOP-7596
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
> Affects Versions: 0.20.204.0
> Environment: Java 6, RedHat EL 5.6
> Reporter: Eric Yang
> Assignee: Eric Yang
> Fix For: 0.20.205.0
>
> Attachments: HADOOP-7596.patch
>
>
> For secure Hadoop 0.20.2xx cluster, datanode can only run with 32 bit jvm because Hadoop only packages 32 bit jsvc. The build process should download proper jsvc versions base on the build architecture. In addition, the shell script should be enhanced to locate hadoop jar files in the proper location.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7596) Enable jsvc to work with Hadoop
RPM package
Posted by "Eric Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13096542#comment-13096542 ]
Eric Yang commented on HADOOP-7596:
-----------------------------------
Issues fixed in this jira does not require forward porting to trunk because the changes are already in trunk. Thanks to Devaraj for reviewing this patch.
> Enable jsvc to work with Hadoop RPM package
> -------------------------------------------
>
> Key: HADOOP-7596
> URL: https://issues.apache.org/jira/browse/HADOOP-7596
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
> Affects Versions: 0.20.204.0
> Environment: Java 6, RedHat EL 5.6
> Reporter: Eric Yang
> Assignee: Eric Yang
> Fix For: 0.20.205.0
>
> Attachments: HADOOP-7596-2.patch, HADOOP-7596-3.patch, HADOOP-7596.patch
>
>
> For secure Hadoop 0.20.2xx cluster, datanode can only run with 32 bit jvm because Hadoop only packages 32 bit jsvc. The build process should download proper jsvc versions base on the build architecture. In addition, the shell script should be enhanced to locate hadoop jar files in the proper location.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-7596) Enable jsvc to work with Hadoop RPM
package
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj Das updated HADOOP-7596:
--------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
I just committed this (to the 20-security branch). Thanks, Eric!
> Enable jsvc to work with Hadoop RPM package
> -------------------------------------------
>
> Key: HADOOP-7596
> URL: https://issues.apache.org/jira/browse/HADOOP-7596
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
> Affects Versions: 0.20.204.0
> Environment: Java 6, RedHat EL 5.6
> Reporter: Eric Yang
> Assignee: Eric Yang
> Fix For: 0.20.205.0
>
> Attachments: HADOOP-7596-2.patch, HADOOP-7596-3.patch, HADOOP-7596.patch
>
>
> For secure Hadoop 0.20.2xx cluster, datanode can only run with 32 bit jvm because Hadoop only packages 32 bit jsvc. The build process should download proper jsvc versions base on the build architecture. In addition, the shell script should be enhanced to locate hadoop jar files in the proper location.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7596) Enable jsvc to work with Hadoop
RPM package
Posted by "Allen Wittenauer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13095500#comment-13095500 ]
Allen Wittenauer commented on HADOOP-7596:
------------------------------------------
http://wiki.apache.org/hadoop/HadoopJavaVersions
> Enable jsvc to work with Hadoop RPM package
> -------------------------------------------
>
> Key: HADOOP-7596
> URL: https://issues.apache.org/jira/browse/HADOOP-7596
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
> Affects Versions: 0.20.204.0
> Environment: Java 6, RedHat EL 5.6
> Reporter: Eric Yang
> Assignee: Eric Yang
> Fix For: 0.20.205.0
>
> Attachments: HADOOP-7596.patch
>
>
> For secure Hadoop 0.20.2xx cluster, datanode can only run with 32 bit jvm because Hadoop only packages 32 bit jsvc. The build process should download proper jsvc versions base on the build architecture. In addition, the shell script should be enhanced to locate hadoop jar files in the proper location.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira