You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Eli Collins (JIRA)" <ji...@apache.org> on 2011/01/06 02:57:45 UTC

[jira] Created: (HADOOP-7089) Use readlink to get absolute paths in the scripts

Use readlink to get absolute paths in the scripts 
--------------------------------------------------

                 Key: HADOOP-7089
                 URL: https://issues.apache.org/jira/browse/HADOOP-7089
             Project: Hadoop Common
          Issue Type: Improvement
          Components: scripts
            Reporter: Eli Collins
            Assignee: Eli Collins
            Priority: Minor
             Fix For: 0.22.0, 0.23.0


The manual link resolution logic in bin/hadoop-config.sh can be replaced with readlink -m -n.  Ditto with other uses of cd + pwd to get an absolute path, which can be fragile.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HADOOP-7089) Fix link resolution logic in hadoop-config.sh

Posted by "Hudson (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HADOOP-7089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12983570#action_12983570 ] 

Hudson commented on HADOOP-7089:
--------------------------------

Integrated in Hadoop-Common-trunk-Commit #479 (See [https://hudson.apache.org/hudson/job/Hadoop-Common-trunk-Commit/479/])
    

> Fix link resolution logic in hadoop-config.sh
> ---------------------------------------------
>
>                 Key: HADOOP-7089
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7089
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: scripts
>            Reporter: Eli Collins
>            Assignee: Eli Collins
>            Priority: Minor
>             Fix For: 0.22.0, 0.23.0
>
>         Attachments: hadoop-7089-1.patch, hadoop-7089-1.patch, hadoop-7089-2.patch, hadoop-7089-3.patch
>
>
> The link resolution logic in bin/hadoop-config.sh fails when when executed via a symlink, from the root directory.  We can replace this logic with cd -P and pwd -P, which should be portable across Linux, Solaris, BSD, and OSX. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (HADOOP-7089) Fix link resolution logic in hadoop-config.sh

Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HADOOP-7089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Eli Collins updated HADOOP-7089:
--------------------------------

    Attachment: hadoop-7089-3.patch

Per Allen's suggestion, updated the patch to add a comment indicating -P requires bash built-ins or POSIX:2001 cd and pwd.

> Fix link resolution logic in hadoop-config.sh
> ---------------------------------------------
>
>                 Key: HADOOP-7089
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7089
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: scripts
>            Reporter: Eli Collins
>            Assignee: Eli Collins
>            Priority: Minor
>         Attachments: hadoop-7089-1.patch, hadoop-7089-1.patch, hadoop-7089-2.patch, hadoop-7089-3.patch
>
>
> The link resolution logic in bin/hadoop-config.sh fails when when executed via a symlink, from the root directory.  We can replace this logic with cd -P and pwd -P, which should be portable across Linux, Solaris, BSD, and OSX. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HADOOP-7089) Use readlink to get absolute paths in the scripts

Posted by "Greg Roelofs (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HADOOP-7089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12978116#action_12978116 ] 

Greg Roelofs commented on HADOOP-7089:
--------------------------------------

Not even all Linux versions use GNU readlink.  The (BSD-derived?) version accepts no options.

> Use readlink to get absolute paths in the scripts 
> --------------------------------------------------
>
>                 Key: HADOOP-7089
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7089
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: scripts
>            Reporter: Eli Collins
>            Assignee: Eli Collins
>            Priority: Minor
>             Fix For: 0.22.0, 0.23.0
>
>         Attachments: hadoop-7089-1.patch, hadoop-7089-1.patch
>
>
> The manual link resolution logic in bin/hadoop-config.sh can be replaced with readlink -m -n.  Ditto with other uses of cd + pwd to get an absolute path, which can be fragile.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HADOOP-7089) Use readlink to get absolute paths in the scripts

Posted by "Greg Roelofs (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HADOOP-7089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12978117#action_12978117 ] 

Greg Roelofs commented on HADOOP-7089:
--------------------------------------

...and looks like older versions of GNU readlink don't accept the -m option.  This is the --help output from version 5.2.1:

  -f, --canonicalize      canonicalize by following every symlink in every
                          component of the given path recursively
  -n, --no-newline        do not output the trailing newline
  -q, --quiet,
  -s, --silent            suppress most error messages
  -v, --verbose           report error messages
      --help     display this help and exit
      --version  output version information and exit


> Use readlink to get absolute paths in the scripts 
> --------------------------------------------------
>
>                 Key: HADOOP-7089
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7089
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: scripts
>            Reporter: Eli Collins
>            Assignee: Eli Collins
>            Priority: Minor
>             Fix For: 0.22.0, 0.23.0
>
>         Attachments: hadoop-7089-1.patch, hadoop-7089-1.patch
>
>
> The manual link resolution logic in bin/hadoop-config.sh can be replaced with readlink -m -n.  Ditto with other uses of cd + pwd to get an absolute path, which can be fragile.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HADOOP-7089) Use readlink to get absolute paths in the scripts

Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HADOOP-7089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12978093#action_12978093 ] 

Eli Collins commented on HADOOP-7089:
-------------------------------------

bq. Does this work on OSX, BSD, and Solaris? I seem to remember this being proposed before, but killed for compatibility reasons.

I checked that -m and -n were supported on Solaris.  Looks like from ZOOKEEPER-303 though that OSX doesn't have this option, think the same workaround is acceptable here?

> Use readlink to get absolute paths in the scripts 
> --------------------------------------------------
>
>                 Key: HADOOP-7089
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7089
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: scripts
>            Reporter: Eli Collins
>            Assignee: Eli Collins
>            Priority: Minor
>             Fix For: 0.22.0, 0.23.0
>
>         Attachments: hadoop-7089-1.patch, hadoop-7089-1.patch
>
>
> The manual link resolution logic in bin/hadoop-config.sh can be replaced with readlink -m -n.  Ditto with other uses of cd + pwd to get an absolute path, which can be fragile.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (HADOOP-7089) Use readlink to get absolute paths in the scripts

Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HADOOP-7089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Eli Collins updated HADOOP-7089:
--------------------------------

    Attachment: hadoop-7089-1.patch

Right patch this time.

> Use readlink to get absolute paths in the scripts 
> --------------------------------------------------
>
>                 Key: HADOOP-7089
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7089
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: scripts
>            Reporter: Eli Collins
>            Assignee: Eli Collins
>            Priority: Minor
>             Fix For: 0.22.0, 0.23.0
>
>         Attachments: hadoop-7089-1.patch, hadoop-7089-1.patch
>
>
> The manual link resolution logic in bin/hadoop-config.sh can be replaced with readlink -m -n.  Ditto with other uses of cd + pwd to get an absolute path, which can be fragile.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HADOOP-7089) Fix link resolution logic in hadoop-config.sh

Posted by "Hudson (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HADOOP-7089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12981719#action_12981719 ] 

Hudson commented on HADOOP-7089:
--------------------------------

Integrated in Hadoop-Common-trunk #576 (See [https://hudson.apache.org/hudson/job/Hadoop-Common-trunk/576/])
    HADOOP-7089. Fix link resolution logic in hadoop-config.sh. Contributed by Eli Collins


> Fix link resolution logic in hadoop-config.sh
> ---------------------------------------------
>
>                 Key: HADOOP-7089
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7089
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: scripts
>            Reporter: Eli Collins
>            Assignee: Eli Collins
>            Priority: Minor
>             Fix For: 0.22.0, 0.23.0
>
>         Attachments: hadoop-7089-1.patch, hadoop-7089-1.patch, hadoop-7089-2.patch, hadoop-7089-3.patch
>
>
> The link resolution logic in bin/hadoop-config.sh fails when when executed via a symlink, from the root directory.  We can replace this logic with cd -P and pwd -P, which should be portable across Linux, Solaris, BSD, and OSX. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (HADOOP-7089) Fix link resolution logic in hadoop-config.sh

Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HADOOP-7089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Eli Collins updated HADOOP-7089:
--------------------------------

    Release Note: Updates hadoop-config.sh to always resolve symlinks when determining HADOOP_HOME. Bash built-ins or POSIX:2001 compliant cmds are now required.

> Fix link resolution logic in hadoop-config.sh
> ---------------------------------------------
>
>                 Key: HADOOP-7089
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7089
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: scripts
>            Reporter: Eli Collins
>            Assignee: Eli Collins
>            Priority: Minor
>             Fix For: 0.22.0, 0.23.0
>
>         Attachments: hadoop-7089-1.patch, hadoop-7089-1.patch, hadoop-7089-2.patch, hadoop-7089-3.patch
>
>
> The link resolution logic in bin/hadoop-config.sh fails when when executed via a symlink, from the root directory.  We can replace this logic with cd -P and pwd -P, which should be portable across Linux, Solaris, BSD, and OSX. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (HADOOP-7089) Use readlink to get absolute paths in the scripts

Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HADOOP-7089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Eli Collins updated HADOOP-7089:
--------------------------------

    Description: The link resolution logic in bin/hadoop-config.sh fails when when executed via a symlink, from the root directory.  We can replace this logic with cd -P and pwd -P, which should be portable across Linux, Solaris, BSD, and OSX.   (was: The manual link resolution logic in bin/hadoop-config.sh can be replaced with readlink -m -n.  Ditto with other uses of cd + pwd to get an absolute path, which can be fragile.)

> Use readlink to get absolute paths in the scripts 
> --------------------------------------------------
>
>                 Key: HADOOP-7089
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7089
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: scripts
>            Reporter: Eli Collins
>            Assignee: Eli Collins
>            Priority: Minor
>             Fix For: 0.22.0, 0.23.0
>
>         Attachments: hadoop-7089-1.patch, hadoop-7089-1.patch, hadoop-7089-2.patch
>
>
> The link resolution logic in bin/hadoop-config.sh fails when when executed via a symlink, from the root directory.  We can replace this logic with cd -P and pwd -P, which should be portable across Linux, Solaris, BSD, and OSX. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (HADOOP-7089) Use readlink to get absolute paths in the scripts

Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HADOOP-7089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Eli Collins updated HADOOP-7089:
--------------------------------

    Issue Type: Bug  (was: Improvement)

Yea, looks like readlink isn't going to work portably.  I think Allen's suggestion of using pwd -P is most portable, should work on Linux, BSD, OSX and Solaris, and is much cleaner than the current code. I'll update the patch.

The motivation is that there's a bug in the manual symlink resolution code, it fails when the script is called using a symlink from /, which is how Debian runs init scripts.  

{noformat}
/ $ ls -al /home/eli/test2
total 56
drwxr-xr-x   2 eli eli  4096 2011-01-07 10:07 .
drwxr-xr-x 136 eli eli 49152 2011-01-07 10:07 ..
lrwxrwxrwx   1 eli eli    17 2011-01-07 09:58 testl1.sh -> ../test1/test1.sh
lrwxrwxrwx   1 eli eli    17 2011-01-07 10:00 testl2.sh -> ../test1/test2.sh
/ $ cat /home/eli/test1/test1.sh 
#!/bin/bash
set -e
this="${BASH_SOURCE-$0}"
while [ -h "$this" ]; do
  ls=`ls -ld "$this"`
  link=`expr "$ls" : '.*-> \(.*\)$'`
  if expr "$link" : '.*/.*' > /dev/null; then
    this="$link"
  else
    this=`dirname "$this"`/"$link"
  fi
done
bin=`dirname "$this"`
script=`basename "$this"`
bin=`cd "$bin"; pwd`
this="$bin/$script"
echo $this
{noformat}

Called w/o a symlink:
{noformat}
/ $ /home/eli/test1/test1.sh
/home/eli/test1/test1.sh
{noformat}

Called via a symlink:
{noformat}
/ $ /home/eli/test2/testl1.sh 
/home/eli/test2/testl1.sh: line 15: cd: ../test1: No such file or directory
//test1.sh
{noformat}

Re-written to use pwd -P
{noformat}
/ $ cat /home/eli/test2/testl2.sh 
#!/bin/bash
bin=$(cd -P -- "$(dirname -- "${BASH_SOURCE-$0}")" && pwd -P)
script="$(basename -- "${BASH_SOURCE-$0}")"
this="$bin/$script"
echo $this
{noformat}

Called via a symlink:
{noformat}
/ $ /home/eli/test2/testl2.sh 
/home/eli/test2/testl2.sh
{noformat}

> Use readlink to get absolute paths in the scripts 
> --------------------------------------------------
>
>                 Key: HADOOP-7089
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7089
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: scripts
>            Reporter: Eli Collins
>            Assignee: Eli Collins
>            Priority: Minor
>             Fix For: 0.22.0, 0.23.0
>
>         Attachments: hadoop-7089-1.patch, hadoop-7089-1.patch
>
>
> The manual link resolution logic in bin/hadoop-config.sh can be replaced with readlink -m -n.  Ditto with other uses of cd + pwd to get an absolute path, which can be fragile.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HADOOP-7089) Fix link resolution logic in hadoop-config.sh

Posted by "Allen Wittenauer (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HADOOP-7089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12981599#action_12981599 ] 

Allen Wittenauer commented on HADOOP-7089:
------------------------------------------

Eli asked me to double check it.  This is a change in behavior, but I think it will be safe as well as portable so long as the bash built-ins are used.  It might be worthwhile adding that in the comments that we're depending upon the built-ins in case someone does something like port Hadoop to busybox.  [Think appliance.]

So, +1 if some changes are made to the comments.

> Fix link resolution logic in hadoop-config.sh
> ---------------------------------------------
>
>                 Key: HADOOP-7089
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7089
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: scripts
>            Reporter: Eli Collins
>            Assignee: Eli Collins
>            Priority: Minor
>         Attachments: hadoop-7089-1.patch, hadoop-7089-1.patch, hadoop-7089-2.patch
>
>
> The link resolution logic in bin/hadoop-config.sh fails when when executed via a symlink, from the root directory.  We can replace this logic with cd -P and pwd -P, which should be portable across Linux, Solaris, BSD, and OSX. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (HADOOP-7089) Use readlink to get absolute paths in the scripts

Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HADOOP-7089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Eli Collins updated HADOOP-7089:
--------------------------------

    Attachment: hadoop-7089-2.patch

Patch attached.  Tested running from / via a symlink works and that setting HADOOP_CORE_HOME from HDFS/MR root can start the daemons.

> Use readlink to get absolute paths in the scripts 
> --------------------------------------------------
>
>                 Key: HADOOP-7089
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7089
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: scripts
>            Reporter: Eli Collins
>            Assignee: Eli Collins
>            Priority: Minor
>             Fix For: 0.22.0, 0.23.0
>
>         Attachments: hadoop-7089-1.patch, hadoop-7089-1.patch, hadoop-7089-2.patch
>
>
> The manual link resolution logic in bin/hadoop-config.sh can be replaced with readlink -m -n.  Ditto with other uses of cd + pwd to get an absolute path, which can be fragile.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (HADOOP-7089) Use readlink to get absolute paths in the scripts

Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HADOOP-7089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Eli Collins updated HADOOP-7089:
--------------------------------

    Attachment: hadoop-7089-1.patch

Patch attached.  Tested running a daemon out of the bin dir w/ absolute and relative paths.

> Use readlink to get absolute paths in the scripts 
> --------------------------------------------------
>
>                 Key: HADOOP-7089
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7089
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: scripts
>            Reporter: Eli Collins
>            Assignee: Eli Collins
>            Priority: Minor
>             Fix For: 0.22.0, 0.23.0
>
>         Attachments: hadoop-7089-1.patch
>
>
> The manual link resolution logic in bin/hadoop-config.sh can be replaced with readlink -m -n.  Ditto with other uses of cd + pwd to get an absolute path, which can be fragile.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Closed: (HADOOP-7089) Fix link resolution logic in hadoop-config.sh

Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HADOOP-7089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Eli Collins closed HADOOP-7089.
-------------------------------


> Fix link resolution logic in hadoop-config.sh
> ---------------------------------------------
>
>                 Key: HADOOP-7089
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7089
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: scripts
>            Reporter: Eli Collins
>            Assignee: Eli Collins
>            Priority: Minor
>             Fix For: 0.22.0, 0.23.0
>
>         Attachments: hadoop-7089-1.patch, hadoop-7089-1.patch, hadoop-7089-2.patch, hadoop-7089-3.patch
>
>
> The link resolution logic in bin/hadoop-config.sh fails when when executed via a symlink, from the root directory.  We can replace this logic with cd -P and pwd -P, which should be portable across Linux, Solaris, BSD, and OSX. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HADOOP-7089) Use readlink to get absolute paths in the scripts

Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HADOOP-7089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12978087#action_12978087 ] 

Todd Lipcon commented on HADOOP-7089:
-------------------------------------

Does this work on OSX, BSD, and Solaris? I seem to remember this being proposed before, but killed for compatibility reasons.

> Use readlink to get absolute paths in the scripts 
> --------------------------------------------------
>
>                 Key: HADOOP-7089
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7089
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: scripts
>            Reporter: Eli Collins
>            Assignee: Eli Collins
>            Priority: Minor
>             Fix For: 0.22.0, 0.23.0
>
>         Attachments: hadoop-7089-1.patch
>
>
> The manual link resolution logic in bin/hadoop-config.sh can be replaced with readlink -m -n.  Ditto with other uses of cd + pwd to get an absolute path, which can be fragile.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (HADOOP-7089) Fix link resolution logic in hadoop-config.sh

Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HADOOP-7089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Eli Collins updated HADOOP-7089:
--------------------------------

    Summary: Fix link resolution logic in hadoop-config.sh  (was: Use readlink to get absolute paths in the scripts )

> Fix link resolution logic in hadoop-config.sh
> ---------------------------------------------
>
>                 Key: HADOOP-7089
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7089
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: scripts
>            Reporter: Eli Collins
>            Assignee: Eli Collins
>            Priority: Minor
>             Fix For: 0.22.0, 0.23.0
>
>         Attachments: hadoop-7089-1.patch, hadoop-7089-1.patch, hadoop-7089-2.patch
>
>
> The link resolution logic in bin/hadoop-config.sh fails when when executed via a symlink, from the root directory.  We can replace this logic with cd -P and pwd -P, which should be portable across Linux, Solaris, BSD, and OSX. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HADOOP-7089) Use readlink to get absolute paths in the scripts

Posted by "Allen Wittenauer (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HADOOP-7089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12978165#action_12978165 ] 

Allen Wittenauer commented on HADOOP-7089:
------------------------------------------

readlink is definitely tricky from a portability perspective; it is fairly recent as far as the commercial OSes go.  [I think it was added in AIX 7, but maybe Stephen or someone else can confirm since I no longer have access to an AIX machine.  On Solaris, GNU readlink was added in either 10 or NV, IIRC.]

Also, with  the change to use BASH_SOURCE, is cd+pwd still super fragile?   It may not get it 100% correct, but does it get it 'correct enough'? 

Another option might be to use file -h, but I'm not certain how portable that is either.  Since we require bash, there is also the (built-in only!) pwd -P.

> Use readlink to get absolute paths in the scripts 
> --------------------------------------------------
>
>                 Key: HADOOP-7089
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7089
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: scripts
>            Reporter: Eli Collins
>            Assignee: Eli Collins
>            Priority: Minor
>             Fix For: 0.22.0, 0.23.0
>
>         Attachments: hadoop-7089-1.patch, hadoop-7089-1.patch
>
>
> The manual link resolution logic in bin/hadoop-config.sh can be replaced with readlink -m -n.  Ditto with other uses of cd + pwd to get an absolute path, which can be fragile.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Resolved: (HADOOP-7089) Fix link resolution logic in hadoop-config.sh

Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HADOOP-7089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Eli Collins resolved HADOOP-7089.
---------------------------------

       Resolution: Fixed
    Fix Version/s: 0.23.0
                   0.22.0
     Hadoop Flags: [Reviewed]

Committed to trunk and branch 22. Thanks Allen.

> Fix link resolution logic in hadoop-config.sh
> ---------------------------------------------
>
>                 Key: HADOOP-7089
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7089
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: scripts
>            Reporter: Eli Collins
>            Assignee: Eli Collins
>            Priority: Minor
>             Fix For: 0.22.0, 0.23.0
>
>         Attachments: hadoop-7089-1.patch, hadoop-7089-1.patch, hadoop-7089-2.patch, hadoop-7089-3.patch
>
>
> The link resolution logic in bin/hadoop-config.sh fails when when executed via a symlink, from the root directory.  We can replace this logic with cd -P and pwd -P, which should be portable across Linux, Solaris, BSD, and OSX. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HADOOP-7089) Fix link resolution logic in hadoop-config.sh

Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HADOOP-7089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12981098#action_12981098 ] 

Eli Collins commented on HADOOP-7089:
-------------------------------------

Some extra explanation of the rationale:  the current code only resolves the final symlink in the path, ie only starts _if the file that sources hadoop-config.sh is a symlink_, where the patch here unconditionally resolves all the symlinks. So the current code sometimes does, and sometimes does not, fully resolve links in the path to HADOOP_HOME.  Therefore the current code should be agnostic as to whether you fully resolve HADOOP_HOME (ie it should be OK to always resolve the symlinks).

I think it makes sense to resolve links (and this probably was the intent of the original symlink resolution code) as that way the logic is uniform, ie the way HADOOP_HOME is determined is the same regardless of whether hadoop-config.sh is sourced from a symlink.

> Fix link resolution logic in hadoop-config.sh
> ---------------------------------------------
>
>                 Key: HADOOP-7089
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7089
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: scripts
>            Reporter: Eli Collins
>            Assignee: Eli Collins
>            Priority: Minor
>         Attachments: hadoop-7089-1.patch, hadoop-7089-1.patch, hadoop-7089-2.patch
>
>
> The link resolution logic in bin/hadoop-config.sh fails when when executed via a symlink, from the root directory.  We can replace this logic with cd -P and pwd -P, which should be portable across Linux, Solaris, BSD, and OSX. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HADOOP-7089) Fix link resolution logic in hadoop-config.sh

Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HADOOP-7089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12981610#action_12981610 ] 

Eli Collins commented on HADOOP-7089:
-------------------------------------

Thanks for the review Allen.

According to the following Posix includes -P for cd and pwd so I think this change will work even on systems w/o bash built ins.  On my linux host both the built-in and pwd binary support -P.

IEEE Std 1003.1, 2004 Edition
http://www.opengroup.org/onlinepubs/000095399/utilities/cd.html
http://www.opengroup.org/onlinepubs/000095399/utilities/pwd.html



> Fix link resolution logic in hadoop-config.sh
> ---------------------------------------------
>
>                 Key: HADOOP-7089
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7089
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: scripts
>            Reporter: Eli Collins
>            Assignee: Eli Collins
>            Priority: Minor
>         Attachments: hadoop-7089-1.patch, hadoop-7089-1.patch, hadoop-7089-2.patch
>
>
> The link resolution logic in bin/hadoop-config.sh fails when when executed via a symlink, from the root directory.  We can replace this logic with cd -P and pwd -P, which should be portable across Linux, Solaris, BSD, and OSX. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HADOOP-7089) Fix link resolution logic in hadoop-config.sh

Posted by "Hudson (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HADOOP-7089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12981692#action_12981692 ] 

Hudson commented on HADOOP-7089:
--------------------------------

Integrated in Hadoop-Common-22-branch #15 (See [https://hudson.apache.org/hudson/job/Hadoop-Common-22-branch/15/])
    HADOOP-7089. svn merge -c 1058881 from trunk


> Fix link resolution logic in hadoop-config.sh
> ---------------------------------------------
>
>                 Key: HADOOP-7089
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7089
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: scripts
>            Reporter: Eli Collins
>            Assignee: Eli Collins
>            Priority: Minor
>             Fix For: 0.22.0, 0.23.0
>
>         Attachments: hadoop-7089-1.patch, hadoop-7089-1.patch, hadoop-7089-2.patch, hadoop-7089-3.patch
>
>
> The link resolution logic in bin/hadoop-config.sh fails when when executed via a symlink, from the root directory.  We can replace this logic with cd -P and pwd -P, which should be portable across Linux, Solaris, BSD, and OSX. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.