You are viewing a plain text version of this content. The canonical link for it is here.

Posted to common-issues@hadoop.apache.org by "Eli Collins (Created) (JIRA)" <ji...@apache.org> on 2012/01/30 23:13:10 UTC

[jira] [Created] (HADOOP-8007) HA: use substitution token for fencing argument

HA: use substitution token for fencing argument
-----------------------------------------------

                 Key: HADOOP-8007
                 URL: https://issues.apache.org/jira/browse/HADOOP-8007
             Project: Hadoop Common
          Issue Type: Sub-task
          Components: ha
    Affects Versions: HA Branch (HDFS-1623)
            Reporter: Eli Collins
            Assignee: Eli Collins


Per HADOOP-7983 currently the fencer always passes the target host:port to fence as the first argument to the fence script, it would be better to use a substitution token. That is to say, the user would configure "myfence.sh $TARGETHOST foo bar" and Hadoop would substitute the target. This would allow use of pre-existing scripts that might have a different ordering of arguments without a wrapper.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HADOOP-8007) HA: use substitution token for fencing argument

Posted by "Todd Lipcon (Updated) (JIRA)" <ji...@apache.org>.

     [ https://issues.apache.org/jira/browse/HADOOP-8007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Todd Lipcon updated HADOOP-8007:
--------------------------------

    Attachment: hadoop-8007.txt

Attached patch changes the configuration of the fencing script as follows:

By default, no arguments are passed to the script. This is different than the previous implementation, but since HA hasn't been in a release yet, I think it's fine to break this.

Instead, users are expected to use the environment to choose whatever arguments they need. Since we pass the shell command through to bash, environment variables are interpolated.

We already handled passing the configuration through. This patch also adds the ability for an HAServiceTarget to add additional key/value pairs, which are prefixed with {{target_}} in the environment. The three passed by the Common code are {{$target_host}}, {{$target_port}}, and {{$target_address}} (which is the same as host:port).

HDFS-3084 will add the NNID and NSID to this map.

I will also file a separate JIRA to update the docs, which unfortunately live in the MapReduce project.
                
> HA: use substitution token for fencing argument
> -----------------------------------------------
>
>                 Key: HADOOP-8007
>                 URL: https://issues.apache.org/jira/browse/HADOOP-8007
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: ha
>    Affects Versions: 2.0.0
>            Reporter: Aaron T. Myers
>            Assignee: Todd Lipcon
>         Attachments: hadoop-8007.txt
>
>
> Per HADOOP-7983 currently the fencer always passes the target host:port to fence as the first argument to the fence script, it would be better to use a substitution token. That is to say, the user would configure "myfence.sh $TARGETHOST foo bar" and Hadoop would substitute the target. This would allow use of pre-existing scripts that might have a different ordering of arguments without a wrapper.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HADOOP-8007) HA: use substitution token for fencing argument

Posted by "Aaron T. Myers (Updated) (JIRA)" <ji...@apache.org>.

     [ https://issues.apache.org/jira/browse/HADOOP-8007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Aaron T. Myers updated HADOOP-8007:
-----------------------------------

    Issue Type: Improvement  (was: Sub-task)
      Reporter: Aaron T. Myers  (was: Eli Collins)
        Parent:     (was: HADOOP-7454)
    
> HA: use substitution token for fencing argument
> -----------------------------------------------
>
>                 Key: HADOOP-8007
>                 URL: https://issues.apache.org/jira/browse/HADOOP-8007
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: ha
>    Affects Versions: 0.24.0
>            Reporter: Aaron T. Myers
>            Assignee: Eli Collins
>
> Per HADOOP-7983 currently the fencer always passes the target host:port to fence as the first argument to the fence script, it would be better to use a substitution token. That is to say, the user would configure "myfence.sh $TARGETHOST foo bar" and Hadoop would substitute the target. This would allow use of pre-existing scripts that might have a different ordering of arguments without a wrapper.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HADOOP-8007) HA: use substitution token for fencing argument

Posted by "Aaron T. Myers (Commented) (JIRA)" <ji...@apache.org>.

    [ https://issues.apache.org/jira/browse/HADOOP-8007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13246074#comment-13246074 ] 

Aaron T. Myers commented on HADOOP-8007:
----------------------------------------

+1, the patch looks good to me. Thanks a lot for doing this, Todd.

Please do file the separate JIRA for the docs, and I'll be happy to review those as well.
                
> HA: use substitution token for fencing argument
> -----------------------------------------------
>
>                 Key: HADOOP-8007
>                 URL: https://issues.apache.org/jira/browse/HADOOP-8007
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: ha
>    Affects Versions: 2.0.0
>            Reporter: Aaron T. Myers
>            Assignee: Todd Lipcon
>         Attachments: hadoop-8007.txt
>
>
> Per HADOOP-7983 currently the fencer always passes the target host:port to fence as the first argument to the fence script, it would be better to use a substitution token. That is to say, the user would configure "myfence.sh $TARGETHOST foo bar" and Hadoop would substitute the target. This would allow use of pre-existing scripts that might have a different ordering of arguments without a wrapper.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (HADOOP-8007) HA: use substitution token for fencing argument

Posted by "Todd Lipcon (Assigned) (JIRA)" <ji...@apache.org>.

     [ https://issues.apache.org/jira/browse/HADOOP-8007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Todd Lipcon reassigned HADOOP-8007:
-----------------------------------

    Assignee: Todd Lipcon  (was: Eli Collins)
    
> HA: use substitution token for fencing argument
> -----------------------------------------------
>
>                 Key: HADOOP-8007
>                 URL: https://issues.apache.org/jira/browse/HADOOP-8007
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: ha
>    Affects Versions: 0.24.0
>            Reporter: Aaron T. Myers
>            Assignee: Todd Lipcon
>
> Per HADOOP-7983 currently the fencer always passes the target host:port to fence as the first argument to the fence script, it would be better to use a substitution token. That is to say, the user would configure "myfence.sh $TARGETHOST foo bar" and Hadoop would substitute the target. This would allow use of pre-existing scripts that might have a different ordering of arguments without a wrapper.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HADOOP-8007) HA: use substitution token for fencing argument

Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.

    [ https://issues.apache.org/jira/browse/HADOOP-8007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13246098#comment-13246098 ] 

Hadoop QA commented on HADOOP-8007:
-----------------------------------

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12521287/hadoop-8007.txt
  against trunk revision .

    +1 @author.  The patch does not contain any @author tags.

    +1 tests included.  The patch appears to include 3 new or modified tests.

    +1 javadoc.  The javadoc tool did not generate any warning messages.

    +1 javac.  The applied patch does not increase the total number of javac compiler warnings.

    +1 eclipse:eclipse.  The patch built with eclipse:eclipse.

    +1 findbugs.  The patch does not introduce any new Findbugs (version 1.3.9) warnings.

    +1 release audit.  The applied patch does not increase the total number of release audit warnings.

    -1 core tests.  The patch failed these unit tests:
                  org.apache.hadoop.fs.viewfs.TestViewFsTrash
                  org.apache.hadoop.ha.TestZKFailoverController

    +1 contrib tests.  The patch passed contrib unit tests.

Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/818//testReport/
Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/818//console

This message is automatically generated.
                
> HA: use substitution token for fencing argument
> -----------------------------------------------
>
>                 Key: HADOOP-8007
>                 URL: https://issues.apache.org/jira/browse/HADOOP-8007
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: ha
>    Affects Versions: 2.0.0
>            Reporter: Aaron T. Myers
>            Assignee: Todd Lipcon
>         Attachments: hadoop-8007.txt, hadoop-8007.txt
>
>
> Per HADOOP-7983 currently the fencer always passes the target host:port to fence as the first argument to the fence script, it would be better to use a substitution token. That is to say, the user would configure "myfence.sh $TARGETHOST foo bar" and Hadoop would substitute the target. This would allow use of pre-existing scripts that might have a different ordering of arguments without a wrapper.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HADOOP-8007) HA: use substitution token for fencing argument

Posted by "Aaron T. Myers (Updated) (JIRA)" <ji...@apache.org>.

     [ https://issues.apache.org/jira/browse/HADOOP-8007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Aaron T. Myers updated HADOOP-8007:
-----------------------------------

     Target Version/s: 0.24.0  (was: HA Branch (HDFS-1623))
    Affects Version/s:     (was: HA Branch (HDFS-1623))
                       0.24.0

Converting to top-level issue with commit of HADOOP-7454.
                
> HA: use substitution token for fencing argument
> -----------------------------------------------
>
>                 Key: HADOOP-8007
>                 URL: https://issues.apache.org/jira/browse/HADOOP-8007
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: ha
>    Affects Versions: 0.24.0
>            Reporter: Eli Collins
>            Assignee: Eli Collins
>
> Per HADOOP-7983 currently the fencer always passes the target host:port to fence as the first argument to the fence script, it would be better to use a substitution token. That is to say, the user would configure "myfence.sh $TARGETHOST foo bar" and Hadoop would substitute the target. This would allow use of pre-existing scripts that might have a different ordering of arguments without a wrapper.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HADOOP-8007) HA: use substitution token for fencing argument

Posted by "Todd Lipcon (Updated) (JIRA)" <ji...@apache.org>.

     [ https://issues.apache.org/jira/browse/HADOOP-8007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Todd Lipcon updated HADOOP-8007:
--------------------------------

     Target Version/s: 2.0.0  (was: 0.24.0)
    Affects Version/s:     (was: 0.24.0)
                       2.0.0
               Status: Patch Available  (was: Open)
    
> HA: use substitution token for fencing argument
> -----------------------------------------------
>
>                 Key: HADOOP-8007
>                 URL: https://issues.apache.org/jira/browse/HADOOP-8007
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: ha
>    Affects Versions: 2.0.0
>            Reporter: Aaron T. Myers
>            Assignee: Todd Lipcon
>         Attachments: hadoop-8007.txt
>
>
> Per HADOOP-7983 currently the fencer always passes the target host:port to fence as the first argument to the fence script, it would be better to use a substitution token. That is to say, the user would configure "myfence.sh $TARGETHOST foo bar" and Hadoop would substitute the target. This would allow use of pre-existing scripts that might have a different ordering of arguments without a wrapper.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HADOOP-8007) HA: use substitution token for fencing argument

Posted by "Eli Collins (Commented) (JIRA)" <ji...@apache.org>.

    [ https://issues.apache.org/jira/browse/HADOOP-8007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13246080#comment-13246080 ] 

Eli Collins commented on HADOOP-8007:
-------------------------------------

+1  looks great
                
> HA: use substitution token for fencing argument
> -----------------------------------------------
>
>                 Key: HADOOP-8007
>                 URL: https://issues.apache.org/jira/browse/HADOOP-8007
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: ha
>    Affects Versions: 2.0.0
>            Reporter: Aaron T. Myers
>            Assignee: Todd Lipcon
>         Attachments: hadoop-8007.txt
>
>
> Per HADOOP-7983 currently the fencer always passes the target host:port to fence as the first argument to the fence script, it would be better to use a substitution token. That is to say, the user would configure "myfence.sh $TARGETHOST foo bar" and Hadoop would substitute the target. This would allow use of pre-existing scripts that might have a different ordering of arguments without a wrapper.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira