[jira] [Commented] (HADOOP-7777) Implement a base class for DNSToSwitchMapping implementations that can offer extra topology information

["Steve Loughran (Commented) (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HADOOP-7777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13138645#comment-13138645 ] 

Steve Loughran commented on HADOOP-7777:
----------------------------------------

This adds a new base class {{AbstractDNSToSwitchMapping}} to offer a common implementation class for the bundled DNS mappings, and a new predicate {{isSingleSwitch()}} which should be true if
a class believes that its topology is only on a single switch.

I initially tried to have the base class extend {{Configurable}} but that triggered calls to things like {{ScriptBasedMapping.setConf()}} from Configurable's constructor, which had lots of consequences as the subclasses weren't ready for that yet. I added the workarounds -checks for other fields being initialised- but this left the code brittle and made it harder to make any custom mappers subclasses of this. I reverted those changes and now the {{AbstractDNSToSwitchMapping}} class just implements {{Configurable}}.

This base class (and the sub classes) do what I wanted, namely indicate whether or not they are single switch or not, with the presumption being "multi-switch unless stated".

# {{ScriptBasedMapping}} is single rack if it has no script.
# {{CachedDNSToSwitchMapping}} relays its query to the raw mapping.
# {{StaticMapping}} says it is single rack if its map is empty (that is, all resolved nodes
will be reported as being in the default rack).
There's tests for all this, and in {{TestSwitchMapping}} verification that implementations of {{DNSToSwitchMapping}} are always considered multi-switch. The tests also see what happens when you pass down null configurations (the existing stuff mostly broke)

Once committed I can fix HDFS BlockManager to use the switch predicate to decide whether to assume multi-rack placement or not.
                
> Implement a base class for DNSToSwitchMapping implementations that can offer extra topology information
> -------------------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-7777
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7777
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: util
>    Affects Versions: 0.23.0
>            Reporter: Steve Loughran
>            Assignee: Steve Loughran
>         Attachments: HADOOP-7777-switch.patch, HADOOP-7777-switch.patch
>
>
> HDFS-2492 has identified a need for DNSToSwitchMapping implementations to provide a bit more topology information (e.g. whether or not there are multiple switches). This could be done by writing an extended interface, querying its methods if present and coming up with a default action if there is no extended interface. 
> Alternatively, we have a base class that all the standard mappings implement, with a boolean isMultiRack() method; all the standard subclasses would extend this, as could any third party topology provider. The advantage of this approach is that it is easier to add new operations without going into a multi-interface mess.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira