[jira] Commented: (HIVE-232) metastore.warehouse configuration should use inherited hadoop configuration

["Joydeep Sen Sarma (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HIVE-232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12663990#action_12663990 ] 

Joydeep Sen Sarma commented on HIVE-232:
----------------------------------------

ok with the change in the metastore. DB.java also has a reference to METASTOREWAREHOUSEDIR - but not clear that it's being used at all.

the change in LoadSemanticAnalyzer is probably not necessary (a test case might have made this clear). if i am reading this right - the net effect being desired is that the the from path should use fs.default.name for scheme/authority - right?

but this was already happening, a little twined but reading the (current) code:

    fs = FileSystem.get(fromURI, conf);
    String fromAuthority = null;

    // fall back to configuration based scheme if necessary
    if(StringUtils.isEmpty(fromScheme)) {
      fromScheme = fs.getUri().getScheme();
      fromAuthority = fs.getUri().getAuthority();
    }

    if(fromScheme.equals("hdfs")) {
      fromAuthority = StringUtils.isEmpty(fromURI.getAuthority()) ?
        fs.getUri().getAuthority() : fromURI.getAuthority();
    }

the fs initialization will get the scheme and authority from fs.default.name if the fromURI does not supply them. Then we test one by one if scheme and authority are already supplied - if not - we use the one supplied by fs (so use fs.default.name for both of these as a default).

unless i am missing something - we are just trying to do the same thing in a slightly different way?




> metastore.warehouse configuration should use inherited hadoop configuration
> ---------------------------------------------------------------------------
>
>                 Key: HIVE-232
>                 URL: https://issues.apache.org/jira/browse/HIVE-232
>             Project: Hadoop Hive
>          Issue Type: Bug
>            Reporter: Josh Ferguson
>         Attachments: hive-232.patch
>
>
> the hive.metastore.warehouse.dir configuration property in hive-*.xml needs to use the protocol, host, and port when it is inherited from the fs.name property in hadoop-site.xml.
> When it doesn't and no protocol is found then a broad range of "Move" operations when the source and target are both in the DFS will fail.
> Currently this can be worked around by prepending the protocol, host and port of the hadoop nameserver into the value of the hive.metastore.warehouse.dir property.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.