You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flink.apache.org by "hehuiyuan (Jira)" <ji...@apache.org> on 2021/03/16 15:30:00 UTC
[jira] [Created] (FLINK-21829) create HiveCatalog with custom hadoopconfdir first

hehuiyuan created FLINK-21829:
---------------------------------

             Summary: create HiveCatalog with custom hadoopconfdir first
                 Key: FLINK-21829
                 URL: https://issues.apache.org/jira/browse/FLINK-21829
             Project: Flink
          Issue Type: Wish
          Components: Connectors / Hive
            Reporter: hehuiyuan


here is no prompt when the the path to hadoop conf configured is wrong unintentional.
{code:java}
private static HiveConf createHiveConf(
        @Nullable String hiveConfDir, @Nullable String hadoopConfDir) {
    // create HiveConf from hadoop configuration with hadoop conf directory configured.
    Configuration hadoopConf = null;
    if (isNullOrWhitespaceOnly(hadoopConfDir)) {
        for (String possibleHadoopConfPath :
                HadoopUtils.possibleHadoopConfPaths(
                        new org.apache.flink.configuration.Configuration())) {
            hadoopConf = getHadoopConfiguration(possibleHadoopConfPath);
            if (hadoopConf != null) {
                break;
            }
        }
    } else {
        hadoopConf = getHadoopConfiguration(hadoopConfDir);
    }
    if (hadoopConf == null) {
        hadoopConf = new Configuration();
    }
    HiveConf hiveConf = new HiveConf(hadoopConf, HiveConf.class);

{code}

It is better to load hadoop conf from possiable hadoop path when the path is wrong.

(1) try to load from the custom hadoop conf path
(2) try to load from possiable hadoop conf path if {{Configuration hadoopConf}} is null.
(3) new Configuration if {{Configuration hadoopConf}} is null
{code:java}

private static HiveConf createHiveConf(
        @Nullable String hiveConfDir, @Nullable String hadoopConfDir) {
    // create HiveConf from hadoop configuration with hadoop conf directory configured.
    Configuration hadoopConf = null;
    if (!isNullOrWhitespaceOnly(hadoopConfDir)) {
        hadoopConf = getHadoopConfiguration(hadoopConfDir);
    }
    if (hadoopConf == null) {
        for (String possibleHadoopConfPath :
                HadoopUtils.possibleHadoopConfPaths(
                        new org.apache.flink.configuration.Configuration())) {
            hadoopConf = getHadoopConfiguration(possibleHadoopConfPath);
            if (hadoopConf != null) {
                break;
            }
        }
    }
    if (hadoopConf == null) {
        hadoopConf = new Configuration();
    }
    HiveConf hiveConf = new HiveConf(hadoopConf, HiveConf.class);

{code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)