[jira] [Created] (ZEPPELIN-1909) spark session with hive support can't be created correctly

["lichenglin (JIRA)" <ji...@apache.org>]

lichenglin created ZEPPELIN-1909:
------------------------------------

             Summary: spark session with hive support can't be created correctly
                 Key: ZEPPELIN-1909
                 URL: https://issues.apache.org/jira/browse/ZEPPELIN-1909
             Project: Zeppelin
          Issue Type: Bug
          Components: Interpreters
    Affects Versions: 0.7.0
            Reporter: lichenglin
            Priority: Blocker
             Fix For: 0.7.0


{code}

    if (useHiveContext()) {
      if (hiveClassesArePresent()) {
        Utils.invokeMethod(builder, "enableHiveSupport");
        sparkSession = Utils.invokeMethod(builder, "getOrCreate");
        logger.info("Created Spark session with Hive support");
      } else {
        Utils.invokeMethod(builder, "config",
            new Class[]{ String.class, String.class},
            new Object[]{ "spark.sql.catalogImplementation", "in-memory"});
        sparkSession = Utils.invokeMethod(builder, "getOrCreate");
        logger.info("Created Spark session with Hive support");
      }
    } 
{code}

{code}
 /**
   * See org.apache.spark.sql.SparkSession.hiveClassesArePresent
   * @return
   */
  private boolean hiveClassesArePresent() {
    try {
      this.getClass().forName("org.apache.spark.sql.hive.HiveSessionState");
      this.getClass().forName("org.apache.spark.sql.hive.HiveSharedState");
      this.getClass().forName("org.apache.hadoop.hive.conf.HiveConf");
      return true;
    } catch (ClassNotFoundException | NoClassDefFoundError e) {
      return false;
    }
  }
{code}

since "org.apache.spark.sql.hive.HiveSharedState" has been removed from spark2.1.x 
So hiveClassesArePresent will always return false.
This will cause spark session with hive support would be created with 
"spark.sql.catalogImplementation=in-memory"

May be just remove 
{code}
      this.getClass().forName("org.apache.spark.sql.hive.HiveSharedState");
{code}

is OK?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)