[jira] [Created] (SPARK-23089) "Unable to create operation log session directory" when parent directory not present

["Sean Roberts (JIRA)" <ji...@apache.org>]

Sean Roberts created SPARK-23089:
------------------------------------

             Summary: "Unable to create operation log session directory" when parent directory not present
                 Key: SPARK-23089
                 URL: https://issues.apache.org/jira/browse/SPARK-23089
             Project: Spark
          Issue Type: Bug
          Components: SQL
    Affects Versions: 2.2.0
         Environment: When creating a session directory, Thrift should create the parent directory _(i.e. /tmp/hive/operation_logs)_ if it is not present.

It's common for operators to clean-up old and empty directories in /tmp, or to have tools (systemd-tmpfiles or tmpwatch) that do it automatically.

This was fixed in HIVE-12262 but not in Spark Thrift as seen by this:
{code}18/01/15 14:22:49 WARN HiveSessionImpl: Unable to create operation log session directory: /tmp/hive/operation_logs/683a6318-adc4-42c4-b665-11dad14d7ec7{code}

Resolved by manually creating /tmp/hive/operation_logs/

We are experiencing this with Spark Thrift provided by: /usr/hdp/2.6.3.0-235/spark2/jars/spark-hive-thriftserver_2.11-2.2.0.2.6.3.0-235.jar
            Reporter: Sean Roberts






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org