You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Sean Roberts (JIRA)" <ji...@apache.org> on 2018/01/16 15:32:00 UTC
[jira] [Updated] (SPARK-23089) "Unable to create operation log
session directory" when parent directory not present
[ https://issues.apache.org/jira/browse/SPARK-23089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sean Roberts updated SPARK-23089:
---------------------------------
Description:
When creating a session directory, Thrift should create the parent directory _(i.e. /tmp/hive/operation_logs)_ if it is not present.
It's common for operators to clean-up old and empty directories in /tmp, or to have tools (systemd-tmpfiles or tmpwatch) that do it automatically.
This was fixed in HIVE-12262 but not in Spark Thrift as seen by this:
{code}18/01/15 14:22:49 WARN HiveSessionImpl: Unable to create operation log session directory: /tmp/hive/operation_logs/683a6318-adc4-42c4-b665-11dad14d7ec7{code}
Resolved by manually creating /tmp/hive/operation_logs/
We are experiencing this with Spark Thrift provided by: /usr/hdp/2.6.3.0-235/spark2/jars/spark-hive-thriftserver_2.11-2.2.0.2.6.3.0-235.jar
> "Unable to create operation log session directory" when parent directory not present
> ------------------------------------------------------------------------------------
>
> Key: SPARK-23089
> URL: https://issues.apache.org/jira/browse/SPARK-23089
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 2.2.0
> Reporter: Sean Roberts
> Priority: Minor
>
> When creating a session directory, Thrift should create the parent directory _(i.e. /tmp/hive/operation_logs)_ if it is not present.
> It's common for operators to clean-up old and empty directories in /tmp, or to have tools (systemd-tmpfiles or tmpwatch) that do it automatically.
> This was fixed in HIVE-12262 but not in Spark Thrift as seen by this:
> {code}18/01/15 14:22:49 WARN HiveSessionImpl: Unable to create operation log session directory: /tmp/hive/operation_logs/683a6318-adc4-42c4-b665-11dad14d7ec7{code}
> Resolved by manually creating /tmp/hive/operation_logs/
> We are experiencing this with Spark Thrift provided by: /usr/hdp/2.6.3.0-235/spark2/jars/spark-hive-thriftserver_2.11-2.2.0.2.6.3.0-235.jar
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org