You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Apache Spark (JIRA)" <ji...@apache.org> on 2018/01/16 19:48:00 UTC
[jira] [Commented] (SPARK-23089) "Unable to create operation log
session directory" when parent directory not present
[ https://issues.apache.org/jira/browse/SPARK-23089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16327674#comment-16327674 ]
Apache Spark commented on SPARK-23089:
--------------------------------------
User 'mgaido91' has created a pull request for this issue:
https://github.com/apache/spark/pull/20281
> "Unable to create operation log session directory" when parent directory not present
> ------------------------------------------------------------------------------------
>
> Key: SPARK-23089
> URL: https://issues.apache.org/jira/browse/SPARK-23089
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 2.2.0
> Environment: /usr/hdp/2.6.3.0-235/spark2/jars/spark-hive-thriftserver_2.11-2.2.0.2.6.3.0-235.jar
> $ cat /etc/redhat-release
> Red Hat Enterprise Linux Server release 7.4 (Maipo)
> $ ps aux|grep ^hive.*spark.*thrift
> hive 1468503 0.9 0.5 13319628 1411676 ? Sl Jan15 10:18 /usr/java/default/bin/java -Dhdp.version=2.6.3.0-235 -cp /usr/hdp/current/spark2-thriftserver/conf/:/usr/hdp/current/spark2-thriftserver/jars/*:/usr/hdp/current/hadoop-client/conf/ -Xmx2048m org.apache.spark.deploy.SparkSubmit --properties-file /usr/hdp/current/spark2-thriftserver/conf/spark-thrift-sparkconf.conf --class org.apache.spark.sql.hive.thriftserver.HiveThriftServer2 --name Thrift JDBC/ODBC Server spark-internal
> Reporter: Sean Roberts
> Priority: Minor
>
> When creating a session directory, Thrift should create the parent directory _(i.e. /tmp/hive/operation_logs)_ if it is not present.
> It's common for operators to clean-up old and empty directories in /tmp, or to have tools (systemd-tmpfiles or tmpwatch) that do it automatically.
> This was fixed in HIVE-12262 but not in Spark Thrift as seen by this:
> {code}18/01/15 14:22:49 WARN HiveSessionImpl: Unable to create operation log session directory: /tmp/hive/operation_logs/683a6318-adc4-42c4-b665-11dad14d7ec7{code}
> Resolved by manually creating /tmp/hive/operation_logs/
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org