You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Apache Spark (Jira)" <ji...@apache.org> on 2020/09/08 07:41:00 UTC

[jira] [Assigned] (SPARK-32818) Make hive metastore convert config can be changed session level

     [ https://issues.apache.org/jira/browse/SPARK-32818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Apache Spark reassigned SPARK-32818:
------------------------------------

    Assignee: Apache Spark

> Make hive metastore convert config  can be changed session level
> ----------------------------------------------------------------
>
>                 Key: SPARK-32818
>                 URL: https://issues.apache.org/jira/browse/SPARK-32818
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>    Affects Versions: 3.1.0
>            Reporter: angerszhu
>            Assignee: Apache Spark
>            Priority: Major
>
> There are some cases that we have history hive  table, in origin, it use textfile serde, later  it's change to orc serde, since during convert, it will use table level serde to scan all partition, if some old partition file format is not same , will failed to convert , we need to close metastoer convert and restart spark program.
> It's bad for adhoc engine such as long running spark thrift server. so I think it's necessary to make these two config can be changed for use in session level



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org