You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Xin Ren (JIRA)" <ji...@apache.org> on 2019/07/29 17:36:00 UTC
[jira] [Commented] (SPARK-28555) Recover options and properties and
pass them back into the v1 API
[ https://issues.apache.org/jira/browse/SPARK-28555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16895470#comment-16895470 ]
Xin Ren commented on SPARK-28555:
---------------------------------
I'm working on it :)
> Recover options and properties and pass them back into the v1 API
> -----------------------------------------------------------------
>
> Key: SPARK-28555
> URL: https://issues.apache.org/jira/browse/SPARK-28555
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
> Affects Versions: 3.0.0
> Reporter: Xin Ren
> Priority: Minor
>
> When tables are created, the {{CREATE TABLE}} syntax supports both {{TBLPROPERTIES}} and {{OPTIONS}}. Options were used in v1 to configure the table itself, like options passed to {{DataFrameReader}}. Right now, both properties and options are stored in v2 table properties, because v2 only has properties, not both. But, we aren't able to recover which properties were set through {{OPTIONS}} and which were set through {{TBLPROPERTIES}}.
> Instead of the current behavior, I think options should be prefixed with {{option.}}. That way, we can recover options and properties and pass them back into the v1 API.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org