You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Mihailo Aleksic (Jira)" <ji...@apache.org> on 2025/08/08 07:31:00 UTC
[jira] [Updated] (SPARK-53175) Override toString in TableSpec to make it deterministic
[ https://issues.apache.org/jira/browse/SPARK-53175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mihailo Aleksic updated SPARK-53175:
------------------------------------
Description:
In this issue I propose that we override TableSpec.toString. Sometimes we can get an ArrayBuffer and sometimes we can get a List for TableSpec.constraints when creating a string version of the plan (for example in the representation of the analyzed plan). For example when we create a table of third party data source (Delta) sometimes we get:
CreateTableAsSelect TableSpec(Map(),Some(delta),Map(),None,None,None,None,false,false,Set(),None,None,None,List()), false, true
and sometimes
CreateTableAsSelect TableSpec(Map(),Some(delta),Map(),None,None,None,None,false,false,Set(),None,None,None,ArrayBuffer()), false, true
This is non deterministic so I propose that we fix it.
was:In this issue I propose that we override TableSpec.toString. Sometimes we can get an ArrayBuffer and sometimes we can get a List for TableSpec.constraints when creating a string version of the plan (for example in the representation of the analyzed plan). This is non deterministic so I propose that we fix it.
> Override toString in TableSpec to make it deterministic
> -------------------------------------------------------
>
> Key: SPARK-53175
> URL: https://issues.apache.org/jira/browse/SPARK-53175
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 4.0.0
> Reporter: Mihailo Aleksic
> Priority: Major
> Fix For: 4.1.0
>
>
> In this issue I propose that we override TableSpec.toString. Sometimes we can get an ArrayBuffer and sometimes we can get a List for TableSpec.constraints when creating a string version of the plan (for example in the representation of the analyzed plan). For example when we create a table of third party data source (Delta) sometimes we get:
> CreateTableAsSelect TableSpec(Map(),Some(delta),Map(),None,None,None,None,false,false,Set(),None,None,None,List()), false, true
> and sometimes
> CreateTableAsSelect TableSpec(Map(),Some(delta),Map(),None,None,None,None,false,false,Set(),None,None,None,ArrayBuffer()), false, true
> This is non deterministic so I propose that we fix it.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org