You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Simeon H.K. Fitch (JIRA)" <ji...@apache.org> on 2019/05/21 13:05:00 UTC
[jira] [Commented] (SPARK-23696) StructType.fromString swallows
exceptions from DataType.fromJson
[ https://issues.apache.org/jira/browse/SPARK-23696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16844808#comment-16844808 ]
Simeon H.K. Fitch commented on SPARK-23696:
-------------------------------------------
[~hyukjin.kwon] is there a commit associated with this resolution?
> StructType.fromString swallows exceptions from DataType.fromJson
> ----------------------------------------------------------------
>
> Key: SPARK-23696
> URL: https://issues.apache.org/jira/browse/SPARK-23696
> Project: Spark
> Issue Type: Bug
> Components: ML
> Affects Versions: 2.2.1
> Reporter: Simeon H.K. Fitch
> Priority: Trivial
> Labels: bulk-closed
>
> `StructType.fromString` swallows exceptions from `DataType.fromJson`, assuming they are an indication that the `LegacyTypeStringParser.parse` should be called instead. When that fails (because it throws an excreption), an error message is generated that does not reflect the true problem at hand, effectively swallowing the exception from `DataType.fromJson`. This makes debugging Parquet schema issues more difficult.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org