You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Maryann Xue (JIRA)" <ji...@apache.org> on 2019/03/20 21:14:00 UTC
[jira] [Updated] (SPARK-27223) Remove private methods that skip
conversion when passing user schemas for constructing a DataFrame
[ https://issues.apache.org/jira/browse/SPARK-27223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Maryann Xue updated SPARK-27223:
--------------------------------
Summary: Remove private methods that skip conversion when passing user schemas for constructing a DataFrame (was: Remove private methods that allow no conversion when passing user schemas for constructing a DataFrame)
> Remove private methods that skip conversion when passing user schemas for constructing a DataFrame
> --------------------------------------------------------------------------------------------------
>
> Key: SPARK-27223
> URL: https://issues.apache.org/jira/browse/SPARK-27223
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 2.4.0
> Reporter: Maryann Xue
> Priority: Minor
>
> When passing in a user schema to create a DataFrame, there might be mismatched nullability between the user schema and the the actual data. All related public interfaces now perform catalyst conversion using the user provided schema, which catches such mismatches to avoid runtime errors later on. However, there're private methods which allow this conversion to be skipped, so we need to remove these private methods which may lead to confusion and potential issues.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org