You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Hyukjin Kwon (Jira)" <ji...@apache.org> on 2020/01/20 05:00:08 UTC
[jira] [Resolved] (SPARK-30554) Return Iterable from
FailureSafeParser.rawParser
[ https://issues.apache.org/jira/browse/SPARK-30554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hyukjin Kwon resolved SPARK-30554.
----------------------------------
Fix Version/s: 3.0.0
Resolution: Fixed
Issue resolved by pull request 27264
[https://github.com/apache/spark/pull/27264]
> Return Iterable from FailureSafeParser.rawParser
> ------------------------------------------------
>
> Key: SPARK-30554
> URL: https://issues.apache.org/jira/browse/SPARK-30554
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 2.4.4
> Reporter: Maxim Gekk
> Assignee: Maxim Gekk
> Priority: Minor
> Fix For: 3.0.0
>
>
> Currently, rawParser of FailureSafeParser has the signature `IN => Seq[InternalRow]` which is strict requirement for underlying rawParser. It could return Option, for example like CSV datasource. The ticket aims to refactor FailureSafeParser, and change Seq to Iterable. Also. need to modify CSV and JSON datasources where FailureSafeParser is used, basically.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org