You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Navya Krishnappa (JIRA)" <ji...@apache.org> on 2017/03/30 12:44:41 UTC
[jira] [Comment Edited] (SPARK-18877) Unable to read given csv
data. Excepion: java.lang.IllegalArgumentException: requirement failed:
Decimal precision 28 exceeds max precision 20
[ https://issues.apache.org/jira/browse/SPARK-18877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15754865#comment-15754865 ]
Navya Krishnappa edited comment on SPARK-18877 at 3/30/17 12:44 PM:
--------------------------------------------------------------------
I'm using SparkContext.read() to read the content. Refer the given code using to read the csv file.
Dataset dataset = getSqlContext().read()
.option(HEADER, "true")
.option(PARSER_LIB, "commons")
.option(INFER_SCHEMA, "true")
.option(DELIMITER, ",")
.option(QUOTE, "\"")
.option(ESCAPE, "\\")
.option(MODE, Mode.PERMISSIVE)
.csv(sourceFile);
if we collect the dataset (dataset.collect()). i will get java.lang.IllegalArgumentException: requirement failed: Decimal precision 28 exceeds max precision 20 exception.
was (Author: navya krishnappa):
I'm using SparkContext.read() to read the content. Refer the given code using to read the csv file.
Dataset dataset = getSqlContext().read()
.option(DAWBConstant.HEADER, "true")
.option(DAWBConstant.PARSER_LIB, "commons")
.option(DAWBConstant.INFER_SCHEMA, "true")
.option(DAWBConstant.DELIMITER, ",")
.option(DAWBConstant.QUOTE, "\"")
.option(DAWBConstant.ESCAPE, "\\")
.option(DAWBConstant.MODE, Mode.PERMISSIVE)
.csv(sourceFile);
if we collect the dataset (dataset.collect()). i will get java.lang.IllegalArgumentException: requirement failed: Decimal precision 28 exceeds max precision 20 exception.
> Unable to read given csv data. Excepion: java.lang.IllegalArgumentException: requirement failed: Decimal precision 28 exceeds max precision 20
> ----------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: SPARK-18877
> URL: https://issues.apache.org/jira/browse/SPARK-18877
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 2.0.2
> Reporter: Navya Krishnappa
> Assignee: Dongjoon Hyun
> Fix For: 2.0.3, 2.1.1, 2.2.0
>
>
> When reading below mentioned csv data, even though the maximum decimal precision is 38, following exception is thrown java.lang.IllegalArgumentException: requirement failed: Decimal precision 28 exceeds max precision 20
> Decimal
> 2323366225312000000000000000
> 24335739714000000
> 23233662253000
> 232336622530000
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org