You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2021/08/09 06:22:32 UTC
[GitHub] [spark] beliefer opened a new pull request #33684: [WIP][SPARK-36429][SQL] JacksonParser should throw exception when data type unsupported.
beliefer opened a new pull request #33684:
URL: https://github.com/apache/spark/pull/33684
### What changes were proposed in this pull request?
Currently, when `set spark.sql.timestampType=TIMESTAMP_NTZ`, the behavior is different between `from_json` and `from_csv`.
```
-- !query
select from_json('{"t":"26/October/2015"}', 't Timestamp', map('timestampFormat', 'dd/MMMMM/yyyy'))
-- !query schema
struct<from_json({"t":"26/October/2015"}):struct<t:timestamp_ntz>>
-- !query output
{"t":null}
```
```
-- !query
select from_csv('26/October/2015', 't Timestamp', map('timestampFormat', 'dd/MMMMM/yyyy'))
-- !query schema
struct<>
-- !query output
java.lang.Exception
Unsupported type: timestamp_ntz
```
We should make `from_json` throws exception too.
This PR fix the discussion below
https://github.com/apache/spark/pull/33640#discussion_r682862523
### Why are the changes needed?
Make the behavior of `from_json` more reasonable.
### Does this PR introduce _any_ user-facing change?
'Yes'.
from_json throwing Exception when we set spark.sql.timestampType=TIMESTAMP_NTZ.
### How was this patch tested?
Tests updated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33684: [WIP][SPARK-36429][SQL] JacksonParser should throw exception when data type unsupported.
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33684:
URL: https://github.com/apache/spark/pull/33684#issuecomment-895002324
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46723/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33684: [SPARK-36429][SQL] JacksonParser should throw exception when data type unsupported.
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33684:
URL: https://github.com/apache/spark/pull/33684#issuecomment-895881472
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46774/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #33684: [SPARK-36429][SQL] JacksonParser should throw exception when data type unsupported.
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #33684:
URL: https://github.com/apache/spark/pull/33684#issuecomment-895840360
**[Test build #142267 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142267/testReport)** for PR 33684 at commit [`445140d`](https://github.com/apache/spark/commit/445140dd34c03b49c215d8a2daec5c095c1b1ddb).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33684: [SPARK-36429][SQL] JacksonParser should throw exception when data type unsupported.
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33684:
URL: https://github.com/apache/spark/pull/33684#issuecomment-895783634
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/142247/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33684: [SPARK-36429][SQL] JacksonParser should throw exception when data type unsupported.
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33684:
URL: https://github.com/apache/spark/pull/33684#issuecomment-896026417
**[Test build #142267 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142267/testReport)** for PR 33684 at commit [`445140d`](https://github.com/apache/spark/commit/445140dd34c03b49c215d8a2daec5c095c1b1ddb).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33684: [SPARK-36429][SQL] JacksonParser should throw exception when data type unsupported.
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33684:
URL: https://github.com/apache/spark/pull/33684#issuecomment-895772321
**[Test build #142247 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142247/testReport)** for PR 33684 at commit [`4f0df82`](https://github.com/apache/spark/commit/4f0df82fd0c391944d754d3ff72ea0681e024d31).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33684: [SPARK-36429][SQL] JacksonParser should throw exception when data type unsupported.
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33684:
URL: https://github.com/apache/spark/pull/33684#issuecomment-895673727
**[Test build #142247 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142247/testReport)** for PR 33684 at commit [`4f0df82`](https://github.com/apache/spark/commit/4f0df82fd0c391944d754d3ff72ea0681e024d31).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33684: [WIP][SPARK-36429][SQL] JacksonParser should throw exception when data type unsupported.
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33684:
URL: https://github.com/apache/spark/pull/33684#issuecomment-895029634
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46723/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33684: [SPARK-36429][SQL] JacksonParser should throw exception when data type unsupported.
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33684:
URL: https://github.com/apache/spark/pull/33684#issuecomment-895688926
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46754/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33684: [WIP][SPARK-36429][SQL] JacksonParser should throw exception when data type unsupported.
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33684:
URL: https://github.com/apache/spark/pull/33684#issuecomment-895099693
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/142210/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #33684: [SPARK-36429][SQL] JacksonParser should throw exception when data type unsupported.
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #33684:
URL: https://github.com/apache/spark/pull/33684#discussion_r685781306
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JacksonParser.scala
##########
@@ -330,12 +330,15 @@ class JacksonParser(
case udt: UserDefinedType[_] =>
makeConverter(udt.sqlType)
- case _ =>
+ case _: NullType =>
(parser: JsonParser) =>
// Here, we pass empty `PartialFunction` so that this case can be
// handled as a failed conversion. It will throw an exception as
// long as the value is not null.
Review comment:
shall we update the comment?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #33684: [SPARK-36429][SQL] JacksonParser should throw exception when data type unsupported.
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #33684:
URL: https://github.com/apache/spark/pull/33684#issuecomment-895673727
**[Test build #142247 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142247/testReport)** for PR 33684 at commit [`4f0df82`](https://github.com/apache/spark/commit/4f0df82fd0c391944d754d3ff72ea0681e024d31).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33684: [WIP][SPARK-36429][SQL] JacksonParser should throw exception when data type unsupported.
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33684:
URL: https://github.com/apache/spark/pull/33684#issuecomment-895029600
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46723/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33684: [SPARK-36429][SQL] JacksonParser should throw exception when data type unsupported.
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33684:
URL: https://github.com/apache/spark/pull/33684#issuecomment-895703574
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46754/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33684: [SPARK-36429][SQL] JacksonParser should throw exception when data type unsupported.
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33684:
URL: https://github.com/apache/spark/pull/33684#issuecomment-895918889
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46774/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on pull request #33684: [SPARK-36429][SQL] JacksonParser should throw exception when data type unsupported.
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on pull request #33684:
URL: https://github.com/apache/spark/pull/33684#issuecomment-896096691
The GA failure is not related, I'm re-merging it to master/3.2, thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33684: [SPARK-36429][SQL] JacksonParser should throw exception when data type unsupported.
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33684:
URL: https://github.com/apache/spark/pull/33684#issuecomment-895918917
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46774/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33684: [SPARK-36429][SQL] JacksonParser should throw exception when data type unsupported.
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33684:
URL: https://github.com/apache/spark/pull/33684#issuecomment-895840360
**[Test build #142267 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142267/testReport)** for PR 33684 at commit [`445140d`](https://github.com/apache/spark/commit/445140dd34c03b49c215d8a2daec5c095c1b1ddb).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33684: [SPARK-36429][SQL] JacksonParser should throw exception when data type unsupported.
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33684:
URL: https://github.com/apache/spark/pull/33684#issuecomment-895711931
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46754/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33684: [SPARK-36429][SQL] JacksonParser should throw exception when data type unsupported.
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33684:
URL: https://github.com/apache/spark/pull/33684#issuecomment-895783634
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/142247/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on pull request #33684: [SPARK-36429][SQL] JacksonParser should throw exception when data type unsupported.
Posted by GitBox <gi...@apache.org>.
beliefer commented on pull request #33684:
URL: https://github.com/apache/spark/pull/33684#issuecomment-896446050
@cloud-fan Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33684: [SPARK-36429][SQL] JacksonParser should throw exception when data type unsupported.
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33684:
URL: https://github.com/apache/spark/pull/33684#issuecomment-895029634
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on pull request #33684: [SPARK-36429][SQL] JacksonParser should throw exception when data type unsupported.
Posted by GitBox <gi...@apache.org>.
beliefer commented on pull request #33684:
URL: https://github.com/apache/spark/pull/33684#issuecomment-895672859
retest this please
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33684: [SPARK-36429][SQL] JacksonParser should throw exception when data type unsupported.
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33684:
URL: https://github.com/apache/spark/pull/33684#issuecomment-896028033
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/142267/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33684: [WIP][SPARK-36429][SQL] JacksonParser should throw exception when data type unsupported.
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33684:
URL: https://github.com/apache/spark/pull/33684#issuecomment-894979263
**[Test build #142210 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142210/testReport)** for PR 33684 at commit [`4f0df82`](https://github.com/apache/spark/commit/4f0df82fd0c391944d754d3ff72ea0681e024d31).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan closed pull request #33684: [SPARK-36429][SQL] JacksonParser should throw exception when data type unsupported.
Posted by GitBox <gi...@apache.org>.
cloud-fan closed pull request #33684:
URL: https://github.com/apache/spark/pull/33684
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on pull request #33684: [SPARK-36429][SQL] JacksonParser should throw exception when data type unsupported.
Posted by GitBox <gi...@apache.org>.
beliefer commented on pull request #33684:
URL: https://github.com/apache/spark/pull/33684#issuecomment-895796894
ping @cloud-fan
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #33684: [SPARK-36429][SQL] JacksonParser should throw exception when data type unsupported.
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #33684:
URL: https://github.com/apache/spark/pull/33684#discussion_r685781907
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JacksonParser.scala
##########
@@ -330,12 +330,15 @@ class JacksonParser(
case udt: UserDefinedType[_] =>
makeConverter(udt.sqlType)
- case _ =>
+ case _: NullType =>
(parser: JsonParser) =>
// Here, we pass empty `PartialFunction` so that this case can be
// handled as a failed conversion. It will throw an exception as
// long as the value is not null.
parseJsonToken[AnyRef](parser, dataType)(PartialFunction.empty[JsonToken, AnyRef])
Review comment:
should it be
```
(parser: JsonParser) => parseJsonToken[java.lang.Long](parser, dataType) {
case _ => null
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org