You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2021/08/09 06:22:32 UTC

[GitHub] [spark] beliefer opened a new pull request #33684: [WIP][SPARK-36429][SQL] JacksonParser should throw exception when data type unsupported.

beliefer opened a new pull request #33684:
URL: https://github.com/apache/spark/pull/33684


   ### What changes were proposed in this pull request?
   Currently, when `set spark.sql.timestampType=TIMESTAMP_NTZ`, the behavior is different between `from_json` and `from_csv`.
   ```
   -- !query
   select from_json('{"t":"26/October/2015"}', 't Timestamp', map('timestampFormat', 'dd/MMMMM/yyyy'))
   -- !query schema
   struct<from_json({"t":"26/October/2015"}):struct<t:timestamp_ntz>>
   -- !query output
   {"t":null}
   ```
   
   ```
   -- !query
   select from_csv('26/October/2015', 't Timestamp', map('timestampFormat', 'dd/MMMMM/yyyy'))
   -- !query schema
   struct<>
   -- !query output
   java.lang.Exception
   Unsupported type: timestamp_ntz
   ```
   
   We should make `from_json` throws exception too.
   This PR fix the discussion below
   https://github.com/apache/spark/pull/33640#discussion_r682862523
   
   
   ### Why are the changes needed?
   Make the behavior of `from_json` more reasonable.
   
   
   ### Does this PR introduce _any_ user-facing change?
   'Yes'.
   from_json throwing Exception when we set spark.sql.timestampType=TIMESTAMP_NTZ.
   
   
   ### How was this patch tested?
   Tests updated.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #33684: [WIP][SPARK-36429][SQL] JacksonParser should throw exception when data type unsupported.

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33684:
URL: https://github.com/apache/spark/pull/33684#issuecomment-895002324


   Kubernetes integration test starting
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46723/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #33684: [SPARK-36429][SQL] JacksonParser should throw exception when data type unsupported.

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33684:
URL: https://github.com/apache/spark/pull/33684#issuecomment-895881472


   Kubernetes integration test starting
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46774/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA removed a comment on pull request #33684: [SPARK-36429][SQL] JacksonParser should throw exception when data type unsupported.

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #33684:
URL: https://github.com/apache/spark/pull/33684#issuecomment-895840360


   **[Test build #142267 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142267/testReport)** for PR 33684 at commit [`445140d`](https://github.com/apache/spark/commit/445140dd34c03b49c215d8a2daec5c095c1b1ddb).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins removed a comment on pull request #33684: [SPARK-36429][SQL] JacksonParser should throw exception when data type unsupported.

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33684:
URL: https://github.com/apache/spark/pull/33684#issuecomment-895783634


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/142247/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #33684: [SPARK-36429][SQL] JacksonParser should throw exception when data type unsupported.

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33684:
URL: https://github.com/apache/spark/pull/33684#issuecomment-896026417


   **[Test build #142267 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142267/testReport)** for PR 33684 at commit [`445140d`](https://github.com/apache/spark/commit/445140dd34c03b49c215d8a2daec5c095c1b1ddb).
    * This patch passes all tests.
    * This patch merges cleanly.
    * This patch adds no public classes.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #33684: [SPARK-36429][SQL] JacksonParser should throw exception when data type unsupported.

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33684:
URL: https://github.com/apache/spark/pull/33684#issuecomment-895772321


   **[Test build #142247 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142247/testReport)** for PR 33684 at commit [`4f0df82`](https://github.com/apache/spark/commit/4f0df82fd0c391944d754d3ff72ea0681e024d31).
    * This patch passes all tests.
    * This patch merges cleanly.
    * This patch adds no public classes.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #33684: [SPARK-36429][SQL] JacksonParser should throw exception when data type unsupported.

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33684:
URL: https://github.com/apache/spark/pull/33684#issuecomment-895673727


   **[Test build #142247 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142247/testReport)** for PR 33684 at commit [`4f0df82`](https://github.com/apache/spark/commit/4f0df82fd0c391944d754d3ff72ea0681e024d31).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins commented on pull request #33684: [WIP][SPARK-36429][SQL] JacksonParser should throw exception when data type unsupported.

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33684:
URL: https://github.com/apache/spark/pull/33684#issuecomment-895029634


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46723/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #33684: [SPARK-36429][SQL] JacksonParser should throw exception when data type unsupported.

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33684:
URL: https://github.com/apache/spark/pull/33684#issuecomment-895688926


   Kubernetes integration test starting
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46754/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins commented on pull request #33684: [WIP][SPARK-36429][SQL] JacksonParser should throw exception when data type unsupported.

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33684:
URL: https://github.com/apache/spark/pull/33684#issuecomment-895099693


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/142210/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] cloud-fan commented on a change in pull request #33684: [SPARK-36429][SQL] JacksonParser should throw exception when data type unsupported.

Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #33684:
URL: https://github.com/apache/spark/pull/33684#discussion_r685781306



##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JacksonParser.scala
##########
@@ -330,12 +330,15 @@ class JacksonParser(
     case udt: UserDefinedType[_] =>
       makeConverter(udt.sqlType)
 
-    case _ =>
+    case _: NullType =>
       (parser: JsonParser) =>
         // Here, we pass empty `PartialFunction` so that this case can be
         // handled as a failed conversion. It will throw an exception as
         // long as the value is not null.

Review comment:
       shall we update the comment?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA removed a comment on pull request #33684: [SPARK-36429][SQL] JacksonParser should throw exception when data type unsupported.

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #33684:
URL: https://github.com/apache/spark/pull/33684#issuecomment-895673727


   **[Test build #142247 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142247/testReport)** for PR 33684 at commit [`4f0df82`](https://github.com/apache/spark/commit/4f0df82fd0c391944d754d3ff72ea0681e024d31).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #33684: [WIP][SPARK-36429][SQL] JacksonParser should throw exception when data type unsupported.

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33684:
URL: https://github.com/apache/spark/pull/33684#issuecomment-895029600


   Kubernetes integration test status failure
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46723/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #33684: [SPARK-36429][SQL] JacksonParser should throw exception when data type unsupported.

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33684:
URL: https://github.com/apache/spark/pull/33684#issuecomment-895703574


   Kubernetes integration test status success
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46754/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #33684: [SPARK-36429][SQL] JacksonParser should throw exception when data type unsupported.

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33684:
URL: https://github.com/apache/spark/pull/33684#issuecomment-895918889


   Kubernetes integration test status failure
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46774/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] cloud-fan commented on pull request #33684: [SPARK-36429][SQL] JacksonParser should throw exception when data type unsupported.

Posted by GitBox <gi...@apache.org>.
cloud-fan commented on pull request #33684:
URL: https://github.com/apache/spark/pull/33684#issuecomment-896096691


   The GA failure is not related, I'm re-merging it to master/3.2, thanks!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins commented on pull request #33684: [SPARK-36429][SQL] JacksonParser should throw exception when data type unsupported.

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33684:
URL: https://github.com/apache/spark/pull/33684#issuecomment-895918917


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46774/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #33684: [SPARK-36429][SQL] JacksonParser should throw exception when data type unsupported.

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33684:
URL: https://github.com/apache/spark/pull/33684#issuecomment-895840360


   **[Test build #142267 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142267/testReport)** for PR 33684 at commit [`445140d`](https://github.com/apache/spark/commit/445140dd34c03b49c215d8a2daec5c095c1b1ddb).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins commented on pull request #33684: [SPARK-36429][SQL] JacksonParser should throw exception when data type unsupported.

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33684:
URL: https://github.com/apache/spark/pull/33684#issuecomment-895711931


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46754/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins commented on pull request #33684: [SPARK-36429][SQL] JacksonParser should throw exception when data type unsupported.

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33684:
URL: https://github.com/apache/spark/pull/33684#issuecomment-895783634


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/142247/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] beliefer commented on pull request #33684: [SPARK-36429][SQL] JacksonParser should throw exception when data type unsupported.

Posted by GitBox <gi...@apache.org>.
beliefer commented on pull request #33684:
URL: https://github.com/apache/spark/pull/33684#issuecomment-896446050


   @cloud-fan Thanks.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins removed a comment on pull request #33684: [SPARK-36429][SQL] JacksonParser should throw exception when data type unsupported.

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33684:
URL: https://github.com/apache/spark/pull/33684#issuecomment-895029634






-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] beliefer commented on pull request #33684: [SPARK-36429][SQL] JacksonParser should throw exception when data type unsupported.

Posted by GitBox <gi...@apache.org>.
beliefer commented on pull request #33684:
URL: https://github.com/apache/spark/pull/33684#issuecomment-895672859


   retest this please


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins commented on pull request #33684: [SPARK-36429][SQL] JacksonParser should throw exception when data type unsupported.

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33684:
URL: https://github.com/apache/spark/pull/33684#issuecomment-896028033


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/142267/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #33684: [WIP][SPARK-36429][SQL] JacksonParser should throw exception when data type unsupported.

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33684:
URL: https://github.com/apache/spark/pull/33684#issuecomment-894979263


   **[Test build #142210 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142210/testReport)** for PR 33684 at commit [`4f0df82`](https://github.com/apache/spark/commit/4f0df82fd0c391944d754d3ff72ea0681e024d31).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] cloud-fan closed pull request #33684: [SPARK-36429][SQL] JacksonParser should throw exception when data type unsupported.

Posted by GitBox <gi...@apache.org>.
cloud-fan closed pull request #33684:
URL: https://github.com/apache/spark/pull/33684


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] beliefer commented on pull request #33684: [SPARK-36429][SQL] JacksonParser should throw exception when data type unsupported.

Posted by GitBox <gi...@apache.org>.
beliefer commented on pull request #33684:
URL: https://github.com/apache/spark/pull/33684#issuecomment-895796894


   ping @cloud-fan 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] cloud-fan commented on a change in pull request #33684: [SPARK-36429][SQL] JacksonParser should throw exception when data type unsupported.

Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #33684:
URL: https://github.com/apache/spark/pull/33684#discussion_r685781907



##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JacksonParser.scala
##########
@@ -330,12 +330,15 @@ class JacksonParser(
     case udt: UserDefinedType[_] =>
       makeConverter(udt.sqlType)
 
-    case _ =>
+    case _: NullType =>
       (parser: JsonParser) =>
         // Here, we pass empty `PartialFunction` so that this case can be
         // handled as a failed conversion. It will throw an exception as
         // long as the value is not null.
         parseJsonToken[AnyRef](parser, dataType)(PartialFunction.empty[JsonToken, AnyRef])

Review comment:
       should it be
   ```
   (parser: JsonParser) => parseJsonToken[java.lang.Long](parser, dataType) {
     case _ => null
   }
   ```




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org