You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2020/10/13 13:20:10 UTC

[GitHub] [spark] MaxGekk opened a new pull request #30031: [SPARK-33134][SQL] Return partial results only for root JSON objects

MaxGekk opened a new pull request #30031:
URL: https://github.com/apache/spark/pull/30031


   ### What changes were proposed in this pull request?
   In the PR, I propose to restrict the partial result feature only by root JSON objects.
   
   ### Why are the changes needed?
   1. To not raise exception to users in the PERMISSIVE mode
   2. To have the same behaviour as Spark 2.4.x
   3. Current implementation of partial result is supposed to work only for root (top-level) JSON objects, and not tested for bad nested complex JSON fields.
   
   ### Does this PR introduce _any_ user-facing change?
   Yes
   
   ### How was this patch tested?
   Added a test to `JsonFunctionsSuite`.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] HyukjinKwon commented on pull request #30031: [SPARK-33134][SQL] Return partial results only for root JSON objects

Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #30031:
URL: https://github.com/apache/spark/pull/30031#issuecomment-708128011


   We probably need to redesign/refactoring JSON parsing logic here .. it's now pretty convoluted ..


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] HyukjinKwon closed pull request #30031: [SPARK-33134][SQL] Return partial results only for root JSON objects

Posted by GitBox <gi...@apache.org>.
HyukjinKwon closed pull request #30031:
URL: https://github.com/apache/spark/pull/30031


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] MaxGekk commented on pull request #30031: [SPARK-33134][SQL] Return partial results only for root JSON objects

Posted by GitBox <gi...@apache.org>.
MaxGekk commented on pull request #30031:
URL: https://github.com/apache/spark/pull/30031#issuecomment-707733214


   @HyukjinKwon Could you review this PR.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] HyukjinKwon commented on pull request #30031: [SPARK-33134][SQL] Return partial results only for root JSON objects

Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #30031:
URL: https://github.com/apache/spark/pull/30031#issuecomment-708128835


   Merged to master.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] MaxGekk commented on pull request #30031: [SPARK-33134][SQL] Return partial results only for root JSON objects

Posted by GitBox <gi...@apache.org>.
MaxGekk commented on pull request #30031:
URL: https://github.com/apache/spark/pull/30031#issuecomment-707874151


   The changes conflict with `branch-3.0`. Here is the backport to 3.0: https://github.com/apache/spark/pull/30032


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org