[jira] [Commented] (SPARK-8045) Stack overflow in query parser when there is too many where

["Dongjoon Hyun (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-8045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15416417#comment-15416417 ] 

Dongjoon Hyun commented on SPARK-8045:
--------------------------------------

Hi, [~LabOctoCat].

Spark 2.0 now runs your long query with many WHERE correctly. I attached the test file from your Gist and the following is the result.

{code}
val longSQL = """
SELECT *
FROM VALUES (2015, 1, 1, 1) T(year, month, day, is_active)
WHERE (year = 2015 AND month = 1 AND day = 1 AND is_active = 1) OR
      (year = 2014 AND month = 12 AND day = 7 AND is_active = 1) OR
      (year = 2014 AND month = 11 AND day = 25 AND is_active = 1) OR
      (year = 2014 AND month = 8 AND day = 10 AND is_active = 1) OR
      (year = 2014 AND month = 7 AND day = 25 AND is_active = 1) OR
      (year = 2014 AND month = 6 AND day = 11 AND is_active = 1) OR
      (year = 2014 AND month = 5 AND day = 18 AND is_active = 1) OR
      (year = 2014 AND month = 11 AND day = 20 AND is_active = 1) OR
      (year = 2014 AND month = 8 AND day = 9 AND is_active = 1) OR
      (year = 2014 AND month = 11 AND day = 30 AND is_active = 1) OR
      (year = 2014 AND month...
"""
sql(longSQL).show

// Exiting paste mode, now interpreting.

+----+-----+---+---------+
|year|month|day|is_active|
+----+-----+---+---------+
|2015|    1|  1|        1|
+----+-----+---+---------+
{code}

> Stack overflow in query parser when there is too many where
> -----------------------------------------------------------
>
>                 Key: SPARK-8045
>                 URL: https://issues.apache.org/jira/browse/SPARK-8045
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>    Affects Versions: 1.3.0, 1.3.1
>            Reporter: Olivier Toupin
>            Priority: Minor
>         Attachments: longSql.txt
>
>
> A while ago we ran into a stack overflow in the parsing with one our query. It's still an issue (tested this morning) so we decided to report it as a bug.
> If run the query in attachment you get a stack overflow error probably because there to many where. Granted, this query isn't pretty, but still the parser shouldn't stack overflow.
> FYI, the where clause where used for partition pruning.
> Attachments: https://gist.github.com/anonymous/9aa852b7796b3918013c



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org