You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Julian Hyde <jh...@apache.org> on 2019/04/01 20:22:12 UTC

HIVE-18624 SQL parser performance bug

HIVE-18624 [1] is a serious performance bug in the SQL parser. It causes parse times that are literally exponential in the number of parentheses in the expression, thus parsing of a query that has complex expressions may take minutes or not terminate. According to JIRA, the bug was fixed on 2.4.0, 3.1.0, 4.0.0 code lines in August but has not yet been released.

I work for Looker, a BI tool that generates SQL with deeply nested expressions, and therefore they hit this bug. Hive 2.2, 2.3 and 3.0 are unusable for our customers due to this bug.

I do not know the schedule for 2.4.0, 3.1.0 or 4.0.0 releases, but if they are a way off, would it be possible to fix this bug in a patch release?

Julian

[1] https://issues.apache.org/jira/browse/HIVE-18624 <https://issues.apache.org/jira/browse/HIVE-18624>

Re: HIVE-18624 SQL parser performance bug

Posted by Zoltan Haindrich <ki...@rxd.hu>.
Hello Julian!

Around the time I've submitted that patch I've only pushed it to branch-2 and branch-3.
I wasn't thinking about also putting it on branch-2.3 and branch-3.1.. if I would have been; it would been already out...
I've just pushed it to branch-2.3 and branch-3.1 so if there will be a patch release - it will contain this fix.

cheers,
Zoltan

On 4/1/19 10:22 PM, Julian Hyde wrote:
> HIVE-18624 [1] is a serious performance bug in the SQL parser. It causes parse times that are literally exponential in the number of parentheses in the expression, thus parsing of a query that has complex expressions may take minutes or not terminate. According to JIRA, the bug was fixed on 2.4.0, 3.1.0, 4.0.0 code lines in August but has not yet been released.
> 
> I work for Looker, a BI tool that generates SQL with deeply nested expressions, and therefore they hit this bug. Hive 2.2, 2.3 and 3.0 are unusable for our customers due to this bug.
> 
> I do not know the schedule for 2.4.0, 3.1.0 or 4.0.0 releases, but if they are a way off, would it be possible to fix this bug in a patch release?
> 
> Julian
> 
> [1] https://issues.apache.org/jira/browse/HIVE-18624 <https://issues.apache.org/jira/browse/HIVE-18624>
> 
>