[GitHub] [hive] miklosgergely commented on a change in pull request #1258: HIVE-23857: Fix HiveParser 'code too large' problem (Miklos Gergely, reviewed by David Mollitor)

[GitBox <gi...@apache.org>]

miklosgergely commented on a change in pull request #1258:
URL: https://github.com/apache/hive/pull/1258#discussion_r455352739



##########
File path: parser/bin/fixHiveParser.sh
##########
@@ -0,0 +1,44 @@
+#!/bin/bash
+
+# This is a temporary solution for the issue of the "code too large" problem related to HiveParser.java
+# We got to a point where adding anything to the antlr files lead to an issue about having a HiveParser.java that can not be compiled due to the compiled code size limitation in java (maximum 65536 bytes), so to avoid it we temorarly add this script to remove the huge tokenNames array into a separate file.
+# The real solution would be to switch to antlr 4
+
+tokenFile="target/generated-sources/antlr3/org/apache/hadoop/hive/ql/parse/HiveParserTokens.java"
+input="target/generated-sources/antlr3/org/apache/hadoop/hive/ql/parse/HiveParser.java"
+output="target/generated-sources/antlr3/org/apache/hadoop/hive/ql/parse/HiveParser.java-fixed"
+
+rm $tokenFile > /dev/null 2>&1
+rm $output > /dev/null 2>&1
+
+echo "package org.apache.hadoop.hive.ql.parse;" >> $tokenFile
+echo "" >> $tokenFile
+echo "public class HiveParserTokens {" >> $tokenFile
+
+state="STAY"
+while IFS= read -r line

Review comment:
       Thank you @t3rmin4t0r, I've modified the patch using awk.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org