[jira] [Created] (IMPALA-9153) Avoid unneccessary cup compilation

["Quanlong Huang (Jira)" <ji...@apache.org>]

Quanlong Huang created IMPALA-9153:
--------------------------------------

             Summary: Avoid unneccessary cup compilation
                 Key: IMPALA-9153
                 URL: https://issues.apache.org/jira/browse/IMPALA-9153
             Project: IMPALA
          Issue Type: Test
            Reporter: Quanlong Huang


Everytime when I run a single FE test, maven always recompile the whole module since it detects changes. The change is made by cup-maven-plugin. It recompiles sql-parser.cup to SqlParser.java and SqlParserSymbols.java everytime even there're no changes in sql-parser.cup. The recompilation cost some time and slow down the reproducing of some flasky test that requires running it many times.

An output example:
{code:java}
$ (pushd fe && mvn test -fae -Dtest=AuthorizationStmtTest#test*Enabled[1])
~/workspace/Impala/fe ~/workspace/Impala
[INFO] Scanning for projects...
[INFO] 
[INFO] -----------------< org.apache.impala:impala-frontend >------------------
[INFO] Building Apache Impala Query Engine Frontend 0.1-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[WARNING] Could not transfer metadata com.cloudera.cdh:cdh-root:6.x-SNAPSHOT/maven-metadata.xml from/to ${distMgmtSnapshotsId} (${distMgmtSnapshotsUrl}): Cannot access ${distMgmtSnapshotsUrl} with type default using the available connector factories: BasicRepositoryConnectorFactory
[INFO] 
[INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (enforce-banned-dependencies) @ impala-frontend ---
[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.6.201602180812:prepare-agent (prepare-jacoco-agent) @ impala-frontend ---
[INFO] Skipping JaCoCo execution because property jacoco.skip is set.
[INFO] surefireJacocoArg set to empty
[INFO] 
[INFO] --- cup-maven-plugin:1.6-cdh:generate (cup) @ impala-frontend ---
[INFO] CUP: Processing 1 cup files   <------------ Always regenerate the files
Warning : Terminal "EMPTY_IDENT" was declared but never used
Warning : Terminal "UNEXPECTED_CHAR" was declared but never used
------- CUP v0.11a czt01 beta Parser Generation Summary -------
  0 errors and 2 warnings
  229 terminals, 228 non-terminals, and 875 productions declared, 
  producing 1620 unique parse states.
  2 terminals declared but not used.
  0 non-terminals declared but not used.
  0 productions never reduced.
  0 conflicts detected (0 expected).
  Code written to "SqlParser.java", and "SqlParserSymbols.java".
---------------------------------------------------- (v0.11a czt01 beta)
[WARNING] /home/quanlong/workspace/Impala/fe/src/main/cup/sql-parser.cup [0:0]: Terminal "EMPTY_IDENT" was declared but never used
[WARNING] /home/quanlong/workspace/Impala/fe/src/main/cup/sql-parser.cup [0:0]: Terminal "UNEXPECTED_CHAR" was declared but never used
[INFO] CUP: generated /home/quanlong/workspace/Impala/fe/target/generated-sources/cup/org/apache/impala/analysis/SqlParser.java
[INFO] CUP: generated /home/quanlong/workspace/Impala/fe/target/generated-sources/cup/org/apache/impala/analysis/SqlParserSymbols.java
[INFO] CUP: generated /home/quanlong/workspace/Impala/fe/target/generated-sources/cup/org/apache/impala/analysis/action_table.dat
[INFO] 
[INFO] --- maven-jflex-plugin:1.4.3:generate (jflex) @ impala-frontend ---
[INFO]   SqlScanner.java is up to date.
[INFO] 
[INFO] --- maven-dependency-plugin:3.1.1:build-classpath (write-classpath) @ impala-frontend ---
[INFO] Skipped writing classpath file '/home/quanlong/workspace/Impala/fe/target/build-classpath.txt'.  No changes found.
[INFO] 
[INFO] --- build-helper-maven-plugin:1.5:add-source (add-source) @ impala-frontend ---
[INFO] Source directory: /home/quanlong/workspace/Impala/fe/generated-sources/gen-java added.
[INFO] Source directory: /home/quanlong/workspace/Impala/fe/target/generated-sources/cup added.
[INFO] Source directory: /home/quanlong/workspace/Impala/fe/src/compat-hive-2/java added.
[INFO] 
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ impala-frontend ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /home/quanlong/workspace/Impala/fe/src/main/resources
[INFO] Copying 1 resource
[INFO] 
[INFO] --- maven-compiler-plugin:3.3:compile (default-compile) @ impala-frontend ---
[INFO] Changes detected - recompiling the module! <--------- Decide to recomple
[INFO] Compiling 907 source files to /home/quanlong/workspace/Impala/fe/target/classes
[WARNING] /home/quanlong/workspace/Impala/fe/src/main/java/org/apache/impala/util/UnsafeUtil.java:[24,16] sun.misc.Unsafe is internal proprietary API and may be removed in a future release
[WARNING] /home/quanlong/workspace/Impala/fe/src/main/java/org/apache/impala/util/UnsafeUtil.java:[24,16] sun.misc.Unsafe is internal proprietary API and may be removed in a future release
[WARNING] /home/quanlong/workspace/Impala/fe/src/main/java/org/apache/impala/util/UnsafeUtil.java:[24,16] sun.misc.Unsafe is internal proprietary API and may be removed in a future release
[WARNING] /home/quanlong/workspace/Impala/fe/src/main/java/org/apache/impala/util/UnsafeUtil.java:[33,23] sun.misc.Unsafe is internal proprietary API and may be removed in a future release
[WARNING] /home/quanlong/workspace/Impala/fe/src/main/java/org/apache/impala/util/UnsafeUtil.java:[40,15] sun.misc.Unsafe is internal proprietary API and may be removed in a future release
[WARNING] /home/quanlong/workspace/Impala/fe/src/main/java/org/apache/impala/util/UnsafeUtil.java:[45,25] sun.misc.Unsafe is internal proprietary API and may be removed in a future release
[INFO] /home/quanlong/workspace/Impala/fe/src/main/java/org/apache/impala/hive/executor/UdfExecutor.java: Some input files use or override a deprecated API.
[INFO] /home/quanlong/workspace/Impala/fe/src/main/java/org/apache/impala/hive/executor/UdfExecutor.java: Recompile with -Xlint:deprecation for details.
[INFO] /home/quanlong/workspace/Impala/fe/src/main/java/org/apache/impala/analysis/Expr.java: Some input files use unchecked or unsafe operations.
[INFO] /home/quanlong/workspace/Impala/fe/src/main/java/org/apache/impala/analysis/Expr.java: Recompile with -Xlint:unchecked for details.
[INFO] 
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ impala-frontend ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 43 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.3:testCompile (default-testCompile) @ impala-frontend ---
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] --- maven-surefire-plugin:2.20:test (default-test) @ impala-frontend ---
[INFO] 
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.apache.impala.authorization.AuthorizationStmtTest
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 69.847 s - in org.apache.impala.authorization.AuthorizationStmtTest
[INFO] 
[INFO] Results:
[INFO] 
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0
[INFO] 
[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.6.201602180812:report (jacoco-report) @ impala-frontend ---
[INFO] Skipping JaCoCo execution because property jacoco.skip is set.
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  01:23 min
[INFO] Finished at: 2019-11-13T15:50:33+08:00
[INFO] ------------------------------------------------------------------------ {code}
We should configure cup-maven-plugin to avoid unneccessary recompilation.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)