You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@lucene.apache.org by GitBox <gi...@apache.org> on 2020/01/28 12:21:29 UTC
[GitHub] [lucene-solr] ErickErickson opened a new pull request #1219:
LUCENE-9134: Javacc skeleton for Gradle regenerate
ErickErickson opened a new pull request #1219: LUCENE-9134: Javacc skeleton for Gradle regenerate
URL: https://github.com/apache/lucene-solr/pull/1219
Here's the build changes to get javacc to run, modeled on the jflex changes , many thanks for the model. Only two files changed here ;)
If the structure is OK, I'll fill in the "doLast" blocks with the cleanup code and maybe be able extract some common parts. NOTE: you can't even compile the result of running this because I wanted the changes to the build structure to be clear first so didn't include the cleanup tasks yet.
So if this structure is OK, should I merge it into master before or after the rest of the cleanup? My assumption is after. I want to try to get all the warnings etc. out of the generated code in the next phase to reduce the temptation for people to make hand-edits.
I didn't intentionally change the line endings in defaults-java, there's no other change there...
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene-solr] madrob commented on a change in pull request #1219:
LUCENE-9134: Javacc skeleton for Gradle regenerate
Posted by GitBox <gi...@apache.org>.
madrob commented on a change in pull request #1219: LUCENE-9134: Javacc skeleton for Gradle regenerate
URL: https://github.com/apache/lucene-solr/pull/1219#discussion_r372050229
##########
File path: build.gradle
##########
@@ -40,7 +40,8 @@ ext {
// https://github.com/palantir/gradle-consistent-versions/issues/383
scriptDepVersions = [
"apache-rat": "0.11",
- "jflex": "1.7.0"
+ "jflex": "1.7.0",
+ "javacc": "5.0"
Review comment:
nit: can we alphabetize this? or some other consistent ordering?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene-solr] ErickErickson commented on a change in pull request
#1219: LUCENE-9134: Javacc skeleton for Gradle regenerate
Posted by GitBox <gi...@apache.org>.
ErickErickson commented on a change in pull request #1219: LUCENE-9134: Javacc skeleton for Gradle regenerate
URL: https://github.com/apache/lucene-solr/pull/1219#discussion_r372713221
##########
File path: gradle/generation/javacc.gradle
##########
@@ -0,0 +1,102 @@
+// Add a top-level pseudo-task to which we will attach individual regenerate tasks.
+import static groovy.io.FileType.*
+
+configure(rootProject) {
+ configurations {
+ javacc
+ }
+
+ dependencies {
+ javacc "net.java.dev.javacc:javacc:${scriptDepVersions['javacc']}"
+ }
+
+ task javacc() {
+ description "Regenerate sources for corresponding javacc grammar files."
+ group "generation"
+
+ dependsOn ":lucene:queryparser:javaccParserClassic"
+ dependsOn ":lucene:queryparser:javaccParserSurround"
+ dependsOn ":lucene:queryparser:javaccParserFlexible"
+ }
+}
+
+// We always regenerate, no need to declare outputs.
+class JavaCCTask extends DefaultTask {
+ @Input
+ File javaccFile
+
+ JavaCCTask() {
+ dependsOn(project.rootProject.configurations.javacc)
+ }
+
+ @TaskAction
+ def generate() {
+ if (!javaccFile || !javaccFile.exists()) {
+ throw new RuntimeException("JavaCC input file does not exist: ${javaccFile}")
+ }
+ // Remove old files so we can regenerate them
+ def parentDir = javaccFile.parentFile
+ parentDir.eachFileMatch FILES, ~/.*\.java/, { file ->
+ if (file.text.contains("Generated By:JavaCC")) {
+ file.delete()
+ }
+ }
+ logger.lifecycle("Regenerating JavaCC:\n from: ${javaccFile}\n to: ${parentDir}")
+
+ project.javaexec {
+ classpath {
+ project.rootProject.configurations.javacc
+ }
+ main = "org.javacc.parser.Main"
+ args += "-OUTPUT_DIRECTORY=${parentDir}"
+ args += [javaccFile]
Review comment:
Done by Dawid.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene-solr] madrob commented on a change in pull request #1219:
LUCENE-9134: Javacc skeleton for Gradle regenerate
Posted by GitBox <gi...@apache.org>.
madrob commented on a change in pull request #1219: LUCENE-9134: Javacc skeleton for Gradle regenerate
URL: https://github.com/apache/lucene-solr/pull/1219#discussion_r372053503
##########
File path: gradle/generation/javacc.gradle
##########
@@ -0,0 +1,102 @@
+// Add a top-level pseudo-task to which we will attach individual regenerate tasks.
+import static groovy.io.FileType.*
+
+configure(rootProject) {
+ configurations {
+ javacc
+ }
+
+ dependencies {
+ javacc "net.java.dev.javacc:javacc:${scriptDepVersions['javacc']}"
+ }
+
+ task javacc() {
+ description "Regenerate sources for corresponding javacc grammar files."
+ group "generation"
+
+ dependsOn ":lucene:queryparser:javaccParserClassic"
+ dependsOn ":lucene:queryparser:javaccParserSurround"
+ dependsOn ":lucene:queryparser:javaccParserFlexible"
+ }
+}
+
+// We always regenerate, no need to declare outputs.
+class JavaCCTask extends DefaultTask {
+ @Input
+ File javaccFile
+
+ JavaCCTask() {
+ dependsOn(project.rootProject.configurations.javacc)
+ }
+
+ @TaskAction
+ def generate() {
+ if (!javaccFile || !javaccFile.exists()) {
+ throw new RuntimeException("JavaCC input file does not exist: ${javaccFile}")
+ }
+ // Remove old files so we can regenerate them
+ def parentDir = javaccFile.parentFile
+ parentDir.eachFileMatch FILES, ~/.*\.java/, { file ->
+ if (file.text.contains("Generated By:JavaCC")) {
+ file.delete()
+ }
+ }
+ logger.lifecycle("Regenerating JavaCC:\n from: ${javaccFile}\n to: ${parentDir}")
+
+ project.javaexec {
+ classpath {
+ project.rootProject.configurations.javacc
+ }
+ main = "org.javacc.parser.Main"
+ args += "-OUTPUT_DIRECTORY=${parentDir}"
+ args += [javaccFile]
+ }
+ }
+}
+
+
+configure(project(":lucene:queryparser")) {
+ task javaccParserClassic(type: JavaCCTask) {
+ description "Regenerate classic query parser from java CC.java"
Review comment:
I think this is missing the specific description?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene-solr] madrob commented on a change in pull request #1219:
LUCENE-9134: Javacc skeleton for Gradle regenerate
Posted by GitBox <gi...@apache.org>.
madrob commented on a change in pull request #1219: LUCENE-9134: Javacc skeleton for Gradle regenerate
URL: https://github.com/apache/lucene-solr/pull/1219#discussion_r372052356
##########
File path: gradle/generation/javacc.gradle
##########
@@ -0,0 +1,102 @@
+// Add a top-level pseudo-task to which we will attach individual regenerate tasks.
+import static groovy.io.FileType.*
+
+configure(rootProject) {
+ configurations {
+ javacc
+ }
+
+ dependencies {
+ javacc "net.java.dev.javacc:javacc:${scriptDepVersions['javacc']}"
+ }
+
+ task javacc() {
+ description "Regenerate sources for corresponding javacc grammar files."
+ group "generation"
+
+ dependsOn ":lucene:queryparser:javaccParserClassic"
+ dependsOn ":lucene:queryparser:javaccParserSurround"
+ dependsOn ":lucene:queryparser:javaccParserFlexible"
+ }
+}
+
+// We always regenerate, no need to declare outputs.
+class JavaCCTask extends DefaultTask {
+ @Input
+ File javaccFile
+
+ JavaCCTask() {
+ dependsOn(project.rootProject.configurations.javacc)
+ }
+
+ @TaskAction
+ def generate() {
+ if (!javaccFile || !javaccFile.exists()) {
+ throw new RuntimeException("JavaCC input file does not exist: ${javaccFile}")
+ }
+ // Remove old files so we can regenerate them
+ def parentDir = javaccFile.parentFile
+ parentDir.eachFileMatch FILES, ~/.*\.java/, { file ->
+ if (file.text.contains("Generated By:JavaCC")) {
+ file.delete()
+ }
+ }
+ logger.lifecycle("Regenerating JavaCC:\n from: ${javaccFile}\n to: ${parentDir}")
+
+ project.javaexec {
+ classpath {
+ project.rootProject.configurations.javacc
+ }
+ main = "org.javacc.parser.Main"
+ args += "-OUTPUT_DIRECTORY=${parentDir}"
+ args += [javaccFile]
Review comment:
is this right to pass an array?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene-solr] ErickErickson closed pull request #1219: LUCENE-9134:
Javacc skeleton for Gradle regenerate
Posted by GitBox <gi...@apache.org>.
ErickErickson closed pull request #1219: LUCENE-9134: Javacc skeleton for Gradle regenerate
URL: https://github.com/apache/lucene-solr/pull/1219
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene-solr] dweiss commented on issue #1219: LUCENE-9134: Javacc
skeleton for Gradle regenerate
Posted by GitBox <gi...@apache.org>.
dweiss commented on issue #1219: LUCENE-9134: Javacc skeleton for Gradle regenerate
URL: https://github.com/apache/lucene-solr/pull/1219#issuecomment-579467903
I thought so at first... but then I realized it will be a nightmare to regenerate some of these -- like the jflex one that requires 10 gigs and 20 minutes on my machine (!).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene-solr] ErickErickson commented on a change in pull request
#1219: LUCENE-9134: Javacc skeleton for Gradle regenerate
Posted by GitBox <gi...@apache.org>.
ErickErickson commented on a change in pull request #1219: LUCENE-9134: Javacc skeleton for Gradle regenerate
URL: https://github.com/apache/lucene-solr/pull/1219#discussion_r372713105
##########
File path: gradle/generation/javacc.gradle
##########
@@ -0,0 +1,102 @@
+// Add a top-level pseudo-task to which we will attach individual regenerate tasks.
+import static groovy.io.FileType.*
+
+configure(rootProject) {
+ configurations {
+ javacc
+ }
+
+ dependencies {
+ javacc "net.java.dev.javacc:javacc:${scriptDepVersions['javacc']}"
+ }
+
+ task javacc() {
+ description "Regenerate sources for corresponding javacc grammar files."
+ group "generation"
+
+ dependsOn ":lucene:queryparser:javaccParserClassic"
+ dependsOn ":lucene:queryparser:javaccParserSurround"
+ dependsOn ":lucene:queryparser:javaccParserFlexible"
+ }
+}
+
+// We always regenerate, no need to declare outputs.
+class JavaCCTask extends DefaultTask {
+ @Input
+ File javaccFile
+
+ JavaCCTask() {
+ dependsOn(project.rootProject.configurations.javacc)
+ }
+
+ @TaskAction
+ def generate() {
+ if (!javaccFile || !javaccFile.exists()) {
+ throw new RuntimeException("JavaCC input file does not exist: ${javaccFile}")
+ }
+ // Remove old files so we can regenerate them
+ def parentDir = javaccFile.parentFile
+ parentDir.eachFileMatch FILES, ~/.*\.java/, { file ->
+ if (file.text.contains("Generated By:JavaCC")) {
+ file.delete()
+ }
+ }
+ logger.lifecycle("Regenerating JavaCC:\n from: ${javaccFile}\n to: ${parentDir}")
+
+ project.javaexec {
+ classpath {
+ project.rootProject.configurations.javacc
+ }
+ main = "org.javacc.parser.Main"
+ args += "-OUTPUT_DIRECTORY=${parentDir}"
+ args += [javaccFile]
+ }
+ }
+}
+
+
+configure(project(":lucene:queryparser")) {
+ task javaccParserClassic(type: JavaCCTask) {
+ description "Regenerate classic query parser from java CC.java"
Review comment:
Copy/paste, I'll fix if Dawid didn't beat me to it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene-solr] ErickErickson commented on a change in pull request
#1219: LUCENE-9134: Javacc skeleton for Gradle regenerate
Posted by GitBox <gi...@apache.org>.
ErickErickson commented on a change in pull request #1219: LUCENE-9134: Javacc skeleton for Gradle regenerate
URL: https://github.com/apache/lucene-solr/pull/1219#discussion_r372712928
##########
File path: build.gradle
##########
@@ -40,7 +40,8 @@ ext {
// https://github.com/palantir/gradle-consistent-versions/issues/383
scriptDepVersions = [
"apache-rat": "0.11",
- "jflex": "1.7.0"
+ "jflex": "1.7.0",
+ "javacc": "5.0"
Review comment:
OK, I'm not quite sure why I can't reply to your comment about the misleading message so I'll combine them.
Yeah, **File "TokenMgrError.java" does not exist. Will create one.** is because the build file deletes the outputs first. If they _aren't_ deleted, you get output about "file is obsolete, please delete or rename it". So I have no faith that javacc really overwrites them.
Besides, if for some mysterious reason the generated files ever change, this'll delete the no-longer-generated ones. Unlikely I admit...
Dawid alphabetized this already.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene-solr] ErickErickson commented on issue #1219: LUCENE-9134:
Javacc skeleton for Gradle regenerate
Posted by GitBox <gi...@apache.org>.
ErickErickson commented on issue #1219: LUCENE-9134: Javacc skeleton for Gradle regenerate
URL: https://github.com/apache/lucene-solr/pull/1219#issuecomment-580036377
OK, on to the next phase so closing this one.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene-solr] dweiss commented on issue #1219: LUCENE-9134: Javacc
skeleton for Gradle regenerate
Posted by GitBox <gi...@apache.org>.
dweiss commented on issue #1219: LUCENE-9134: Javacc skeleton for Gradle regenerate
URL: https://github.com/apache/lucene-solr/pull/1219#issuecomment-579468672
bq. Which is somewhat misleading? Does the task not work if we don't delete files first?
javacc tries to be smart and analyzes those files looking for any local changes. Since we do introduce such changes it wouldn't touch those files, hence the delete.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene-solr] dweiss commented on a change in pull request #1219:
LUCENE-9134: Javacc skeleton for Gradle regenerate
Posted by GitBox <gi...@apache.org>.
dweiss commented on a change in pull request #1219: LUCENE-9134: Javacc skeleton for Gradle regenerate
URL: https://github.com/apache/lucene-solr/pull/1219#discussion_r372071747
##########
File path: gradle/generation/javacc.gradle
##########
@@ -0,0 +1,102 @@
+// Add a top-level pseudo-task to which we will attach individual regenerate tasks.
+import static groovy.io.FileType.*
+
+configure(rootProject) {
+ configurations {
+ javacc
+ }
+
+ dependencies {
+ javacc "net.java.dev.javacc:javacc:${scriptDepVersions['javacc']}"
+ }
+
+ task javacc() {
+ description "Regenerate sources for corresponding javacc grammar files."
+ group "generation"
+
+ dependsOn ":lucene:queryparser:javaccParserClassic"
+ dependsOn ":lucene:queryparser:javaccParserSurround"
+ dependsOn ":lucene:queryparser:javaccParserFlexible"
+ }
+}
+
+// We always regenerate, no need to declare outputs.
+class JavaCCTask extends DefaultTask {
+ @Input
+ File javaccFile
+
+ JavaCCTask() {
+ dependsOn(project.rootProject.configurations.javacc)
+ }
+
+ @TaskAction
+ def generate() {
+ if (!javaccFile || !javaccFile.exists()) {
+ throw new RuntimeException("JavaCC input file does not exist: ${javaccFile}")
+ }
+ // Remove old files so we can regenerate them
+ def parentDir = javaccFile.parentFile
+ parentDir.eachFileMatch FILES, ~/.*\.java/, { file ->
+ if (file.text.contains("Generated By:JavaCC")) {
+ file.delete()
+ }
+ }
+ logger.lifecycle("Regenerating JavaCC:\n from: ${javaccFile}\n to: ${parentDir}")
+
+ project.javaexec {
+ classpath {
+ project.rootProject.configurations.javacc
+ }
+ main = "org.javacc.parser.Main"
+ args += "-OUTPUT_DIRECTORY=${parentDir}"
+ args += [javaccFile]
Review comment:
it is correct but unnecessary
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene-solr] madrob commented on issue #1219: LUCENE-9134: Javacc
skeleton for Gradle regenerate
Posted by GitBox <gi...@apache.org>.
madrob commented on issue #1219: LUCENE-9134: Javacc skeleton for Gradle regenerate
URL: https://github.com/apache/lucene-solr/pull/1219#issuecomment-579461009
Do we want to have a top level regenerate task that will regenerate everything? Fine to push that out to a later issue also.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org