You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@lucene.apache.org by GitBox <gi...@apache.org> on 2020/01/22 17:03:54 UTC
[GitHub] [lucene-solr] ErickErickson opened a new pull request #1199:
LUCENE-9134: Port ant-regenerate tasks to Gradle build
ErickErickson opened a new pull request #1199: LUCENE-9134: Port ant-regenerate tasks to Gradle build
URL: https://github.com/apache/lucene-solr/pull/1199
I need to put this down for a while, so please feel free to modify/commit/whatever.
Current state:
I think this covers everything except the analysis part of the regenerate task that Dawid is working on.
I left in the ant javacc task because I ran out of patience. The plugin I thought was going to work would keep executing even for the "assemble" task. so I went back to using the ant javacc task.
You can skim pretty much everything except the gradle.build and createLevAutomatoa.py files. The rest are are the results of getting the lint warnings out of the generated code so people don't go back in and hand-edit them. There's a bit of superfluous whitespace that's been changed, sorry about that. StandardSyntaxParser.java has some methods reordered, why I don't know. But they're identical so I'm not going to worry about them.
If I missed any warnings and the like we can address in another go-round.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene-solr] ErickErickson commented on a change in pull request
#1199: LUCENE-9134: Port ant-regenerate tasks to Gradle build
Posted by GitBox <gi...@apache.org>.
ErickErickson commented on a change in pull request #1199: LUCENE-9134: Port ant-regenerate tasks to Gradle build
URL: https://github.com/apache/lucene-solr/pull/1199#discussion_r370968095
##########
File path: lucene/core/build.gradle
##########
@@ -1,7 +1,117 @@
-
apply plugin: 'java-library'
+apply plugin: "de.undercouch.download"
dependencies {
testImplementation project(':lucene:codecs')
testImplementation project(':lucene:test-framework')
}
+
+def momanCommitHash = "5c5c2a1e4dea"
+def momanUrl = "https://bitbucket.org/jpbarrette/moman/get/${momanCommitHash}.zip"
+def python_exe = "python"
+
+def jflexUrl = "https://jflex.de/release/jflex-1.7.0.zip"
+
+// Download and extract Moman
+task installJFlex(type: Download) {
+ src jflexUrl
+ dest file("${buildDir}/jflex.zip")
+ onlyIfModified true
+ doLast {
+ ant.unzip(src: file("${buildDir}/jflex.zip"), dest: file("${buildDir}/jflex"), overwrite: "true") {
+ ant.cutdirsmapper(dirs: "1")
+ }
+ }
+}
+
+task installMoman(type: Download) {
+ src momanUrl
+ dest file("${buildDir}/moman.zip")
+ onlyIfModified true
+
+ doLast {
+ ant.unzip(src: file("${buildDir}/moman.zip"), dest: file("${buildDir}/moman"), overwrite: "true") {
+ ant.cutdirsmapper(dirs: "1")
+ }
+ }
+}
+
+task createPackedIntSources {
+ doLast {
+ exec {
+ workingDir 'src/java/org/apache/lucene/util/packed'
+ executable "${python_exe}"
+ args = ['-B', 'gen_BulkOperation.py']
+ }
+ exec {
+ workingDir 'src/java/org/apache/lucene/util/packed'
+ executable "${python_exe}"
+ args = ['-B', 'gen_Packed64SingleBlock.py']
+ }
+ ant.fixcrlf(srcdir: 'src/java/org/apache/lucene/util/packed', includes: 'Packed64SingleBlock.java', encoding: 'UTF-8')
+ ant.fixcrlf(srcdir: 'src/java/org/apache/lucene/util/packed', includes: 'BulkOperation*.java', encoding: 'UTF-8')
+ }
+}
+
+task createLevAutomaton(dependsOn: installMoman) {
+ doLast {
+ exec {
Review comment:
Slick. I can do something similar in createPackedIntSources too.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene-solr] madrob commented on a change in pull request #1199:
LUCENE-9134: Port ant-regenerate tasks to Gradle build
Posted by GitBox <gi...@apache.org>.
madrob commented on a change in pull request #1199: LUCENE-9134: Port ant-regenerate tasks to Gradle build
URL: https://github.com/apache/lucene-solr/pull/1199#discussion_r370860485
##########
File path: lucene/queryparser/src/java/org/apache/lucene/queryparser/classic/QueryParser.java
##########
@@ -879,7 +878,7 @@ private void jj_add_error_token(int kind, int pos) {
for (int i = 0; i < jj_endpos; i++) {
jj_expentry[i] = jj_lasttokens[i];
}
- jj_entries_loop: for (java.util.Iterator it = jj_expentries.iterator(); it.hasNext();) {
Review comment:
This needs to be a wildcard generic and not `int[]`?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene-solr] ErickErickson commented on a change in pull request
#1199: LUCENE-9134: Port ant-regenerate tasks to Gradle build
Posted by GitBox <gi...@apache.org>.
ErickErickson commented on a change in pull request #1199: LUCENE-9134: Port ant-regenerate tasks to Gradle build
URL: https://github.com/apache/lucene-solr/pull/1199#discussion_r370969094
##########
File path: lucene/queryparser/build.gradle
##########
@@ -7,3 +7,305 @@ dependencies {
testImplementation project(':lucene:test-framework')
}
+
+configure(":lucene:queryparser") {
+ configurations {
+ javaCCDeps
+
+ dependencies {
+ javaCCDeps "net.java.dev.javacc:javacc:5.0"
+ }
+ }
+}
+def conf = [
+ "queryParser": ["inputFile": file('src/java/org/apache/lucene/queryparser/classic/QueryParser.jj'), "outputDir": file('src/java/org/apache/lucene/queryparser/classic')],
+ "surround" : ["inputFile": file('src/java/org/apache/lucene/queryparser/surround/parser/QueryParser.jj'), "outputDir": file('src/java/org/apache/lucene/queryparser/surround/parser/')],
+ "flexible" : ["inputFile": file('src/java/org/apache/lucene/queryparser/flexible/standard/parser/StandardSyntaxParser.jj'), "outputDir": file('src/java/org/apache/lucene/queryparser/flexible/standard/parser')]
+]
+
+String javaCCClasspath
+String javaCCHome
+
+task installJavacc {
+ doLast {
+ javaCCClasspath = project.project(":lucene:queryparser").configurations.javaCCDeps.asPath
+ javaCCHome = javaCCClasspath.substring(0, javaCCClasspath.lastIndexOf("/"))
+
+ String hacky = javaCCHome + "/" + "javacc.jar"
+ File hackyFile = new File(hacky)
+ println hackyFile
+ println hackyFile.exists()
+ if (hackyFile.exists() == false) {
+ hackyFile.bytes = new File(javaCCClasspath).bytes
+ hackyFile.setExecutable(true)
+ }
+ }
+}
+String lineSeparator = System.lineSeparator()
+
+task cleanOldGeneratedFiles {
+ doLast {
+ println "Removing old generated files"
+ conf.each { key, val ->
+ val["outputDir"].eachFileMatch ~/.*\.java/, { f ->
+ if (f.text =~ /Generated.*By.*JavaCC/) {
+ f.delete()
+ }
+ }
+ }
+ }
+}
+
+task runJavaccQueryParser(dependsOn: installJavacc) {
+ doLast {
+ println "Generating QueryParser parser"
+
+ def inputFile = conf["queryParser"]["inputFile"].getAbsolutePath()
+ def outputDir = conf["queryParser"]["outputDir"].getAbsolutePath()
+ ant.javacc(target: inputFile, outputDirectory: outputDir, javacchome: javaCCHome)
+
+
+ ant.replaceregexp(file: new File(outputDir, "QueryParser.java"),
+ byline: "true",
+ match: "public QueryParser\\(CharStream ",
+ replace: "protected QueryParser(CharStream ")
+ ant.replaceregexp(file: new File(outputDir, "QueryParser.java"),
+ byline: "true",
+ match: "public QueryParser\\(QueryParserTokenManager ",
+ replace: "protected QueryParser(QueryParserTokenManager ")
+ }
+}
+
+task runJavaccSurround(dependsOn: installJavacc) {
+ doLast {
+ println "Generating Surroundf parser"
+ def inputFile = conf["surround"]["inputFile"].getAbsolutePath()
+ def outputDir = conf["surround"]["outputDir"].getAbsolutePath()
+
+ ant.javacc(target: inputFile, outputDirectory: outputDir, javacchome: javaCCHome)
+
+ // This is very clumsy, but these are unused imports that are necessary other generated files
+ [
+ "import java.util.Vector;",
+ "import org.apache.lucene.queryparser.flexible.messages.Message;",
+ "import org.apache.lucene.queryparser.flexible.messages.MessageImpl;",
+ "import org.apache.lucene.queryparser.flexible.core.QueryNodeParseException;",
+ "import org.apache.lucene.queryparser.flexible.core.messages.QueryParserMessages;",
+ "import org.apache.lucene.queryparser.flexible.core.nodes.AndQueryNode;",
+ "import org.apache.lucene.queryparser.flexible.core.nodes.BooleanQueryNode;",
+ "import org.apache.lucene.queryparser.flexible.core.nodes.BoostQueryNode;",
+ "import org.apache.lucene.queryparser.flexible.core.nodes.FieldQueryNode;",
+ "import org.apache.lucene.queryparser.flexible.core.nodes.FuzzyQueryNode;",
+ "import org.apache.lucene.queryparser.flexible.core.nodes.ModifierQueryNode;",
+ "import org.apache.lucene.queryparser.flexible.core.nodes.GroupQueryNode;",
+ "import org.apache.lucene.queryparser.flexible.core.nodes.OrQueryNode;",
+ "import org.apache.lucene.queryparser.flexible.standard.nodes.RegexpQueryNode;",
+ "import org.apache.lucene.queryparser.flexible.core.nodes.SlopQueryNode;",
+ "import org.apache.lucene.queryparser.flexible.core.nodes.QueryNode;",
+ "import org.apache.lucene.queryparser.flexible.core.nodes.QuotedFieldQueryNode;",
+ "import org.apache.lucene.queryparser.flexible.core.parser.SyntaxParser;",
+ "import org.apache.lucene.queryparser.flexible.standard.nodes.TermRangeQueryNode;",
+ "import java.io.StringReader;",
+ "import java.util.ArrayList;",
+ "import java.util.Arrays;",
+ "import java.util.HashSet;",
+ "import java.util.List;",
+ "import java.util.Locale;",
+ "import java.util.Set;",
+ "import org.apache.lucene.analysis.Analyzer;",
+ "import org.apache.lucene.document.DateTools;",
+ "import org.apache.lucene.search.BooleanClause;",
+ "import org.apache.lucene.search.Query;",
+ "import org.apache.lucene.search.TermRangeQuery;",
+ "import org.apache.lucene.analysis.TokenStream;",
+ "import org.apache.lucene.queryparser.surround.query.SrndQuery;",
+ "import org.apache.lucene.queryparser.surround.query.FieldsQuery;",
+ "import org.apache.lucene.queryparser.surround.query.OrQuery;",
+ "import org.apache.lucene.queryparser.surround.query.AndQuery;",
+ "import org.apache.lucene.queryparser.surround.query.NotQuery;",
+ "import org.apache.lucene.queryparser.surround.query.DistanceQuery;",
+ "import org.apache.lucene.queryparser.surround.query.SrndTermQuery;",
+ "import org.apache.lucene.queryparser.surround.query.SrndPrefixQuery;",
+ "import org.apache.lucene.queryparser.surround.query.SrndTruncQuery;"
+ ].each {
+ ant.replaceregexp(match: "${it}.*${lineSeparator}", replace: "", encoding: "UTF-8") {
+ ant.fileset(dir: outputDir, includes: "*TokenManager.java")
+ }
+ }
+ // StringBuffer -> StringBuilder
+ ant.replace(token: "StringBuffer", value: "StringBuilder", encoding: "UTF-8") {
+ ant.fileset(dir: outputDir, includes: "ParseException.java TokenMgrError.java")
+ }
+
+ [
+ "import org.apache.lucene.analysis.TokenStream;",
+ ].each {
+ ant.replaceregexp(match: "${it}.*${lineSeparator}", replace: "", encoding: "UTF-8") {
+ ant.fileset(dir: outputDir, includes: "QueryParser.java")
+ }
+ }
+ // StringBuffer -> StringBuilder
Review comment:
Not any more ;)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene-solr] ErickErickson commented on issue #1199: LUCENE-9134:
Port ant-regenerate tasks to Gradle build
Posted by GitBox <gi...@apache.org>.
ErickErickson commented on issue #1199: LUCENE-9134: Port ant-regenerate tasks to Gradle build
URL: https://github.com/apache/lucene-solr/pull/1199#issuecomment-578548576
Starting over with a new model
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene-solr] ErickErickson commented on a change in pull request
#1199: LUCENE-9134: Port ant-regenerate tasks to Gradle build
Posted by GitBox <gi...@apache.org>.
ErickErickson commented on a change in pull request #1199: LUCENE-9134: Port ant-regenerate tasks to Gradle build
URL: https://github.com/apache/lucene-solr/pull/1199#discussion_r370968598
##########
File path: lucene/queryparser/src/java/org/apache/lucene/queryparser/classic/QueryParser.java
##########
@@ -879,7 +878,7 @@ private void jj_add_error_token(int kind, int pos) {
for (int i = 0; i < jj_endpos; i++) {
jj_expentry[i] = jj_lasttokens[i];
}
- jj_entries_loop: for (java.util.Iterator it = jj_expentries.iterator(); it.hasNext();) {
Review comment:
The line you attached this comment to _does_ have a wildcard generic (line 881) in the changed code. Or did you mean line 866?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene-solr] ErickErickson commented on a change in pull request
#1199: LUCENE-9134: Port ant-regenerate tasks to Gradle build
Posted by GitBox <gi...@apache.org>.
ErickErickson commented on a change in pull request #1199: LUCENE-9134: Port ant-regenerate tasks to Gradle build
URL: https://github.com/apache/lucene-solr/pull/1199#discussion_r370969039
##########
File path: lucene/core/build.gradle
##########
@@ -1,7 +1,117 @@
-
apply plugin: 'java-library'
+apply plugin: "de.undercouch.download"
dependencies {
testImplementation project(':lucene:codecs')
testImplementation project(':lucene:test-framework')
}
+
+def momanCommitHash = "5c5c2a1e4dea"
+def momanUrl = "https://bitbucket.org/jpbarrette/moman/get/${momanCommitHash}.zip"
+def python_exe = "python"
+
+def jflexUrl = "https://jflex.de/release/jflex-1.7.0.zip"
+
+// Download and extract Moman
+task installJFlex(type: Download) {
+ src jflexUrl
+ dest file("${buildDir}/jflex.zip")
+ onlyIfModified true
+ doLast {
+ ant.unzip(src: file("${buildDir}/jflex.zip"), dest: file("${buildDir}/jflex"), overwrite: "true") {
+ ant.cutdirsmapper(dirs: "1")
+ }
+ }
+}
+
+task installMoman(type: Download) {
Review comment:
Done.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene-solr] madrob commented on a change in pull request #1199:
LUCENE-9134: Port ant-regenerate tasks to Gradle build
Posted by GitBox <gi...@apache.org>.
madrob commented on a change in pull request #1199: LUCENE-9134: Port ant-regenerate tasks to Gradle build
URL: https://github.com/apache/lucene-solr/pull/1199#discussion_r370858176
##########
File path: lucene/core/build.gradle
##########
@@ -1,7 +1,117 @@
-
apply plugin: 'java-library'
+apply plugin: "de.undercouch.download"
dependencies {
testImplementation project(':lucene:codecs')
testImplementation project(':lucene:test-framework')
}
+
+def momanCommitHash = "5c5c2a1e4dea"
+def momanUrl = "https://bitbucket.org/jpbarrette/moman/get/${momanCommitHash}.zip"
+def python_exe = "python"
+
+def jflexUrl = "https://jflex.de/release/jflex-1.7.0.zip"
+
+// Download and extract Moman
+task installJFlex(type: Download) {
+ src jflexUrl
+ dest file("${buildDir}/jflex.zip")
+ onlyIfModified true
+ doLast {
+ ant.unzip(src: file("${buildDir}/jflex.zip"), dest: file("${buildDir}/jflex"), overwrite: "true") {
+ ant.cutdirsmapper(dirs: "1")
+ }
+ }
+}
+
+task installMoman(type: Download) {
+ src momanUrl
+ dest file("${buildDir}/moman.zip")
+ onlyIfModified true
+
+ doLast {
+ ant.unzip(src: file("${buildDir}/moman.zip"), dest: file("${buildDir}/moman"), overwrite: "true") {
+ ant.cutdirsmapper(dirs: "1")
+ }
+ }
+}
+
+task createPackedIntSources {
+ doLast {
+ exec {
+ workingDir 'src/java/org/apache/lucene/util/packed'
+ executable "${python_exe}"
+ args = ['-B', 'gen_BulkOperation.py']
+ }
+ exec {
+ workingDir 'src/java/org/apache/lucene/util/packed'
+ executable "${python_exe}"
+ args = ['-B', 'gen_Packed64SingleBlock.py']
+ }
+ ant.fixcrlf(srcdir: 'src/java/org/apache/lucene/util/packed', includes: 'Packed64SingleBlock.java', encoding: 'UTF-8')
+ ant.fixcrlf(srcdir: 'src/java/org/apache/lucene/util/packed', includes: 'BulkOperation*.java', encoding: 'UTF-8')
+ }
+}
+
+task createLevAutomaton(dependsOn: installMoman) {
+ doLast {
+ exec {
Review comment:
```
[1, 2].each { edits ->
[True, False].each { transpose ->
exec {
workingDir...
executable...
args = ['-B', createLevAutomata.py, edits, transpose]
}
}
}
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene-solr] madrob commented on a change in pull request #1199:
LUCENE-9134: Port ant-regenerate tasks to Gradle build
Posted by GitBox <gi...@apache.org>.
madrob commented on a change in pull request #1199: LUCENE-9134: Port ant-regenerate tasks to Gradle build
URL: https://github.com/apache/lucene-solr/pull/1199#discussion_r370856696
##########
File path: lucene/core/build.gradle
##########
@@ -1,7 +1,117 @@
-
apply plugin: 'java-library'
+apply plugin: "de.undercouch.download"
dependencies {
testImplementation project(':lucene:codecs')
testImplementation project(':lucene:test-framework')
}
+
+def momanCommitHash = "5c5c2a1e4dea"
+def momanUrl = "https://bitbucket.org/jpbarrette/moman/get/${momanCommitHash}.zip"
+def python_exe = "python"
+
+def jflexUrl = "https://jflex.de/release/jflex-1.7.0.zip"
+
+// Download and extract Moman
+task installJFlex(type: Download) {
+ src jflexUrl
+ dest file("${buildDir}/jflex.zip")
+ onlyIfModified true
+ doLast {
+ ant.unzip(src: file("${buildDir}/jflex.zip"), dest: file("${buildDir}/jflex"), overwrite: "true") {
+ ant.cutdirsmapper(dirs: "1")
+ }
+ }
+}
+
+task installMoman(type: Download) {
Review comment:
Same comments apply here as for `installJFlex`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene-solr] madrob commented on a change in pull request #1199:
LUCENE-9134: Port ant-regenerate tasks to Gradle build
Posted by GitBox <gi...@apache.org>.
madrob commented on a change in pull request #1199: LUCENE-9134: Port ant-regenerate tasks to Gradle build
URL: https://github.com/apache/lucene-solr/pull/1199#discussion_r370856887
##########
File path: lucene/core/build.gradle
##########
@@ -1,7 +1,117 @@
-
apply plugin: 'java-library'
+apply plugin: "de.undercouch.download"
dependencies {
testImplementation project(':lucene:codecs')
testImplementation project(':lucene:test-framework')
}
+
+def momanCommitHash = "5c5c2a1e4dea"
+def momanUrl = "https://bitbucket.org/jpbarrette/moman/get/${momanCommitHash}.zip"
+def python_exe = "python"
+
+def jflexUrl = "https://jflex.de/release/jflex-1.7.0.zip"
+
+// Download and extract Moman
Review comment:
nit: s/Moman/JFlex
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene-solr] madrob commented on a change in pull request #1199:
LUCENE-9134: Port ant-regenerate tasks to Gradle build
Posted by GitBox <gi...@apache.org>.
madrob commented on a change in pull request #1199: LUCENE-9134: Port ant-regenerate tasks to Gradle build
URL: https://github.com/apache/lucene-solr/pull/1199#discussion_r370864083
##########
File path: lucene/queryparser/build.gradle
##########
@@ -7,3 +7,305 @@ dependencies {
testImplementation project(':lucene:test-framework')
}
+
+configure(":lucene:queryparser") {
+ configurations {
+ javaCCDeps
+
+ dependencies {
+ javaCCDeps "net.java.dev.javacc:javacc:5.0"
+ }
+ }
+}
+def conf = [
+ "queryParser": ["inputFile": file('src/java/org/apache/lucene/queryparser/classic/QueryParser.jj'), "outputDir": file('src/java/org/apache/lucene/queryparser/classic')],
+ "surround" : ["inputFile": file('src/java/org/apache/lucene/queryparser/surround/parser/QueryParser.jj'), "outputDir": file('src/java/org/apache/lucene/queryparser/surround/parser/')],
+ "flexible" : ["inputFile": file('src/java/org/apache/lucene/queryparser/flexible/standard/parser/StandardSyntaxParser.jj'), "outputDir": file('src/java/org/apache/lucene/queryparser/flexible/standard/parser')]
+]
+
+String javaCCClasspath
+String javaCCHome
+
+task installJavacc {
+ doLast {
+ javaCCClasspath = project.project(":lucene:queryparser").configurations.javaCCDeps.asPath
+ javaCCHome = javaCCClasspath.substring(0, javaCCClasspath.lastIndexOf("/"))
+
+ String hacky = javaCCHome + "/" + "javacc.jar"
+ File hackyFile = new File(hacky)
+ println hackyFile
+ println hackyFile.exists()
+ if (hackyFile.exists() == false) {
+ hackyFile.bytes = new File(javaCCClasspath).bytes
+ hackyFile.setExecutable(true)
+ }
+ }
+}
+String lineSeparator = System.lineSeparator()
+
+task cleanOldGeneratedFiles {
+ doLast {
+ println "Removing old generated files"
+ conf.each { key, val ->
+ val["outputDir"].eachFileMatch ~/.*\.java/, { f ->
+ if (f.text =~ /Generated.*By.*JavaCC/) {
+ f.delete()
+ }
+ }
+ }
+ }
+}
+
+task runJavaccQueryParser(dependsOn: installJavacc) {
Review comment:
I think because there is so much commonality between all of the javacc tasks, I would create a `task JavaCC` class and put all the clean up logic in one place rather than repeating it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene-solr] madrob commented on a change in pull request #1199:
LUCENE-9134: Port ant-regenerate tasks to Gradle build
Posted by GitBox <gi...@apache.org>.
madrob commented on a change in pull request #1199: LUCENE-9134: Port ant-regenerate tasks to Gradle build
URL: https://github.com/apache/lucene-solr/pull/1199#discussion_r370861960
##########
File path: lucene/queryparser/build.gradle
##########
@@ -7,3 +7,305 @@ dependencies {
testImplementation project(':lucene:test-framework')
}
+
+configure(":lucene:queryparser") {
+ configurations {
+ javaCCDeps
+
+ dependencies {
+ javaCCDeps "net.java.dev.javacc:javacc:5.0"
+ }
+ }
+}
+def conf = [
+ "queryParser": ["inputFile": file('src/java/org/apache/lucene/queryparser/classic/QueryParser.jj'), "outputDir": file('src/java/org/apache/lucene/queryparser/classic')],
+ "surround" : ["inputFile": file('src/java/org/apache/lucene/queryparser/surround/parser/QueryParser.jj'), "outputDir": file('src/java/org/apache/lucene/queryparser/surround/parser/')],
+ "flexible" : ["inputFile": file('src/java/org/apache/lucene/queryparser/flexible/standard/parser/StandardSyntaxParser.jj'), "outputDir": file('src/java/org/apache/lucene/queryparser/flexible/standard/parser')]
+]
+
+String javaCCClasspath
+String javaCCHome
+
+task installJavacc {
+ doLast {
+ javaCCClasspath = project.project(":lucene:queryparser").configurations.javaCCDeps.asPath
+ javaCCHome = javaCCClasspath.substring(0, javaCCClasspath.lastIndexOf("/"))
+
+ String hacky = javaCCHome + "/" + "javacc.jar"
Review comment:
I don't understand what is going on here.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene-solr] madrob commented on a change in pull request #1199:
LUCENE-9134: Port ant-regenerate tasks to Gradle build
Posted by GitBox <gi...@apache.org>.
madrob commented on a change in pull request #1199: LUCENE-9134: Port ant-regenerate tasks to Gradle build
URL: https://github.com/apache/lucene-solr/pull/1199#discussion_r370855505
##########
File path: lucene/core/build.gradle
##########
@@ -1,7 +1,117 @@
-
apply plugin: 'java-library'
+apply plugin: "de.undercouch.download"
dependencies {
testImplementation project(':lucene:codecs')
testImplementation project(':lucene:test-framework')
}
+
+def momanCommitHash = "5c5c2a1e4dea"
+def momanUrl = "https://bitbucket.org/jpbarrette/moman/get/${momanCommitHash}.zip"
+def python_exe = "python"
+
+def jflexUrl = "https://jflex.de/release/jflex-1.7.0.zip"
+
+// Download and extract Moman
+task installJFlex(type: Download) {
+ src jflexUrl
+ dest file("${buildDir}/jflex.zip")
Review comment:
I think this can just be `dest buildDir`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene-solr] madrob commented on a change in pull request #1199:
LUCENE-9134: Port ant-regenerate tasks to Gradle build
Posted by GitBox <gi...@apache.org>.
madrob commented on a change in pull request #1199: LUCENE-9134: Port ant-regenerate tasks to Gradle build
URL: https://github.com/apache/lucene-solr/pull/1199#discussion_r370863721
##########
File path: lucene/queryparser/build.gradle
##########
@@ -7,3 +7,305 @@ dependencies {
testImplementation project(':lucene:test-framework')
}
+
+configure(":lucene:queryparser") {
+ configurations {
+ javaCCDeps
+
+ dependencies {
+ javaCCDeps "net.java.dev.javacc:javacc:5.0"
+ }
+ }
+}
+def conf = [
+ "queryParser": ["inputFile": file('src/java/org/apache/lucene/queryparser/classic/QueryParser.jj'), "outputDir": file('src/java/org/apache/lucene/queryparser/classic')],
+ "surround" : ["inputFile": file('src/java/org/apache/lucene/queryparser/surround/parser/QueryParser.jj'), "outputDir": file('src/java/org/apache/lucene/queryparser/surround/parser/')],
+ "flexible" : ["inputFile": file('src/java/org/apache/lucene/queryparser/flexible/standard/parser/StandardSyntaxParser.jj'), "outputDir": file('src/java/org/apache/lucene/queryparser/flexible/standard/parser')]
+]
+
+String javaCCClasspath
+String javaCCHome
+
+task installJavacc {
+ doLast {
+ javaCCClasspath = project.project(":lucene:queryparser").configurations.javaCCDeps.asPath
+ javaCCHome = javaCCClasspath.substring(0, javaCCClasspath.lastIndexOf("/"))
+
+ String hacky = javaCCHome + "/" + "javacc.jar"
+ File hackyFile = new File(hacky)
+ println hackyFile
+ println hackyFile.exists()
+ if (hackyFile.exists() == false) {
+ hackyFile.bytes = new File(javaCCClasspath).bytes
+ hackyFile.setExecutable(true)
+ }
+ }
+}
+String lineSeparator = System.lineSeparator()
+
+task cleanOldGeneratedFiles {
+ doLast {
+ println "Removing old generated files"
+ conf.each { key, val ->
+ val["outputDir"].eachFileMatch ~/.*\.java/, { f ->
+ if (f.text =~ /Generated.*By.*JavaCC/) {
+ f.delete()
+ }
+ }
+ }
+ }
+}
+
+task runJavaccQueryParser(dependsOn: installJavacc) {
+ doLast {
+ println "Generating QueryParser parser"
+
+ def inputFile = conf["queryParser"]["inputFile"].getAbsolutePath()
+ def outputDir = conf["queryParser"]["outputDir"].getAbsolutePath()
+ ant.javacc(target: inputFile, outputDirectory: outputDir, javacchome: javaCCHome)
+
+
+ ant.replaceregexp(file: new File(outputDir, "QueryParser.java"),
+ byline: "true",
+ match: "public QueryParser\\(CharStream ",
+ replace: "protected QueryParser(CharStream ")
+ ant.replaceregexp(file: new File(outputDir, "QueryParser.java"),
+ byline: "true",
+ match: "public QueryParser\\(QueryParserTokenManager ",
+ replace: "protected QueryParser(QueryParserTokenManager ")
+ }
+}
+
+task runJavaccSurround(dependsOn: installJavacc) {
+ doLast {
+ println "Generating Surroundf parser"
+ def inputFile = conf["surround"]["inputFile"].getAbsolutePath()
+ def outputDir = conf["surround"]["outputDir"].getAbsolutePath()
+
+ ant.javacc(target: inputFile, outputDirectory: outputDir, javacchome: javaCCHome)
+
+ // This is very clumsy, but these are unused imports that are necessary other generated files
+ [
+ "import java.util.Vector;",
+ "import org.apache.lucene.queryparser.flexible.messages.Message;",
+ "import org.apache.lucene.queryparser.flexible.messages.MessageImpl;",
+ "import org.apache.lucene.queryparser.flexible.core.QueryNodeParseException;",
+ "import org.apache.lucene.queryparser.flexible.core.messages.QueryParserMessages;",
+ "import org.apache.lucene.queryparser.flexible.core.nodes.AndQueryNode;",
+ "import org.apache.lucene.queryparser.flexible.core.nodes.BooleanQueryNode;",
+ "import org.apache.lucene.queryparser.flexible.core.nodes.BoostQueryNode;",
+ "import org.apache.lucene.queryparser.flexible.core.nodes.FieldQueryNode;",
+ "import org.apache.lucene.queryparser.flexible.core.nodes.FuzzyQueryNode;",
+ "import org.apache.lucene.queryparser.flexible.core.nodes.ModifierQueryNode;",
+ "import org.apache.lucene.queryparser.flexible.core.nodes.GroupQueryNode;",
+ "import org.apache.lucene.queryparser.flexible.core.nodes.OrQueryNode;",
+ "import org.apache.lucene.queryparser.flexible.standard.nodes.RegexpQueryNode;",
+ "import org.apache.lucene.queryparser.flexible.core.nodes.SlopQueryNode;",
+ "import org.apache.lucene.queryparser.flexible.core.nodes.QueryNode;",
+ "import org.apache.lucene.queryparser.flexible.core.nodes.QuotedFieldQueryNode;",
+ "import org.apache.lucene.queryparser.flexible.core.parser.SyntaxParser;",
+ "import org.apache.lucene.queryparser.flexible.standard.nodes.TermRangeQueryNode;",
+ "import java.io.StringReader;",
+ "import java.util.ArrayList;",
+ "import java.util.Arrays;",
+ "import java.util.HashSet;",
+ "import java.util.List;",
+ "import java.util.Locale;",
+ "import java.util.Set;",
+ "import org.apache.lucene.analysis.Analyzer;",
+ "import org.apache.lucene.document.DateTools;",
+ "import org.apache.lucene.search.BooleanClause;",
+ "import org.apache.lucene.search.Query;",
+ "import org.apache.lucene.search.TermRangeQuery;",
+ "import org.apache.lucene.analysis.TokenStream;",
+ "import org.apache.lucene.queryparser.surround.query.SrndQuery;",
+ "import org.apache.lucene.queryparser.surround.query.FieldsQuery;",
+ "import org.apache.lucene.queryparser.surround.query.OrQuery;",
+ "import org.apache.lucene.queryparser.surround.query.AndQuery;",
+ "import org.apache.lucene.queryparser.surround.query.NotQuery;",
+ "import org.apache.lucene.queryparser.surround.query.DistanceQuery;",
+ "import org.apache.lucene.queryparser.surround.query.SrndTermQuery;",
+ "import org.apache.lucene.queryparser.surround.query.SrndPrefixQuery;",
+ "import org.apache.lucene.queryparser.surround.query.SrndTruncQuery;"
+ ].each {
+ ant.replaceregexp(match: "${it}.*${lineSeparator}", replace: "", encoding: "UTF-8") {
+ ant.fileset(dir: outputDir, includes: "*TokenManager.java")
+ }
+ }
+ // StringBuffer -> StringBuilder
+ ant.replace(token: "StringBuffer", value: "StringBuilder", encoding: "UTF-8") {
+ ant.fileset(dir: outputDir, includes: "ParseException.java TokenMgrError.java")
+ }
+
+ [
+ "import org.apache.lucene.analysis.TokenStream;",
+ ].each {
+ ant.replaceregexp(match: "${it}.*${lineSeparator}", replace: "", encoding: "UTF-8") {
+ ant.fileset(dir: outputDir, includes: "QueryParser.java")
+ }
+ }
+ // StringBuffer -> StringBuilder
Review comment:
You have this twice?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene-solr] ErickErickson commented on a change in pull request
#1199: LUCENE-9134: Port ant-regenerate tasks to Gradle build
Posted by GitBox <gi...@apache.org>.
ErickErickson commented on a change in pull request #1199: LUCENE-9134: Port ant-regenerate tasks to Gradle build
URL: https://github.com/apache/lucene-solr/pull/1199#discussion_r370970749
##########
File path: lucene/queryparser/build.gradle
##########
@@ -7,3 +7,305 @@ dependencies {
testImplementation project(':lucene:test-framework')
}
+
+configure(":lucene:queryparser") {
+ configurations {
+ javaCCDeps
+
+ dependencies {
+ javaCCDeps "net.java.dev.javacc:javacc:5.0"
+ }
+ }
+}
+def conf = [
+ "queryParser": ["inputFile": file('src/java/org/apache/lucene/queryparser/classic/QueryParser.jj'), "outputDir": file('src/java/org/apache/lucene/queryparser/classic')],
+ "surround" : ["inputFile": file('src/java/org/apache/lucene/queryparser/surround/parser/QueryParser.jj'), "outputDir": file('src/java/org/apache/lucene/queryparser/surround/parser/')],
+ "flexible" : ["inputFile": file('src/java/org/apache/lucene/queryparser/flexible/standard/parser/StandardSyntaxParser.jj'), "outputDir": file('src/java/org/apache/lucene/queryparser/flexible/standard/parser')]
+]
+
+String javaCCClasspath
+String javaCCHome
+
+task installJavacc {
+ doLast {
+ javaCCClasspath = project.project(":lucene:queryparser").configurations.javaCCDeps.asPath
+ javaCCHome = javaCCClasspath.substring(0, javaCCClasspath.lastIndexOf("/"))
+
+ String hacky = javaCCHome + "/" + "javacc.jar"
+ File hackyFile = new File(hacky)
+ println hackyFile
+ println hackyFile.exists()
+ if (hackyFile.exists() == false) {
+ hackyFile.bytes = new File(javaCCClasspath).bytes
+ hackyFile.setExecutable(true)
+ }
+ }
+}
+String lineSeparator = System.lineSeparator()
+
+task cleanOldGeneratedFiles {
+ doLast {
+ println "Removing old generated files"
+ conf.each { key, val ->
+ val["outputDir"].eachFileMatch ~/.*\.java/, { f ->
+ if (f.text =~ /Generated.*By.*JavaCC/) {
+ f.delete()
+ }
+ }
+ }
+ }
+}
+
+task runJavaccQueryParser(dependsOn: installJavacc) {
Review comment:
Gah, it's worse than that. The common bits _are_ in one place, see cleanUpAfter. I missed taking it out of a place or two. Fixing it up.
So there's only one small bit that's repeated now, I don't think it's worth a new class now.
And many thanks for your review! I'll get the hang of this yet.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene-solr] dweiss commented on issue #1199: LUCENE-9134: Port
ant-regenerate tasks to Gradle build
Posted by GitBox <gi...@apache.org>.
dweiss commented on issue #1199: LUCENE-9134: Port ant-regenerate tasks to Gradle build
URL: https://github.com/apache/lucene-solr/pull/1199#issuecomment-578532163
This is great work, Erick and is very much appreciated. I do have a "but" though -- it's large and goes through a number of those tasks at once. I'm sorry I've been slow in taking in your patches. I can't really find a chunk of time large enough to review and correct certain issues in a large patch like this one.
I'd really like to have minimalistic build fragments that only deal with one thing at a time. It's different from ant (and arguably different from how other projects structure gradle builds) but to me it makes reasoning about a particular build aspect simpler.
Take jflex for example as it is really self-contained. You need to have access to jflex at a given version (no need to download anything -- you just declare a configuration and a dependency), you need a top-level task (so that it shows up in help) and you need to configure tasks that are attached to it in each project where we generate stuff from jflex files.
I just committed an example that regenerates StandardTokenizerImpl in lucene/core - please take a look at the sources and see if it matches what I tried to express above. When you run "gradlew jflex" it'll recreate StandardTokenizerImpl.java... in fact when you run git diff you won't even see the difference because the regenerated file is identical to what it was before (which I think should be an ideal goal for now because we don't want to generate stuff other than ant does).
The remaining jflex regeneration targets can be appended to this file, making it a clean, single-objective concern.
When or if at some point somebody decides that a different way to deal with jflex files is more attractive (for example use an external plugin or move the custom task to buildSrc) those changes remain pretty much local to this file.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene-solr] ErickErickson commented on a change in pull request
#1199: LUCENE-9134: Port ant-regenerate tasks to Gradle build
Posted by GitBox <gi...@apache.org>.
ErickErickson commented on a change in pull request #1199: LUCENE-9134: Port ant-regenerate tasks to Gradle build
URL: https://github.com/apache/lucene-solr/pull/1199#discussion_r370967681
##########
File path: lucene/core/build.gradle
##########
@@ -1,7 +1,117 @@
-
apply plugin: 'java-library'
+apply plugin: "de.undercouch.download"
dependencies {
testImplementation project(':lucene:codecs')
testImplementation project(':lucene:test-framework')
}
+
+def momanCommitHash = "5c5c2a1e4dea"
+def momanUrl = "https://bitbucket.org/jpbarrette/moman/get/${momanCommitHash}.zip"
+def python_exe = "python"
+
+def jflexUrl = "https://jflex.de/release/jflex-1.7.0.zip"
+
+// Download and extract Moman
Review comment:
See previous.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene-solr] dweiss commented on issue #1199: LUCENE-9134: Port
ant-regenerate tasks to Gradle build
Posted by GitBox <gi...@apache.org>.
dweiss commented on issue #1199: LUCENE-9134: Port ant-regenerate tasks to Gradle build
URL: https://github.com/apache/lucene-solr/pull/1199#issuecomment-577339879
Thanks Erick. I'll get back to it, I just didn't have as much time as before.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene-solr] ErickErickson commented on a change in pull request
#1199: LUCENE-9134: Port ant-regenerate tasks to Gradle build
Posted by GitBox <gi...@apache.org>.
ErickErickson commented on a change in pull request #1199: LUCENE-9134: Port ant-regenerate tasks to Gradle build
URL: https://github.com/apache/lucene-solr/pull/1199#discussion_r370967657
##########
File path: lucene/core/build.gradle
##########
@@ -1,7 +1,117 @@
-
apply plugin: 'java-library'
+apply plugin: "de.undercouch.download"
dependencies {
testImplementation project(':lucene:codecs')
testImplementation project(':lucene:test-framework')
}
+
+def momanCommitHash = "5c5c2a1e4dea"
+def momanUrl = "https://bitbucket.org/jpbarrette/moman/get/${momanCommitHash}.zip"
+def python_exe = "python"
+
+def jflexUrl = "https://jflex.de/release/jflex-1.7.0.zip"
+
+// Download and extract Moman
+task installJFlex(type: Download) {
+ src jflexUrl
+ dest file("${buildDir}/jflex.zip")
Review comment:
Right, done.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene-solr] ErickErickson closed pull request #1199: LUCENE-9134:
Port ant-regenerate tasks to Gradle build
Posted by GitBox <gi...@apache.org>.
ErickErickson closed pull request #1199: LUCENE-9134: Port ant-regenerate tasks to Gradle build
URL: https://github.com/apache/lucene-solr/pull/1199
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene-solr] madrob commented on a change in pull request #1199:
LUCENE-9134: Port ant-regenerate tasks to Gradle build
Posted by GitBox <gi...@apache.org>.
madrob commented on a change in pull request #1199: LUCENE-9134: Port ant-regenerate tasks to Gradle build
URL: https://github.com/apache/lucene-solr/pull/1199#discussion_r370856535
##########
File path: lucene/core/build.gradle
##########
@@ -1,7 +1,117 @@
-
apply plugin: 'java-library'
+apply plugin: "de.undercouch.download"
dependencies {
testImplementation project(':lucene:codecs')
testImplementation project(':lucene:test-framework')
}
+
+def momanCommitHash = "5c5c2a1e4dea"
+def momanUrl = "https://bitbucket.org/jpbarrette/moman/get/${momanCommitHash}.zip"
+def python_exe = "python"
+
+def jflexUrl = "https://jflex.de/release/jflex-1.7.0.zip"
+
+// Download and extract Moman
+task installJFlex(type: Download) {
+ src jflexUrl
+ dest file("${buildDir}/jflex.zip")
+ onlyIfModified true
+ doLast {
Review comment:
I would create a separate Copy task for this part and use Gradle's built in zip support. Example: https://github.com/michel-kraemer/gradle-download-task#download-and-extract-a-zip-file
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene-solr] ErickErickson commented on a change in pull request
#1199: LUCENE-9134: Port ant-regenerate tasks to Gradle build
Posted by GitBox <gi...@apache.org>.
ErickErickson commented on a change in pull request #1199: LUCENE-9134: Port ant-regenerate tasks to Gradle build
URL: https://github.com/apache/lucene-solr/pull/1199#discussion_r370967671
##########
File path: lucene/core/build.gradle
##########
@@ -1,7 +1,117 @@
-
apply plugin: 'java-library'
+apply plugin: "de.undercouch.download"
dependencies {
testImplementation project(':lucene:codecs')
testImplementation project(':lucene:test-framework')
}
+
+def momanCommitHash = "5c5c2a1e4dea"
+def momanUrl = "https://bitbucket.org/jpbarrette/moman/get/${momanCommitHash}.zip"
+def python_exe = "python"
+
+def jflexUrl = "https://jflex.de/release/jflex-1.7.0.zip"
+
+// Download and extract Moman
+task installJFlex(type: Download) {
+ src jflexUrl
+ dest file("${buildDir}/jflex.zip")
+ onlyIfModified true
+ doLast {
Review comment:
Actually, I found a way to combine them all, thanks for suggesting this, made me look closer.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene-solr] madrob commented on a change in pull request #1199:
LUCENE-9134: Port ant-regenerate tasks to Gradle build
Posted by GitBox <gi...@apache.org>.
madrob commented on a change in pull request #1199: LUCENE-9134: Port ant-regenerate tasks to Gradle build
URL: https://github.com/apache/lucene-solr/pull/1199#discussion_r370858964
##########
File path: lucene/core/build.gradle
##########
@@ -1,7 +1,117 @@
-
apply plugin: 'java-library'
+apply plugin: "de.undercouch.download"
dependencies {
testImplementation project(':lucene:codecs')
testImplementation project(':lucene:test-framework')
}
+
+def momanCommitHash = "5c5c2a1e4dea"
+def momanUrl = "https://bitbucket.org/jpbarrette/moman/get/${momanCommitHash}.zip"
+def python_exe = "python"
+
+def jflexUrl = "https://jflex.de/release/jflex-1.7.0.zip"
+
+// Download and extract Moman
+task installJFlex(type: Download) {
+ src jflexUrl
+ dest file("${buildDir}/jflex.zip")
+ onlyIfModified true
+ doLast {
+ ant.unzip(src: file("${buildDir}/jflex.zip"), dest: file("${buildDir}/jflex"), overwrite: "true") {
+ ant.cutdirsmapper(dirs: "1")
+ }
+ }
+}
+
+task installMoman(type: Download) {
+ src momanUrl
+ dest file("${buildDir}/moman.zip")
+ onlyIfModified true
+
+ doLast {
+ ant.unzip(src: file("${buildDir}/moman.zip"), dest: file("${buildDir}/moman"), overwrite: "true") {
+ ant.cutdirsmapper(dirs: "1")
+ }
+ }
+}
+
+task createPackedIntSources {
+ doLast {
+ exec {
+ workingDir 'src/java/org/apache/lucene/util/packed'
+ executable "${python_exe}"
+ args = ['-B', 'gen_BulkOperation.py']
+ }
+ exec {
+ workingDir 'src/java/org/apache/lucene/util/packed'
+ executable "${python_exe}"
+ args = ['-B', 'gen_Packed64SingleBlock.py']
+ }
+ ant.fixcrlf(srcdir: 'src/java/org/apache/lucene/util/packed', includes: 'Packed64SingleBlock.java', encoding: 'UTF-8')
+ ant.fixcrlf(srcdir: 'src/java/org/apache/lucene/util/packed', includes: 'BulkOperation*.java', encoding: 'UTF-8')
+ }
+}
+
+task createLevAutomaton(dependsOn: installMoman) {
+ doLast {
+ exec {
+ workingDir 'src/java/org/apache/lucene/util/automaton'
+ executable "${python_exe}"
+ args = ['-B', 'createLevAutomata.py', '1', 'True']
+ }
+ exec {
+ workingDir 'src/java/org/apache/lucene/util/automaton'
+ executable "${python_exe}"
+ args = ['-B', 'createLevAutomata.py', '1', 'False']
+ }
+ exec {
+ workingDir 'src/java/org/apache/lucene/util/automaton'
+ executable "${python_exe}"
+ args = ['-B', 'createLevAutomata.py', '2', 'True']
+ }
+ exec {
+ workingDir 'src/java/org/apache/lucene/util/automaton'
+ executable "${python_exe}"
+ args = ['-B', 'createLevAutomata.py', '2', 'False']
+ }
+ ant.fixcrlf(srcdir: 'src/java/org/apache/lucene/util/automaton', includes: '*ParametricDescription.java', encoding: 'UTF-8')
+ }
+}
+
+task runJFlex(dependsOn: installJFlex, type: JavaExec) {
+ classpath = files("${buildDir}/jflex/lib/jflex-full-1.7.0.jar")
+ main = "jflex.Main"
+
+ // arguments to pass to the application
+ args = ['--skel', project.project(":lucene:core").projectDir.getAbsolutePath() + "/src/data/jflex/skeleton.disable.buffer.expansion.txt",
+ '--nobak',
+ 'src/java/org/apache/lucene/analysis/standard/StandardTokenizerImpl.jflex'
+ ]
+ doLast {
+ //Since the ZZ_BUFFERSIZE declaration is generated rather than in the skeleton, we have to transform it here.
+ ant.replace(token: "private static final int ZZ_BUFFERSIZE", value: "private int ZZ_BUFFERSIZE", encoding: "UTF-8") {
+ ant.fileset(dir: "src/java/org/apache/lucene/analysis/standard", includes: "StandardTokenizerImpl.java")
+ }
+ }
+}
+
+task cleanJFlex {
Review comment:
if you define a `jflex` task that has registered outputs, then gradle will automatically create a corresponding `cleanJflex` task for you.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene-solr] ErickErickson commented on a change in pull request
#1199: LUCENE-9134: Port ant-regenerate tasks to Gradle build
Posted by GitBox <gi...@apache.org>.
ErickErickson commented on a change in pull request #1199: LUCENE-9134: Port ant-regenerate tasks to Gradle build
URL: https://github.com/apache/lucene-solr/pull/1199#discussion_r370969055
##########
File path: lucene/queryparser/build.gradle
##########
@@ -7,3 +7,305 @@ dependencies {
testImplementation project(':lucene:test-framework')
}
+
+configure(":lucene:queryparser") {
+ configurations {
+ javaCCDeps
+
+ dependencies {
+ javaCCDeps "net.java.dev.javacc:javacc:5.0"
+ }
+ }
+}
+def conf = [
+ "queryParser": ["inputFile": file('src/java/org/apache/lucene/queryparser/classic/QueryParser.jj'), "outputDir": file('src/java/org/apache/lucene/queryparser/classic')],
+ "surround" : ["inputFile": file('src/java/org/apache/lucene/queryparser/surround/parser/QueryParser.jj'), "outputDir": file('src/java/org/apache/lucene/queryparser/surround/parser/')],
+ "flexible" : ["inputFile": file('src/java/org/apache/lucene/queryparser/flexible/standard/parser/StandardSyntaxParser.jj'), "outputDir": file('src/java/org/apache/lucene/queryparser/flexible/standard/parser')]
+]
+
+String javaCCClasspath
+String javaCCHome
+
+task installJavacc {
+ doLast {
+ javaCCClasspath = project.project(":lucene:queryparser").configurations.javaCCDeps.asPath
+ javaCCHome = javaCCClasspath.substring(0, javaCCClasspath.lastIndexOf("/"))
+
+ String hacky = javaCCHome + "/" + "javacc.jar"
Review comment:
From what I can tell, the ant JavaCC task _requires_ the file be named javacc.jar and what's downloaded is javacc-5.0.jar (ancient, I know). I can't find a place to download it from so I'm relying on the Gradle dependency magic to get it in the first place. But then it needs to be renamed. I'll make this a little neater and add a comment.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org