You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@lucene.apache.org by GitBox <gi...@apache.org> on 2021/01/12 22:28:55 UTC
[GitHub] [lucene-solr] HoustonPutman opened a new pull request #2197: SOLR-15075: Solr docker gradle improvements
HoustonPutman opened a new pull request #2197:
URL: https://github.com/apache/lucene-solr/pull/2197
https://issues.apache.org/jira/browse/SOLR-15075
WIP
Goals
- [ ] Remove Gradle docker plugin
- [ ] Remove docker-package intermediate image
- [ ] Move docker gradle help
- [ ] Make Gradle tasks respect changes to inputs
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene-solr] dsmiley commented on a change in pull request #2197: SOLR-15075: Solr docker gradle improvements
Posted by GitBox <gi...@apache.org>.
dsmiley commented on a change in pull request #2197:
URL: https://github.com/apache/lucene-solr/pull/2197#discussion_r559091594
##########
File path: solr/docker/build.gradle
##########
@@ -18,106 +18,192 @@
import com.google.common.base.Preconditions
import com.google.common.base.Strings
-apply plugin: 'base'
-apply plugin: 'com.palantir.docker'
-
-subprojects {
- apply plugin: 'base'
- apply plugin: 'com.palantir.docker'
-}
-
description = 'Solr Docker image'
-def dockerPackage = project(':solr:docker:package')
-
-dependencies {
- docker dockerPackage
-}
+apply plugin: 'base'
+// Solr Docker inputs
def dockerImageRepo = propertyOrEnvOrDefault("solr.docker.imageRepo", "SOLR_DOCKER_IMAGE_REPO", "apache/solr")
def dockerImageTag = propertyOrEnvOrDefault("solr.docker.imageTag", "SOLR_DOCKER_IMAGE_TAG", "${version}")
def dockerImageName = propertyOrEnvOrDefault("solr.docker.imageName", "SOLR_DOCKER_IMAGE_NAME", "${dockerImageRepo}:${dockerImageTag}")
def baseDockerImage = propertyOrEnvOrDefault("solr.docker.baseImage", "SOLR_DOCKER_BASE_IMAGE", 'openjdk:11-jre-slim')
def githubUrlOrMirror = propertyOrEnvOrDefault("solr.docker.githubUrl", "SOLR_DOCKER_GITHUB_URL", 'github.com')
-docker {
- name = dockerImageName
- files file('include')
- buildArgs(['BASE_IMAGE' : baseDockerImage, 'SOLR_PACKAGE_IMAGE' : 'apache/solr-build:local-package', 'SOLR_VERSION': "${version}", 'GITHUB_URL': githubUrlOrMirror])
+// Build directory locations
+def dockerBuildDistribution = "$buildDir/distributions"
+def imageIdFile = "$buildDir/image-id"
+
+configurations {
+ packaging {
+ canBeResolved = true
+ }
+ dockerImage {
+ canBeResolved = true
+ }
+}
+
+dependencies {
+ packaging project(path: ":solr:packaging", configuration: 'archives')
+
+ dockerImage files(imageIdFile) {
+ builtBy 'dockerBuild'
+ }
}
-tasks.docker {
- // In order to create the solr docker image, the solr package image must be created first.
- dependsOn(dockerPackage.tasks.docker)
+task dockerTar(type: Tar) {
Review comment:
Oops; I'm mistaken, as it must contain the Dockerfile at the root.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene-solr] dsmiley commented on a change in pull request #2197: SOLR-15075: Solr docker gradle improvements
Posted by GitBox <gi...@apache.org>.
dsmiley commented on a change in pull request #2197:
URL: https://github.com/apache/lucene-solr/pull/2197#discussion_r560544831
##########
File path: solr/docker/build.gradle
##########
@@ -18,106 +18,192 @@
import com.google.common.base.Preconditions
import com.google.common.base.Strings
-apply plugin: 'base'
-apply plugin: 'com.palantir.docker'
-
-subprojects {
- apply plugin: 'base'
- apply plugin: 'com.palantir.docker'
-}
-
description = 'Solr Docker image'
-def dockerPackage = project(':solr:docker:package')
-
-dependencies {
- docker dockerPackage
-}
+apply plugin: 'base'
+// Solr Docker inputs
def dockerImageRepo = propertyOrEnvOrDefault("solr.docker.imageRepo", "SOLR_DOCKER_IMAGE_REPO", "apache/solr")
def dockerImageTag = propertyOrEnvOrDefault("solr.docker.imageTag", "SOLR_DOCKER_IMAGE_TAG", "${version}")
def dockerImageName = propertyOrEnvOrDefault("solr.docker.imageName", "SOLR_DOCKER_IMAGE_NAME", "${dockerImageRepo}:${dockerImageTag}")
def baseDockerImage = propertyOrEnvOrDefault("solr.docker.baseImage", "SOLR_DOCKER_BASE_IMAGE", 'openjdk:11-jre-slim')
def githubUrlOrMirror = propertyOrEnvOrDefault("solr.docker.githubUrl", "SOLR_DOCKER_GITHUB_URL", 'github.com')
-docker {
- name = dockerImageName
- files file('include')
- buildArgs(['BASE_IMAGE' : baseDockerImage, 'SOLR_PACKAGE_IMAGE' : 'apache/solr-build:local-package', 'SOLR_VERSION': "${version}", 'GITHUB_URL': githubUrlOrMirror])
+// Build directory locations
+def dockerBuildDistribution = "$buildDir/distributions"
+def imageIdFile = "$buildDir/image-id"
+
+configurations {
+ packaging {
+ canBeResolved = true
+ }
+ dockerImage {
+ canBeResolved = true
+ }
+}
+
+dependencies {
+ packaging project(path: ":solr:packaging", configuration: 'archives')
+
+ dockerImage files(imageIdFile) {
+ builtBy 'dockerBuild'
+ }
}
-tasks.docker {
- // In order to create the solr docker image, the solr package image must be created first.
- dependsOn(dockerPackage.tasks.docker)
+task dockerTar(type: Tar) {
Review comment:
The plan for 1P packages does not conflict with this. They remain where they are, and eventually might only ship metadata in the image instead of all the JARs which would be fetched on demand. I don't think there's a Docker impact to that.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene-solr] HoustonPutman commented on a change in pull request #2197: SOLR-15075: Solr docker gradle improvements
Posted by GitBox <gi...@apache.org>.
HoustonPutman commented on a change in pull request #2197:
URL: https://github.com/apache/lucene-solr/pull/2197#discussion_r556607070
##########
File path: gradle/help.gradle
##########
@@ -30,7 +30,7 @@ configure(rootProject) {
["Git", "help/git.txt", "Git assistance and guides."],
["ValidateLogCalls", "help/validateLogCalls.txt", "How to use logging calls efficiently."],
["IDEs", "help/IDEs.txt", "IDE support."],
- ["Docker", "help/docker.txt", "Building Solr Docker images."],
Review comment:
I was moving based off of a comment from @dsmiley. I could go either way. I do think it makes sense that the documentation for the docker module resides in the docker module. The other help files in that directory seem to be applicable project-wide, not for a specific module.
##########
File path: solr/docker/build.gradle
##########
@@ -18,106 +18,187 @@
import com.google.common.base.Preconditions
import com.google.common.base.Strings
-apply plugin: 'base'
-apply plugin: 'com.palantir.docker'
-
-subprojects {
- apply plugin: 'base'
- apply plugin: 'com.palantir.docker'
-}
-
description = 'Solr Docker image'
-def dockerPackage = project(':solr:docker:package')
-
-dependencies {
- docker dockerPackage
-}
+apply plugin: 'base'
+// Solr Docker inputs
def dockerImageRepo = propertyOrEnvOrDefault("solr.docker.imageRepo", "SOLR_DOCKER_IMAGE_REPO", "apache/solr")
def dockerImageTag = propertyOrEnvOrDefault("solr.docker.imageTag", "SOLR_DOCKER_IMAGE_TAG", "${version}")
def dockerImageName = propertyOrEnvOrDefault("solr.docker.imageName", "SOLR_DOCKER_IMAGE_NAME", "${dockerImageRepo}:${dockerImageTag}")
def baseDockerImage = propertyOrEnvOrDefault("solr.docker.baseImage", "SOLR_DOCKER_BASE_IMAGE", 'openjdk:11-jre-slim')
def githubUrlOrMirror = propertyOrEnvOrDefault("solr.docker.githubUrl", "SOLR_DOCKER_GITHUB_URL", 'github.com')
-docker {
- name = dockerImageName
- files file('include')
- buildArgs(['BASE_IMAGE' : baseDockerImage, 'SOLR_PACKAGE_IMAGE' : 'apache/solr-build:local-package', 'SOLR_VERSION': "${version}", 'GITHUB_URL': githubUrlOrMirror])
+// Build directory locations
+def dockerBuildDistribution = "$buildDir/distributions"
+def imageIdFile = "$buildDir/image-id"
+
+configurations {
+ packaging {
+ canBeResolved = true
+ }
+ dockerImage {
+ canBeResolved = true
+ }
+}
+
+dependencies {
+ packaging project(path: ":solr:packaging", configuration: 'archives')
+
+ dockerImage files(imageIdFile) {
+ builtBy 'dockerBuild'
+ }
+}
+
+task dockerTar(type: Tar) {
+ group = 'Docker'
+ description = 'Package docker context to prepare for docker build'
+
+ dependsOn configurations.packaging
+ into('scripts') {
+ from file('scripts')
+ fileMode 755
+ }
+ into('releases') {
+ from configurations.packaging
+ include '*.tgz'
+ }
+ from file('Dockerfile')
+ destinationDirectory = file(dockerBuildDistribution)
+ extension 'tgz'
+ compression = Compression.GZIP
}
-tasks.docker {
- // In order to create the solr docker image, the solr package image must be created first.
- dependsOn(dockerPackage.tasks.docker)
+task dockerBuild(dependsOn: tasks.dockerTar) {
+ group = 'Docker'
+ description = 'Build Solr docker image'
+
+ // Ensure that the docker image is rebuilt on build-arg changes or changes in the docker context
+ inputs.properties([
+ baseDockerImage: baseDockerImage,
+ githubUrlOrMirror: githubUrlOrMirror,
+ version: version
+ ])
+ inputs.dir(dockerBuildDistribution)
+
+ doLast {
+ exec {
+ standardInput = tasks.dockerTar.outputs.files.singleFile.newDataInputStream()
+ commandLine "docker", "build",
+ "--iidfile", imageIdFile,
+ "--build-arg", "BASE_IMAGE=${inputs.properties.baseDockerImage}",
+ "--build-arg", "SOLR_VERSION=${version}",
+ "--build-arg", "GITHUB_URL=${inputs.properties.githubUrlOrMirror}",
+ "-"
+ }
+ }
// Print information on the image after it has been created
doLast {
+ def dockerImageId = file(imageIdFile).text
project.logger.lifecycle("Solr Docker Image Created")
- project.logger.lifecycle("\tName: $dockerImageName")
- project.logger.lifecycle("\tBase Image: $baseDockerImage")
+ project.logger.lifecycle("\tID: \t$dockerImageId")
+ project.logger.lifecycle("\tBase Image: \t$baseDockerImage")
+ project.logger.lifecycle("\tSolr Version: \t$version")
}
+
+ outputs.files(imageIdFile)
}
-abstract class DockerTestSuite extends DefaultTask {
- private String solrImageName = null;
- private List<String> tests = new ArrayList<>();
- private List<String> ignore = new ArrayList<>();
+task dockerTag(dependsOn: tasks.dockerBuild) {
+ group = 'Docker'
+ description = 'Tag Solr docker image'
- @OutputDirectory
- abstract DirectoryProperty getOutputDir()
+ // Ensure that the docker image is re-tagged if the image ID or desired tag changes
+ inputs.properties([
+ dockerImageName: dockerImageName,
+ ])
+ inputs.file(imageIdFile)
- public void setSolrImageName(String solrImageName) {
- this.solrImageName = solrImageName
+ doLast {
+ exec {
+ commandLine "docker", "tag", tasks.dockerBuild.outputs.files.singleFile.text, inputs.properties.dockerImageName
+ }
}
- public String getSolrImageName() {
- Preconditions.checkArgument(!Strings.isNullOrEmpty(solrImageName), "solrImageName is a required dockerTests configuration item.")
- return solrImageName
+ // Print information on the image after it has been created
+ doLast {
+ def dockerImageId = tasks.dockerBuild.outputs.files.singleFile.text
+ project.logger.lifecycle("Solr Docker Image Tagged")
+ project.logger.lifecycle("\tID: \t$dockerImageId")
+ project.logger.lifecycle("\tTag: \t$dockerImageName")
}
+}
- @Option(option = "tests", description = "Only run these specified tests, comma separated.")
- public void setTests(List<String> tests) {
- this.tests = tests;
- }
+task dockerPush(dependsOn: tasks.dockerTag) {
+ group = 'Docker'
+ description = 'Push Solr docker image'
- @Input
- public List<String> getTests() {
- return tests;
- }
+ // Ensure that the docker image is re-pushed if the image ID or tag changes
+ inputs.properties([
+ dockerImageName: dockerImageName,
+ ])
+ inputs.file(imageIdFile)
- @Option(option = "ignore", description = "Ignore these tests, comma separated.")
- public void setIgnore(List<String> ignore) {
- this.ignore = ignore;
+ doLast {
+ exec {
+ commandLine "docker", "push", dockerImageName
+ }
}
- @Input
- public List<String> getIgnore() {
- return ignore;
+ // Print information on the image after it has been created
+ doLast {
+ project.logger.lifecycle("Solr Docker Image Pushed: \t$dockerImageName")
}
+}
+
+
+task testDocker(dependsOn: tasks.dockerBuild) {
+ group = 'Docker'
+ description = 'Test Solr docker image'
+
+ def inputDir = "tests/cases"
+ def outputDir = "$buildDir/tmp/tests"
+
+ // Ensure that the docker image is re-tested if the image ID changes or the test files change
+ inputs.properties([
Review comment:
The type signature of the method takes a Map<String, Object>. But it'd be trivial to use a string. I'll switch just to be safe.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene-solr] dweiss commented on a change in pull request #2197: SOLR-15075: Solr docker gradle improvements
Posted by GitBox <gi...@apache.org>.
dweiss commented on a change in pull request #2197:
URL: https://github.com/apache/lucene-solr/pull/2197#discussion_r557282449
##########
File path: solr/docker/build.gradle
##########
@@ -18,106 +18,203 @@
import com.google.common.base.Preconditions
import com.google.common.base.Strings
-apply plugin: 'base'
-apply plugin: 'com.palantir.docker'
-
-subprojects {
- apply plugin: 'base'
- apply plugin: 'com.palantir.docker'
-}
-
description = 'Solr Docker image'
-def dockerPackage = project(':solr:docker:package')
-
-dependencies {
- docker dockerPackage
-}
+apply plugin: 'base'
+// Solr Docker inputs
def dockerImageRepo = propertyOrEnvOrDefault("solr.docker.imageRepo", "SOLR_DOCKER_IMAGE_REPO", "apache/solr")
def dockerImageTag = propertyOrEnvOrDefault("solr.docker.imageTag", "SOLR_DOCKER_IMAGE_TAG", "${version}")
def dockerImageName = propertyOrEnvOrDefault("solr.docker.imageName", "SOLR_DOCKER_IMAGE_NAME", "${dockerImageRepo}:${dockerImageTag}")
def baseDockerImage = propertyOrEnvOrDefault("solr.docker.baseImage", "SOLR_DOCKER_BASE_IMAGE", 'openjdk:11-jre-slim')
def githubUrlOrMirror = propertyOrEnvOrDefault("solr.docker.githubUrl", "SOLR_DOCKER_GITHUB_URL", 'github.com')
-docker {
- name = dockerImageName
- files file('include')
- buildArgs(['BASE_IMAGE' : baseDockerImage, 'SOLR_PACKAGE_IMAGE' : 'apache/solr-build:local-package', 'SOLR_VERSION': "${version}", 'GITHUB_URL': githubUrlOrMirror])
+// Build directory locations
+def dockerBuildDistribution = "$buildDir/distributions"
+def imageIdFile = "$buildDir/image-id"
+
+configurations {
+ packaging {
+ canBeResolved = true
+ }
+ dockerImage {
+ canBeResolved = true
+ }
}
-tasks.docker {
- // In order to create the solr docker image, the solr package image must be created first.
- dependsOn(dockerPackage.tasks.docker)
+dependencies {
+ packaging project(path: ":solr:packaging", configuration: 'archives')
+
+ dockerImage files(imageIdFile) {
+ builtBy 'dockerBuild'
+ }
+}
+
+task dockerTar(type: Tar) {
+ group = 'Docker'
+ description = 'Package docker context to prepare for docker build'
+
+ dependsOn configurations.packaging
+ into('scripts') {
+ from file('scripts')
+ fileMode 755
+ }
+ into('releases') {
+ from configurations.packaging
+ include '*.tgz'
+ }
+ from file('Dockerfile')
+ destinationDirectory = file(dockerBuildDistribution)
+ extension 'tgz'
+ compression = Compression.GZIP
+}
+
+task dockerBuild(dependsOn: tasks.dockerTar) {
+ group = 'Docker'
+ description = 'Build Solr docker image'
+
+ // Ensure that the docker image is rebuilt on build-arg changes or changes in the docker context
+ inputs.properties([
+ baseDockerImage: baseDockerImage,
+ githubUrlOrMirror: githubUrlOrMirror,
+ version: version
+ ])
+ inputs.dir(dockerBuildDistribution)
+
+ // Docker build must run after the docker context has been created & tarred
+ mustRunAfter tasks.dockerTar
Review comment:
It looks like a dependency to me, not a precondition - the behavior you described (dockerTag rebuilding the image in the second command) makes sense to me. If there is more than one option controlling something, you should just specify them (second command) and not rely on any previous build commands.
This is my gradle side viewpoint. I don't know that much about docker to be able to tell whether this makes sense from how these deployment processes are organized.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene-solr] dweiss commented on a change in pull request #2197: SOLR-15075: Solr docker gradle improvements
Posted by GitBox <gi...@apache.org>.
dweiss commented on a change in pull request #2197:
URL: https://github.com/apache/lucene-solr/pull/2197#discussion_r557687777
##########
File path: solr/docker/build.gradle
##########
@@ -18,106 +18,203 @@
import com.google.common.base.Preconditions
import com.google.common.base.Strings
-apply plugin: 'base'
-apply plugin: 'com.palantir.docker'
-
-subprojects {
- apply plugin: 'base'
- apply plugin: 'com.palantir.docker'
-}
-
description = 'Solr Docker image'
-def dockerPackage = project(':solr:docker:package')
-
-dependencies {
- docker dockerPackage
-}
+apply plugin: 'base'
+// Solr Docker inputs
def dockerImageRepo = propertyOrEnvOrDefault("solr.docker.imageRepo", "SOLR_DOCKER_IMAGE_REPO", "apache/solr")
def dockerImageTag = propertyOrEnvOrDefault("solr.docker.imageTag", "SOLR_DOCKER_IMAGE_TAG", "${version}")
def dockerImageName = propertyOrEnvOrDefault("solr.docker.imageName", "SOLR_DOCKER_IMAGE_NAME", "${dockerImageRepo}:${dockerImageTag}")
def baseDockerImage = propertyOrEnvOrDefault("solr.docker.baseImage", "SOLR_DOCKER_BASE_IMAGE", 'openjdk:11-jre-slim')
def githubUrlOrMirror = propertyOrEnvOrDefault("solr.docker.githubUrl", "SOLR_DOCKER_GITHUB_URL", 'github.com')
-docker {
- name = dockerImageName
- files file('include')
- buildArgs(['BASE_IMAGE' : baseDockerImage, 'SOLR_PACKAGE_IMAGE' : 'apache/solr-build:local-package', 'SOLR_VERSION': "${version}", 'GITHUB_URL': githubUrlOrMirror])
+// Build directory locations
+def dockerBuildDistribution = "$buildDir/distributions"
+def imageIdFile = "$buildDir/image-id"
+
+configurations {
+ packaging {
+ canBeResolved = true
+ }
+ dockerImage {
+ canBeResolved = true
+ }
}
-tasks.docker {
- // In order to create the solr docker image, the solr package image must be created first.
- dependsOn(dockerPackage.tasks.docker)
+dependencies {
+ packaging project(path: ":solr:packaging", configuration: 'archives')
+
+ dockerImage files(imageIdFile) {
+ builtBy 'dockerBuild'
+ }
+}
+
+task dockerTar(type: Tar) {
+ group = 'Docker'
+ description = 'Package docker context to prepare for docker build'
+
+ dependsOn configurations.packaging
+ into('scripts') {
+ from file('scripts')
+ fileMode 755
+ }
+ into('releases') {
+ from configurations.packaging
+ include '*.tgz'
+ }
+ from file('Dockerfile')
+ destinationDirectory = file(dockerBuildDistribution)
+ extension 'tgz'
+ compression = Compression.GZIP
+}
+
+task dockerBuild(dependsOn: tasks.dockerTar) {
+ group = 'Docker'
+ description = 'Build Solr docker image'
+
+ // Ensure that the docker image is rebuilt on build-arg changes or changes in the docker context
+ inputs.properties([
+ baseDockerImage: baseDockerImage,
+ githubUrlOrMirror: githubUrlOrMirror,
+ version: version
+ ])
+ inputs.dir(dockerBuildDistribution)
+
+ // Docker build must run after the docker context has been created & tarred
+ mustRunAfter tasks.dockerTar
Review comment:
Correct!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene-solr] HoustonPutman merged pull request #2197: SOLR-15075: Solr docker gradle improvements
Posted by GitBox <gi...@apache.org>.
HoustonPutman merged pull request #2197:
URL: https://github.com/apache/lucene-solr/pull/2197
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene-solr] dweiss commented on a change in pull request #2197: SOLR-15075: Solr docker gradle improvements
Posted by GitBox <gi...@apache.org>.
dweiss commented on a change in pull request #2197:
URL: https://github.com/apache/lucene-solr/pull/2197#discussion_r556640697
##########
File path: solr/docker/build.gradle
##########
@@ -18,106 +18,187 @@
import com.google.common.base.Preconditions
import com.google.common.base.Strings
-apply plugin: 'base'
-apply plugin: 'com.palantir.docker'
-
-subprojects {
- apply plugin: 'base'
- apply plugin: 'com.palantir.docker'
-}
-
description = 'Solr Docker image'
-def dockerPackage = project(':solr:docker:package')
-
-dependencies {
- docker dockerPackage
-}
+apply plugin: 'base'
+// Solr Docker inputs
def dockerImageRepo = propertyOrEnvOrDefault("solr.docker.imageRepo", "SOLR_DOCKER_IMAGE_REPO", "apache/solr")
def dockerImageTag = propertyOrEnvOrDefault("solr.docker.imageTag", "SOLR_DOCKER_IMAGE_TAG", "${version}")
def dockerImageName = propertyOrEnvOrDefault("solr.docker.imageName", "SOLR_DOCKER_IMAGE_NAME", "${dockerImageRepo}:${dockerImageTag}")
def baseDockerImage = propertyOrEnvOrDefault("solr.docker.baseImage", "SOLR_DOCKER_BASE_IMAGE", 'openjdk:11-jre-slim')
def githubUrlOrMirror = propertyOrEnvOrDefault("solr.docker.githubUrl", "SOLR_DOCKER_GITHUB_URL", 'github.com')
-docker {
- name = dockerImageName
- files file('include')
- buildArgs(['BASE_IMAGE' : baseDockerImage, 'SOLR_PACKAGE_IMAGE' : 'apache/solr-build:local-package', 'SOLR_VERSION': "${version}", 'GITHUB_URL': githubUrlOrMirror])
+// Build directory locations
+def dockerBuildDistribution = "$buildDir/distributions"
+def imageIdFile = "$buildDir/image-id"
+
+configurations {
+ packaging {
+ canBeResolved = true
+ }
+ dockerImage {
+ canBeResolved = true
+ }
+}
+
+dependencies {
+ packaging project(path: ":solr:packaging", configuration: 'archives')
+
+ dockerImage files(imageIdFile) {
+ builtBy 'dockerBuild'
+ }
+}
+
+task dockerTar(type: Tar) {
+ group = 'Docker'
+ description = 'Package docker context to prepare for docker build'
+
+ dependsOn configurations.packaging
+ into('scripts') {
+ from file('scripts')
+ fileMode 755
+ }
+ into('releases') {
+ from configurations.packaging
+ include '*.tgz'
+ }
+ from file('Dockerfile')
+ destinationDirectory = file(dockerBuildDistribution)
+ extension 'tgz'
+ compression = Compression.GZIP
}
-tasks.docker {
- // In order to create the solr docker image, the solr package image must be created first.
- dependsOn(dockerPackage.tasks.docker)
+task dockerBuild(dependsOn: tasks.dockerTar) {
+ group = 'Docker'
+ description = 'Build Solr docker image'
+
+ // Ensure that the docker image is rebuilt on build-arg changes or changes in the docker context
+ inputs.properties([
+ baseDockerImage: baseDockerImage,
+ githubUrlOrMirror: githubUrlOrMirror,
+ version: version
+ ])
+ inputs.dir(dockerBuildDistribution)
+
+ doLast {
+ exec {
+ standardInput = tasks.dockerTar.outputs.files.singleFile.newDataInputStream()
+ commandLine "docker", "build",
+ "--iidfile", imageIdFile,
+ "--build-arg", "BASE_IMAGE=${inputs.properties.baseDockerImage}",
+ "--build-arg", "SOLR_VERSION=${version}",
+ "--build-arg", "GITHUB_URL=${inputs.properties.githubUrlOrMirror}",
+ "-"
+ }
+ }
// Print information on the image after it has been created
doLast {
+ def dockerImageId = file(imageIdFile).text
project.logger.lifecycle("Solr Docker Image Created")
- project.logger.lifecycle("\tName: $dockerImageName")
- project.logger.lifecycle("\tBase Image: $baseDockerImage")
+ project.logger.lifecycle("\tID: \t$dockerImageId")
+ project.logger.lifecycle("\tBase Image: \t$baseDockerImage")
+ project.logger.lifecycle("\tSolr Version: \t$version")
}
+
+ outputs.files(imageIdFile)
}
-abstract class DockerTestSuite extends DefaultTask {
- private String solrImageName = null;
- private List<String> tests = new ArrayList<>();
- private List<String> ignore = new ArrayList<>();
+task dockerTag(dependsOn: tasks.dockerBuild) {
+ group = 'Docker'
+ description = 'Tag Solr docker image'
- @OutputDirectory
- abstract DirectoryProperty getOutputDir()
+ // Ensure that the docker image is re-tagged if the image ID or desired tag changes
+ inputs.properties([
+ dockerImageName: dockerImageName,
+ ])
+ inputs.file(imageIdFile)
- public void setSolrImageName(String solrImageName) {
- this.solrImageName = solrImageName
+ doLast {
+ exec {
+ commandLine "docker", "tag", tasks.dockerBuild.outputs.files.singleFile.text, inputs.properties.dockerImageName
+ }
}
- public String getSolrImageName() {
- Preconditions.checkArgument(!Strings.isNullOrEmpty(solrImageName), "solrImageName is a required dockerTests configuration item.")
- return solrImageName
+ // Print information on the image after it has been created
+ doLast {
+ def dockerImageId = tasks.dockerBuild.outputs.files.singleFile.text
+ project.logger.lifecycle("Solr Docker Image Tagged")
+ project.logger.lifecycle("\tID: \t$dockerImageId")
+ project.logger.lifecycle("\tTag: \t$dockerImageName")
}
+}
- @Option(option = "tests", description = "Only run these specified tests, comma separated.")
- public void setTests(List<String> tests) {
- this.tests = tests;
- }
+task dockerPush(dependsOn: tasks.dockerTag) {
+ group = 'Docker'
+ description = 'Push Solr docker image'
- @Input
- public List<String> getTests() {
- return tests;
- }
+ // Ensure that the docker image is re-pushed if the image ID or tag changes
+ inputs.properties([
+ dockerImageName: dockerImageName,
+ ])
+ inputs.file(imageIdFile)
- @Option(option = "ignore", description = "Ignore these tests, comma separated.")
- public void setIgnore(List<String> ignore) {
- this.ignore = ignore;
+ doLast {
+ exec {
+ commandLine "docker", "push", dockerImageName
+ }
}
- @Input
- public List<String> getIgnore() {
- return ignore;
+ // Print information on the image after it has been created
+ doLast {
+ project.logger.lifecycle("Solr Docker Image Pushed: \t$dockerImageName")
}
+}
+
+
+task testDocker(dependsOn: tasks.dockerBuild) {
+ group = 'Docker'
+ description = 'Test Solr docker image'
+
+ def inputDir = "tests/cases"
+ def outputDir = "$buildDir/tmp/tests"
+
+ // Ensure that the docker image is re-tested if the image ID changes or the test files change
+ inputs.properties([
Review comment:
Thanks. I thought they'd require a serializable type. It's a pity they don't just say ? extends Serializable in the signature.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene-solr] HoustonPutman commented on a change in pull request #2197: SOLR-15075: Solr docker gradle improvements
Posted by GitBox <gi...@apache.org>.
HoustonPutman commented on a change in pull request #2197:
URL: https://github.com/apache/lucene-solr/pull/2197#discussion_r560269157
##########
File path: solr/docker/build.gradle
##########
@@ -18,106 +18,192 @@
import com.google.common.base.Preconditions
import com.google.common.base.Strings
-apply plugin: 'base'
-apply plugin: 'com.palantir.docker'
-
-subprojects {
- apply plugin: 'base'
- apply plugin: 'com.palantir.docker'
-}
-
description = 'Solr Docker image'
-def dockerPackage = project(':solr:docker:package')
-
-dependencies {
- docker dockerPackage
-}
+apply plugin: 'base'
+// Solr Docker inputs
def dockerImageRepo = propertyOrEnvOrDefault("solr.docker.imageRepo", "SOLR_DOCKER_IMAGE_REPO", "apache/solr")
def dockerImageTag = propertyOrEnvOrDefault("solr.docker.imageTag", "SOLR_DOCKER_IMAGE_TAG", "${version}")
def dockerImageName = propertyOrEnvOrDefault("solr.docker.imageName", "SOLR_DOCKER_IMAGE_NAME", "${dockerImageRepo}:${dockerImageTag}")
def baseDockerImage = propertyOrEnvOrDefault("solr.docker.baseImage", "SOLR_DOCKER_BASE_IMAGE", 'openjdk:11-jre-slim')
def githubUrlOrMirror = propertyOrEnvOrDefault("solr.docker.githubUrl", "SOLR_DOCKER_GITHUB_URL", 'github.com')
-docker {
- name = dockerImageName
- files file('include')
- buildArgs(['BASE_IMAGE' : baseDockerImage, 'SOLR_PACKAGE_IMAGE' : 'apache/solr-build:local-package', 'SOLR_VERSION': "${version}", 'GITHUB_URL': githubUrlOrMirror])
+// Build directory locations
+def dockerBuildDistribution = "$buildDir/distributions"
+def imageIdFile = "$buildDir/image-id"
+
+configurations {
+ packaging {
+ canBeResolved = true
+ }
+ dockerImage {
+ canBeResolved = true
+ }
+}
+
+dependencies {
+ packaging project(path: ":solr:packaging", configuration: 'archives')
+
+ dockerImage files(imageIdFile) {
+ builtBy 'dockerBuild'
+ }
}
-tasks.docker {
- // In order to create the solr docker image, the solr package image must be created first.
- dependsOn(dockerPackage.tasks.docker)
+task dockerTar(type: Tar) {
Review comment:
Yeah it'd be nice if the build via TGZ were a bit more flexible.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene-solr] HoustonPutman commented on a change in pull request #2197: SOLR-15075: Solr docker gradle improvements
Posted by GitBox <gi...@apache.org>.
HoustonPutman commented on a change in pull request #2197:
URL: https://github.com/apache/lucene-solr/pull/2197#discussion_r557685791
##########
File path: solr/docker/build.gradle
##########
@@ -18,106 +18,203 @@
import com.google.common.base.Preconditions
import com.google.common.base.Strings
-apply plugin: 'base'
-apply plugin: 'com.palantir.docker'
-
-subprojects {
- apply plugin: 'base'
- apply plugin: 'com.palantir.docker'
-}
-
description = 'Solr Docker image'
-def dockerPackage = project(':solr:docker:package')
-
-dependencies {
- docker dockerPackage
-}
+apply plugin: 'base'
+// Solr Docker inputs
def dockerImageRepo = propertyOrEnvOrDefault("solr.docker.imageRepo", "SOLR_DOCKER_IMAGE_REPO", "apache/solr")
def dockerImageTag = propertyOrEnvOrDefault("solr.docker.imageTag", "SOLR_DOCKER_IMAGE_TAG", "${version}")
def dockerImageName = propertyOrEnvOrDefault("solr.docker.imageName", "SOLR_DOCKER_IMAGE_NAME", "${dockerImageRepo}:${dockerImageTag}")
def baseDockerImage = propertyOrEnvOrDefault("solr.docker.baseImage", "SOLR_DOCKER_BASE_IMAGE", 'openjdk:11-jre-slim')
def githubUrlOrMirror = propertyOrEnvOrDefault("solr.docker.githubUrl", "SOLR_DOCKER_GITHUB_URL", 'github.com')
-docker {
- name = dockerImageName
- files file('include')
- buildArgs(['BASE_IMAGE' : baseDockerImage, 'SOLR_PACKAGE_IMAGE' : 'apache/solr-build:local-package', 'SOLR_VERSION': "${version}", 'GITHUB_URL': githubUrlOrMirror])
+// Build directory locations
+def dockerBuildDistribution = "$buildDir/distributions"
+def imageIdFile = "$buildDir/image-id"
+
+configurations {
+ packaging {
+ canBeResolved = true
+ }
+ dockerImage {
+ canBeResolved = true
+ }
}
-tasks.docker {
- // In order to create the solr docker image, the solr package image must be created first.
- dependsOn(dockerPackage.tasks.docker)
+dependencies {
+ packaging project(path: ":solr:packaging", configuration: 'archives')
+
+ dockerImage files(imageIdFile) {
+ builtBy 'dockerBuild'
+ }
+}
+
+task dockerTar(type: Tar) {
+ group = 'Docker'
+ description = 'Package docker context to prepare for docker build'
+
+ dependsOn configurations.packaging
+ into('scripts') {
+ from file('scripts')
+ fileMode 755
+ }
+ into('releases') {
+ from configurations.packaging
+ include '*.tgz'
+ }
+ from file('Dockerfile')
+ destinationDirectory = file(dockerBuildDistribution)
+ extension 'tgz'
+ compression = Compression.GZIP
+}
+
+task dockerBuild(dependsOn: tasks.dockerTar) {
+ group = 'Docker'
+ description = 'Build Solr docker image'
+
+ // Ensure that the docker image is rebuilt on build-arg changes or changes in the docker context
+ inputs.properties([
+ baseDockerImage: baseDockerImage,
+ githubUrlOrMirror: githubUrlOrMirror,
+ version: version
+ ])
+ inputs.dir(dockerBuildDistribution)
+
+ // Docker build must run after the docker context has been created & tarred
+ mustRunAfter tasks.dockerTar
Review comment:
Ahh ok, that's good to know.
I have one instance of it left, `dockerPush.mustRunAfter dockerTests` so that if the tests fail, the push doesn't happen. It's not a dependency because I don't want to enforce running the tests, in case a user doesn't want to.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene-solr] HoustonPutman commented on a change in pull request #2197: SOLR-15075: Solr docker gradle improvements
Posted by GitBox <gi...@apache.org>.
HoustonPutman commented on a change in pull request #2197:
URL: https://github.com/apache/lucene-solr/pull/2197#discussion_r556931132
##########
File path: solr/docker/build.gradle
##########
@@ -18,106 +18,203 @@
import com.google.common.base.Preconditions
import com.google.common.base.Strings
-apply plugin: 'base'
-apply plugin: 'com.palantir.docker'
-
-subprojects {
- apply plugin: 'base'
- apply plugin: 'com.palantir.docker'
-}
-
description = 'Solr Docker image'
-def dockerPackage = project(':solr:docker:package')
-
-dependencies {
- docker dockerPackage
-}
+apply plugin: 'base'
+// Solr Docker inputs
def dockerImageRepo = propertyOrEnvOrDefault("solr.docker.imageRepo", "SOLR_DOCKER_IMAGE_REPO", "apache/solr")
def dockerImageTag = propertyOrEnvOrDefault("solr.docker.imageTag", "SOLR_DOCKER_IMAGE_TAG", "${version}")
def dockerImageName = propertyOrEnvOrDefault("solr.docker.imageName", "SOLR_DOCKER_IMAGE_NAME", "${dockerImageRepo}:${dockerImageTag}")
def baseDockerImage = propertyOrEnvOrDefault("solr.docker.baseImage", "SOLR_DOCKER_BASE_IMAGE", 'openjdk:11-jre-slim')
def githubUrlOrMirror = propertyOrEnvOrDefault("solr.docker.githubUrl", "SOLR_DOCKER_GITHUB_URL", 'github.com')
-docker {
- name = dockerImageName
- files file('include')
- buildArgs(['BASE_IMAGE' : baseDockerImage, 'SOLR_PACKAGE_IMAGE' : 'apache/solr-build:local-package', 'SOLR_VERSION': "${version}", 'GITHUB_URL': githubUrlOrMirror])
+// Build directory locations
+def dockerBuildDistribution = "$buildDir/distributions"
+def imageIdFile = "$buildDir/image-id"
+
+configurations {
+ packaging {
+ canBeResolved = true
+ }
+ dockerImage {
+ canBeResolved = true
+ }
}
-tasks.docker {
- // In order to create the solr docker image, the solr package image must be created first.
- dependsOn(dockerPackage.tasks.docker)
+dependencies {
+ packaging project(path: ":solr:packaging", configuration: 'archives')
+
+ dockerImage files(imageIdFile) {
+ builtBy 'dockerBuild'
+ }
+}
+
+task dockerTar(type: Tar) {
+ group = 'Docker'
+ description = 'Package docker context to prepare for docker build'
+
+ dependsOn configurations.packaging
+ into('scripts') {
+ from file('scripts')
+ fileMode 755
+ }
+ into('releases') {
+ from configurations.packaging
+ include '*.tgz'
+ }
+ from file('Dockerfile')
+ destinationDirectory = file(dockerBuildDistribution)
+ extension 'tgz'
+ compression = Compression.GZIP
+}
+
+task dockerBuild(dependsOn: tasks.dockerTar) {
+ group = 'Docker'
+ description = 'Build Solr docker image'
+
+ // Ensure that the docker image is rebuilt on build-arg changes or changes in the docker context
+ inputs.properties([
+ baseDockerImage: baseDockerImage,
+ githubUrlOrMirror: githubUrlOrMirror,
+ version: version
+ ])
+ inputs.dir(dockerBuildDistribution)
+
+ // Docker build must run after the docker context has been created & tarred
+ mustRunAfter tasks.dockerTar
Review comment:
It's not neccessary right now. I mainly put it in because I'm contemplating removing the dependency at some point (I actually did add it, but then removed it without removing the `mustRunAfter` lines.
The issue now is that if you run:
```
gradlew dockerBuild -Psolr.docker.baseImage=something-else:latest
gradlew dockerTag -P solr.docker.imageName=custom-name:latest
```
In the above example the first command will build the dockerImage with the custom baseImage. However the second command doesn't have the `baseImage` specified, so in the dependent `dockerBuild` task it will build the docker image with the default baseImage, overwriting the image created in the first command.
```
gradlew dockerBuild dockerTag -Psolr.docker.baseImage=something-else:latest -P solr.docker.imageName=custom-name:latest
```
This is the command necessary to build & tag with a custom baseImage.
I think it might be nice to make `testDocker` and `dockerTag` not dependent on `dockerBuild`, but instead merely check to make sure that `dockerBuild` has already been run. (The resulting image-id should be stored in a file in the build directory). That way the two commands above work exactly the same.
I'm not convinced either way yet, just something I've been thinking about.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene-solr] dweiss commented on a change in pull request #2197: SOLR-15075: Solr docker gradle improvements
Posted by GitBox <gi...@apache.org>.
dweiss commented on a change in pull request #2197:
URL: https://github.com/apache/lucene-solr/pull/2197#discussion_r556429043
##########
File path: solr/docker/build.gradle
##########
@@ -18,106 +18,187 @@
import com.google.common.base.Preconditions
import com.google.common.base.Strings
-apply plugin: 'base'
-apply plugin: 'com.palantir.docker'
-
-subprojects {
- apply plugin: 'base'
- apply plugin: 'com.palantir.docker'
-}
-
description = 'Solr Docker image'
-def dockerPackage = project(':solr:docker:package')
-
-dependencies {
- docker dockerPackage
-}
+apply plugin: 'base'
+// Solr Docker inputs
def dockerImageRepo = propertyOrEnvOrDefault("solr.docker.imageRepo", "SOLR_DOCKER_IMAGE_REPO", "apache/solr")
def dockerImageTag = propertyOrEnvOrDefault("solr.docker.imageTag", "SOLR_DOCKER_IMAGE_TAG", "${version}")
def dockerImageName = propertyOrEnvOrDefault("solr.docker.imageName", "SOLR_DOCKER_IMAGE_NAME", "${dockerImageRepo}:${dockerImageTag}")
def baseDockerImage = propertyOrEnvOrDefault("solr.docker.baseImage", "SOLR_DOCKER_BASE_IMAGE", 'openjdk:11-jre-slim')
def githubUrlOrMirror = propertyOrEnvOrDefault("solr.docker.githubUrl", "SOLR_DOCKER_GITHUB_URL", 'github.com')
-docker {
- name = dockerImageName
- files file('include')
- buildArgs(['BASE_IMAGE' : baseDockerImage, 'SOLR_PACKAGE_IMAGE' : 'apache/solr-build:local-package', 'SOLR_VERSION': "${version}", 'GITHUB_URL': githubUrlOrMirror])
+// Build directory locations
+def dockerBuildDistribution = "$buildDir/distributions"
+def imageIdFile = "$buildDir/image-id"
+
+configurations {
+ packaging {
+ canBeResolved = true
+ }
+ dockerImage {
+ canBeResolved = true
+ }
+}
+
+dependencies {
+ packaging project(path: ":solr:packaging", configuration: 'archives')
+
+ dockerImage files(imageIdFile) {
+ builtBy 'dockerBuild'
+ }
+}
+
+task dockerTar(type: Tar) {
+ group = 'Docker'
+ description = 'Package docker context to prepare for docker build'
+
+ dependsOn configurations.packaging
+ into('scripts') {
+ from file('scripts')
+ fileMode 755
+ }
+ into('releases') {
+ from configurations.packaging
+ include '*.tgz'
+ }
+ from file('Dockerfile')
+ destinationDirectory = file(dockerBuildDistribution)
+ extension 'tgz'
+ compression = Compression.GZIP
}
-tasks.docker {
- // In order to create the solr docker image, the solr package image must be created first.
- dependsOn(dockerPackage.tasks.docker)
+task dockerBuild(dependsOn: tasks.dockerTar) {
+ group = 'Docker'
+ description = 'Build Solr docker image'
+
+ // Ensure that the docker image is rebuilt on build-arg changes or changes in the docker context
+ inputs.properties([
+ baseDockerImage: baseDockerImage,
+ githubUrlOrMirror: githubUrlOrMirror,
+ version: version
+ ])
+ inputs.dir(dockerBuildDistribution)
+
+ doLast {
+ exec {
+ standardInput = tasks.dockerTar.outputs.files.singleFile.newDataInputStream()
+ commandLine "docker", "build",
+ "--iidfile", imageIdFile,
+ "--build-arg", "BASE_IMAGE=${inputs.properties.baseDockerImage}",
+ "--build-arg", "SOLR_VERSION=${version}",
+ "--build-arg", "GITHUB_URL=${inputs.properties.githubUrlOrMirror}",
+ "-"
+ }
+ }
// Print information on the image after it has been created
doLast {
+ def dockerImageId = file(imageIdFile).text
project.logger.lifecycle("Solr Docker Image Created")
- project.logger.lifecycle("\tName: $dockerImageName")
- project.logger.lifecycle("\tBase Image: $baseDockerImage")
+ project.logger.lifecycle("\tID: \t$dockerImageId")
+ project.logger.lifecycle("\tBase Image: \t$baseDockerImage")
+ project.logger.lifecycle("\tSolr Version: \t$version")
}
+
+ outputs.files(imageIdFile)
}
-abstract class DockerTestSuite extends DefaultTask {
- private String solrImageName = null;
- private List<String> tests = new ArrayList<>();
- private List<String> ignore = new ArrayList<>();
+task dockerTag(dependsOn: tasks.dockerBuild) {
+ group = 'Docker'
+ description = 'Tag Solr docker image'
- @OutputDirectory
- abstract DirectoryProperty getOutputDir()
+ // Ensure that the docker image is re-tagged if the image ID or desired tag changes
+ inputs.properties([
+ dockerImageName: dockerImageName,
+ ])
+ inputs.file(imageIdFile)
- public void setSolrImageName(String solrImageName) {
- this.solrImageName = solrImageName
+ doLast {
+ exec {
+ commandLine "docker", "tag", tasks.dockerBuild.outputs.files.singleFile.text, inputs.properties.dockerImageName
+ }
}
- public String getSolrImageName() {
- Preconditions.checkArgument(!Strings.isNullOrEmpty(solrImageName), "solrImageName is a required dockerTests configuration item.")
- return solrImageName
+ // Print information on the image after it has been created
+ doLast {
+ def dockerImageId = tasks.dockerBuild.outputs.files.singleFile.text
+ project.logger.lifecycle("Solr Docker Image Tagged")
+ project.logger.lifecycle("\tID: \t$dockerImageId")
+ project.logger.lifecycle("\tTag: \t$dockerImageName")
}
+}
- @Option(option = "tests", description = "Only run these specified tests, comma separated.")
- public void setTests(List<String> tests) {
- this.tests = tests;
- }
+task dockerPush(dependsOn: tasks.dockerTag) {
+ group = 'Docker'
+ description = 'Push Solr docker image'
- @Input
- public List<String> getTests() {
- return tests;
- }
+ // Ensure that the docker image is re-pushed if the image ID or tag changes
+ inputs.properties([
+ dockerImageName: dockerImageName,
+ ])
+ inputs.file(imageIdFile)
- @Option(option = "ignore", description = "Ignore these tests, comma separated.")
- public void setIgnore(List<String> ignore) {
- this.ignore = ignore;
+ doLast {
+ exec {
+ commandLine "docker", "push", dockerImageName
+ }
}
- @Input
- public List<String> getIgnore() {
- return ignore;
+ // Print information on the image after it has been created
+ doLast {
+ project.logger.lifecycle("Solr Docker Image Pushed: \t$dockerImageName")
}
+}
+
+
+task testDocker(dependsOn: tasks.dockerBuild) {
+ group = 'Docker'
+ description = 'Test Solr docker image'
+
+ def inputDir = "tests/cases"
+ def outputDir = "$buildDir/tmp/tests"
+
+ // Ensure that the docker image is re-tested if the image ID changes or the test files change
+ inputs.properties([
Review comment:
Does this work with complex types (if they're serializable)? I thought they need to be properties (string key-value pairs).
##########
File path: gradle/help.gradle
##########
@@ -30,7 +30,7 @@ configure(rootProject) {
["Git", "help/git.txt", "Git assistance and guides."],
["ValidateLogCalls", "help/validateLogCalls.txt", "How to use logging calls efficiently."],
["IDEs", "help/IDEs.txt", "IDE support."],
- ["Docker", "help/docker.txt", "Building Solr Docker images."],
Review comment:
Do you think it makes sense to scatter those help files? I see some benefits of keeping them in one place.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene-solr] dweiss commented on a change in pull request #2197: SOLR-15075: Solr docker gradle improvements
Posted by GitBox <gi...@apache.org>.
dweiss commented on a change in pull request #2197:
URL: https://github.com/apache/lucene-solr/pull/2197#discussion_r557669428
##########
File path: solr/docker/build.gradle
##########
@@ -18,106 +18,203 @@
import com.google.common.base.Preconditions
import com.google.common.base.Strings
-apply plugin: 'base'
-apply plugin: 'com.palantir.docker'
-
-subprojects {
- apply plugin: 'base'
- apply plugin: 'com.palantir.docker'
-}
-
description = 'Solr Docker image'
-def dockerPackage = project(':solr:docker:package')
-
-dependencies {
- docker dockerPackage
-}
+apply plugin: 'base'
+// Solr Docker inputs
def dockerImageRepo = propertyOrEnvOrDefault("solr.docker.imageRepo", "SOLR_DOCKER_IMAGE_REPO", "apache/solr")
def dockerImageTag = propertyOrEnvOrDefault("solr.docker.imageTag", "SOLR_DOCKER_IMAGE_TAG", "${version}")
def dockerImageName = propertyOrEnvOrDefault("solr.docker.imageName", "SOLR_DOCKER_IMAGE_NAME", "${dockerImageRepo}:${dockerImageTag}")
def baseDockerImage = propertyOrEnvOrDefault("solr.docker.baseImage", "SOLR_DOCKER_BASE_IMAGE", 'openjdk:11-jre-slim')
def githubUrlOrMirror = propertyOrEnvOrDefault("solr.docker.githubUrl", "SOLR_DOCKER_GITHUB_URL", 'github.com')
-docker {
- name = dockerImageName
- files file('include')
- buildArgs(['BASE_IMAGE' : baseDockerImage, 'SOLR_PACKAGE_IMAGE' : 'apache/solr-build:local-package', 'SOLR_VERSION': "${version}", 'GITHUB_URL': githubUrlOrMirror])
+// Build directory locations
+def dockerBuildDistribution = "$buildDir/distributions"
+def imageIdFile = "$buildDir/image-id"
+
+configurations {
+ packaging {
+ canBeResolved = true
+ }
+ dockerImage {
+ canBeResolved = true
+ }
}
-tasks.docker {
- // In order to create the solr docker image, the solr package image must be created first.
- dependsOn(dockerPackage.tasks.docker)
+dependencies {
+ packaging project(path: ":solr:packaging", configuration: 'archives')
+
+ dockerImage files(imageIdFile) {
+ builtBy 'dockerBuild'
+ }
+}
+
+task dockerTar(type: Tar) {
+ group = 'Docker'
+ description = 'Package docker context to prepare for docker build'
+
+ dependsOn configurations.packaging
+ into('scripts') {
+ from file('scripts')
+ fileMode 755
+ }
+ into('releases') {
+ from configurations.packaging
+ include '*.tgz'
+ }
+ from file('Dockerfile')
+ destinationDirectory = file(dockerBuildDistribution)
+ extension 'tgz'
+ compression = Compression.GZIP
+}
+
+task dockerBuild(dependsOn: tasks.dockerTar) {
+ group = 'Docker'
+ description = 'Build Solr docker image'
+
+ // Ensure that the docker image is rebuilt on build-arg changes or changes in the docker context
+ inputs.properties([
+ baseDockerImage: baseDockerImage,
+ githubUrlOrMirror: githubUrlOrMirror,
+ version: version
+ ])
+ inputs.dir(dockerBuildDistribution)
+
+ // Docker build must run after the docker context has been created & tarred
+ mustRunAfter tasks.dockerTar
Review comment:
That gradle code shows you're familiar enough! mustRunAfter is a rare case, really.
I sporadically use it to order certain tasks if they're queued together in (possibly) parallel. An example: schedule integration tests to run after unit tests have passed. Then you can run both separately (no dependency on each other) but if you run them together (as part of check, which depends on both integration and unit tests), unit tests will be executed prior to integration tests.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene-solr] dsmiley commented on a change in pull request #2197: SOLR-15075: Solr docker gradle improvements
Posted by GitBox <gi...@apache.org>.
dsmiley commented on a change in pull request #2197:
URL: https://github.com/apache/lucene-solr/pull/2197#discussion_r559089095
##########
File path: solr/docker/build.gradle
##########
@@ -18,106 +18,192 @@
import com.google.common.base.Preconditions
import com.google.common.base.Strings
-apply plugin: 'base'
-apply plugin: 'com.palantir.docker'
-
-subprojects {
- apply plugin: 'base'
- apply plugin: 'com.palantir.docker'
-}
-
description = 'Solr Docker image'
-def dockerPackage = project(':solr:docker:package')
-
-dependencies {
- docker dockerPackage
-}
+apply plugin: 'base'
+// Solr Docker inputs
def dockerImageRepo = propertyOrEnvOrDefault("solr.docker.imageRepo", "SOLR_DOCKER_IMAGE_REPO", "apache/solr")
def dockerImageTag = propertyOrEnvOrDefault("solr.docker.imageTag", "SOLR_DOCKER_IMAGE_TAG", "${version}")
def dockerImageName = propertyOrEnvOrDefault("solr.docker.imageName", "SOLR_DOCKER_IMAGE_NAME", "${dockerImageRepo}:${dockerImageTag}")
def baseDockerImage = propertyOrEnvOrDefault("solr.docker.baseImage", "SOLR_DOCKER_BASE_IMAGE", 'openjdk:11-jre-slim')
def githubUrlOrMirror = propertyOrEnvOrDefault("solr.docker.githubUrl", "SOLR_DOCKER_GITHUB_URL", 'github.com')
-docker {
- name = dockerImageName
- files file('include')
- buildArgs(['BASE_IMAGE' : baseDockerImage, 'SOLR_PACKAGE_IMAGE' : 'apache/solr-build:local-package', 'SOLR_VERSION': "${version}", 'GITHUB_URL': githubUrlOrMirror])
+// Build directory locations
+def dockerBuildDistribution = "$buildDir/distributions"
+def imageIdFile = "$buildDir/image-id"
+
+configurations {
+ packaging {
+ canBeResolved = true
+ }
+ dockerImage {
+ canBeResolved = true
+ }
+}
+
+dependencies {
+ packaging project(path: ":solr:packaging", configuration: 'archives')
+
+ dockerImage files(imageIdFile) {
+ builtBy 'dockerBuild'
+ }
}
-tasks.docker {
- // In order to create the solr docker image, the solr package image must be created first.
- dependsOn(dockerPackage.tasks.docker)
+task dockerTar(type: Tar) {
Review comment:
I bet this task could be skipped entirely in lieu of the packaging tgz directly being provided as stdin to Docker. If it doesn't contain these scripts, we could arrange for that easily enough. I'm toying with the Dockerfile right now that could operate on this assumption. It's eliminate a tar.gz generation and decompression, and consequently overall simpler.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene-solr] HoustonPutman commented on a change in pull request #2197: SOLR-15075: Solr docker gradle improvements
Posted by GitBox <gi...@apache.org>.
HoustonPutman commented on a change in pull request #2197:
URL: https://github.com/apache/lucene-solr/pull/2197#discussion_r564598544
##########
File path: solr/docker/build.gradle
##########
@@ -18,106 +18,192 @@
import com.google.common.base.Preconditions
import com.google.common.base.Strings
-apply plugin: 'base'
-apply plugin: 'com.palantir.docker'
-
-subprojects {
- apply plugin: 'base'
- apply plugin: 'com.palantir.docker'
-}
-
description = 'Solr Docker image'
-def dockerPackage = project(':solr:docker:package')
-
-dependencies {
- docker dockerPackage
-}
+apply plugin: 'base'
+// Solr Docker inputs
def dockerImageRepo = propertyOrEnvOrDefault("solr.docker.imageRepo", "SOLR_DOCKER_IMAGE_REPO", "apache/solr")
def dockerImageTag = propertyOrEnvOrDefault("solr.docker.imageTag", "SOLR_DOCKER_IMAGE_TAG", "${version}")
def dockerImageName = propertyOrEnvOrDefault("solr.docker.imageName", "SOLR_DOCKER_IMAGE_NAME", "${dockerImageRepo}:${dockerImageTag}")
def baseDockerImage = propertyOrEnvOrDefault("solr.docker.baseImage", "SOLR_DOCKER_BASE_IMAGE", 'openjdk:11-jre-slim')
def githubUrlOrMirror = propertyOrEnvOrDefault("solr.docker.githubUrl", "SOLR_DOCKER_GITHUB_URL", 'github.com')
-docker {
- name = dockerImageName
- files file('include')
- buildArgs(['BASE_IMAGE' : baseDockerImage, 'SOLR_PACKAGE_IMAGE' : 'apache/solr-build:local-package', 'SOLR_VERSION': "${version}", 'GITHUB_URL': githubUrlOrMirror])
+// Build directory locations
+def dockerBuildDistribution = "$buildDir/distributions"
+def imageIdFile = "$buildDir/image-id"
+
+configurations {
+ packaging {
+ canBeResolved = true
+ }
+ dockerImage {
+ canBeResolved = true
+ }
+}
+
+dependencies {
+ packaging project(path: ":solr:packaging", configuration: 'archives')
+
+ dockerImage files(imageIdFile) {
+ builtBy 'dockerBuild'
+ }
}
-tasks.docker {
- // In order to create the solr docker image, the solr package image must be created first.
- dependsOn(dockerPackage.tasks.docker)
+task dockerTar(type: Tar) {
Review comment:
@dsmiley I still think that your idea is something that we should explore. I think that this PR (that I just merged) is just a step in the right direction. But we might as well have something that is working much better now, and add final touches later.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene-solr] HoustonPutman commented on a change in pull request #2197: SOLR-15075: Solr docker gradle improvements
Posted by GitBox <gi...@apache.org>.
HoustonPutman commented on a change in pull request #2197:
URL: https://github.com/apache/lucene-solr/pull/2197#discussion_r560536332
##########
File path: solr/docker/build.gradle
##########
@@ -18,106 +18,192 @@
import com.google.common.base.Preconditions
import com.google.common.base.Strings
-apply plugin: 'base'
-apply plugin: 'com.palantir.docker'
-
-subprojects {
- apply plugin: 'base'
- apply plugin: 'com.palantir.docker'
-}
-
description = 'Solr Docker image'
-def dockerPackage = project(':solr:docker:package')
-
-dependencies {
- docker dockerPackage
-}
+apply plugin: 'base'
+// Solr Docker inputs
def dockerImageRepo = propertyOrEnvOrDefault("solr.docker.imageRepo", "SOLR_DOCKER_IMAGE_REPO", "apache/solr")
def dockerImageTag = propertyOrEnvOrDefault("solr.docker.imageTag", "SOLR_DOCKER_IMAGE_TAG", "${version}")
def dockerImageName = propertyOrEnvOrDefault("solr.docker.imageName", "SOLR_DOCKER_IMAGE_NAME", "${dockerImageRepo}:${dockerImageTag}")
def baseDockerImage = propertyOrEnvOrDefault("solr.docker.baseImage", "SOLR_DOCKER_BASE_IMAGE", 'openjdk:11-jre-slim')
def githubUrlOrMirror = propertyOrEnvOrDefault("solr.docker.githubUrl", "SOLR_DOCKER_GITHUB_URL", 'github.com')
-docker {
- name = dockerImageName
- files file('include')
- buildArgs(['BASE_IMAGE' : baseDockerImage, 'SOLR_PACKAGE_IMAGE' : 'apache/solr-build:local-package', 'SOLR_VERSION': "${version}", 'GITHUB_URL': githubUrlOrMirror])
+// Build directory locations
+def dockerBuildDistribution = "$buildDir/distributions"
+def imageIdFile = "$buildDir/image-id"
+
+configurations {
+ packaging {
+ canBeResolved = true
+ }
+ dockerImage {
+ canBeResolved = true
+ }
+}
+
+dependencies {
+ packaging project(path: ":solr:packaging", configuration: 'archives')
+
+ dockerImage files(imageIdFile) {
+ builtBy 'dockerBuild'
+ }
}
-tasks.docker {
- // In order to create the solr docker image, the solr package image must be created first.
- dependsOn(dockerPackage.tasks.docker)
+task dockerTar(type: Tar) {
Review comment:
I don't think it's a bad idea. Possibly an effort to push for separately. Might be an issue if we start auto-loading first-party-packages in the docker image in the future. I imagine that those wouldn't be included in the solr tgz, but I might be wrong about that.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene-solr] HoustonPutman commented on a change in pull request #2197: SOLR-15075: Solr docker gradle improvements
Posted by GitBox <gi...@apache.org>.
HoustonPutman commented on a change in pull request #2197:
URL: https://github.com/apache/lucene-solr/pull/2197#discussion_r556639260
##########
File path: solr/docker/build.gradle
##########
@@ -18,106 +18,187 @@
import com.google.common.base.Preconditions
import com.google.common.base.Strings
-apply plugin: 'base'
-apply plugin: 'com.palantir.docker'
-
-subprojects {
- apply plugin: 'base'
- apply plugin: 'com.palantir.docker'
-}
-
description = 'Solr Docker image'
-def dockerPackage = project(':solr:docker:package')
-
-dependencies {
- docker dockerPackage
-}
+apply plugin: 'base'
+// Solr Docker inputs
def dockerImageRepo = propertyOrEnvOrDefault("solr.docker.imageRepo", "SOLR_DOCKER_IMAGE_REPO", "apache/solr")
def dockerImageTag = propertyOrEnvOrDefault("solr.docker.imageTag", "SOLR_DOCKER_IMAGE_TAG", "${version}")
def dockerImageName = propertyOrEnvOrDefault("solr.docker.imageName", "SOLR_DOCKER_IMAGE_NAME", "${dockerImageRepo}:${dockerImageTag}")
def baseDockerImage = propertyOrEnvOrDefault("solr.docker.baseImage", "SOLR_DOCKER_BASE_IMAGE", 'openjdk:11-jre-slim')
def githubUrlOrMirror = propertyOrEnvOrDefault("solr.docker.githubUrl", "SOLR_DOCKER_GITHUB_URL", 'github.com')
-docker {
- name = dockerImageName
- files file('include')
- buildArgs(['BASE_IMAGE' : baseDockerImage, 'SOLR_PACKAGE_IMAGE' : 'apache/solr-build:local-package', 'SOLR_VERSION': "${version}", 'GITHUB_URL': githubUrlOrMirror])
+// Build directory locations
+def dockerBuildDistribution = "$buildDir/distributions"
+def imageIdFile = "$buildDir/image-id"
+
+configurations {
+ packaging {
+ canBeResolved = true
+ }
+ dockerImage {
+ canBeResolved = true
+ }
+}
+
+dependencies {
+ packaging project(path: ":solr:packaging", configuration: 'archives')
+
+ dockerImage files(imageIdFile) {
+ builtBy 'dockerBuild'
+ }
+}
+
+task dockerTar(type: Tar) {
+ group = 'Docker'
+ description = 'Package docker context to prepare for docker build'
+
+ dependsOn configurations.packaging
+ into('scripts') {
+ from file('scripts')
+ fileMode 755
+ }
+ into('releases') {
+ from configurations.packaging
+ include '*.tgz'
+ }
+ from file('Dockerfile')
+ destinationDirectory = file(dockerBuildDistribution)
+ extension 'tgz'
+ compression = Compression.GZIP
}
-tasks.docker {
- // In order to create the solr docker image, the solr package image must be created first.
- dependsOn(dockerPackage.tasks.docker)
+task dockerBuild(dependsOn: tasks.dockerTar) {
+ group = 'Docker'
+ description = 'Build Solr docker image'
+
+ // Ensure that the docker image is rebuilt on build-arg changes or changes in the docker context
+ inputs.properties([
+ baseDockerImage: baseDockerImage,
+ githubUrlOrMirror: githubUrlOrMirror,
+ version: version
+ ])
+ inputs.dir(dockerBuildDistribution)
+
+ doLast {
+ exec {
+ standardInput = tasks.dockerTar.outputs.files.singleFile.newDataInputStream()
+ commandLine "docker", "build",
+ "--iidfile", imageIdFile,
+ "--build-arg", "BASE_IMAGE=${inputs.properties.baseDockerImage}",
+ "--build-arg", "SOLR_VERSION=${version}",
+ "--build-arg", "GITHUB_URL=${inputs.properties.githubUrlOrMirror}",
+ "-"
+ }
+ }
// Print information on the image after it has been created
doLast {
+ def dockerImageId = file(imageIdFile).text
project.logger.lifecycle("Solr Docker Image Created")
- project.logger.lifecycle("\tName: $dockerImageName")
- project.logger.lifecycle("\tBase Image: $baseDockerImage")
+ project.logger.lifecycle("\tID: \t$dockerImageId")
+ project.logger.lifecycle("\tBase Image: \t$baseDockerImage")
+ project.logger.lifecycle("\tSolr Version: \t$version")
}
+
+ outputs.files(imageIdFile)
}
-abstract class DockerTestSuite extends DefaultTask {
- private String solrImageName = null;
- private List<String> tests = new ArrayList<>();
- private List<String> ignore = new ArrayList<>();
+task dockerTag(dependsOn: tasks.dockerBuild) {
+ group = 'Docker'
+ description = 'Tag Solr docker image'
- @OutputDirectory
- abstract DirectoryProperty getOutputDir()
+ // Ensure that the docker image is re-tagged if the image ID or desired tag changes
+ inputs.properties([
+ dockerImageName: dockerImageName,
+ ])
+ inputs.file(imageIdFile)
- public void setSolrImageName(String solrImageName) {
- this.solrImageName = solrImageName
+ doLast {
+ exec {
+ commandLine "docker", "tag", tasks.dockerBuild.outputs.files.singleFile.text, inputs.properties.dockerImageName
+ }
}
- public String getSolrImageName() {
- Preconditions.checkArgument(!Strings.isNullOrEmpty(solrImageName), "solrImageName is a required dockerTests configuration item.")
- return solrImageName
+ // Print information on the image after it has been created
+ doLast {
+ def dockerImageId = tasks.dockerBuild.outputs.files.singleFile.text
+ project.logger.lifecycle("Solr Docker Image Tagged")
+ project.logger.lifecycle("\tID: \t$dockerImageId")
+ project.logger.lifecycle("\tTag: \t$dockerImageName")
}
+}
- @Option(option = "tests", description = "Only run these specified tests, comma separated.")
- public void setTests(List<String> tests) {
- this.tests = tests;
- }
+task dockerPush(dependsOn: tasks.dockerTag) {
+ group = 'Docker'
+ description = 'Push Solr docker image'
- @Input
- public List<String> getTests() {
- return tests;
- }
+ // Ensure that the docker image is re-pushed if the image ID or tag changes
+ inputs.properties([
+ dockerImageName: dockerImageName,
+ ])
+ inputs.file(imageIdFile)
- @Option(option = "ignore", description = "Ignore these tests, comma separated.")
- public void setIgnore(List<String> ignore) {
- this.ignore = ignore;
+ doLast {
+ exec {
+ commandLine "docker", "push", dockerImageName
+ }
}
- @Input
- public List<String> getIgnore() {
- return ignore;
+ // Print information on the image after it has been created
+ doLast {
+ project.logger.lifecycle("Solr Docker Image Pushed: \t$dockerImageName")
}
+}
+
+
+task testDocker(dependsOn: tasks.dockerBuild) {
+ group = 'Docker'
+ description = 'Test Solr docker image'
+
+ def inputDir = "tests/cases"
+ def outputDir = "$buildDir/tmp/tests"
+
+ // Ensure that the docker image is re-tested if the image ID changes or the test files change
+ inputs.properties([
Review comment:
Found the [relevant docs](https://docs.gradle.org/current/javadoc/org/gradle/api/tasks/TaskInputs.html#property-java.lang.String-java.lang.Object-). I think having a set should be fine, since it is serializable and has a good `.equals()` method.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene-solr] madrob commented on a change in pull request #2197: SOLR-15075: Solr docker gradle improvements
Posted by GitBox <gi...@apache.org>.
madrob commented on a change in pull request #2197:
URL: https://github.com/apache/lucene-solr/pull/2197#discussion_r556919719
##########
File path: solr/docker/build.gradle
##########
@@ -18,106 +18,203 @@
import com.google.common.base.Preconditions
import com.google.common.base.Strings
-apply plugin: 'base'
-apply plugin: 'com.palantir.docker'
-
-subprojects {
- apply plugin: 'base'
- apply plugin: 'com.palantir.docker'
-}
-
description = 'Solr Docker image'
-def dockerPackage = project(':solr:docker:package')
-
-dependencies {
- docker dockerPackage
-}
+apply plugin: 'base'
+// Solr Docker inputs
def dockerImageRepo = propertyOrEnvOrDefault("solr.docker.imageRepo", "SOLR_DOCKER_IMAGE_REPO", "apache/solr")
def dockerImageTag = propertyOrEnvOrDefault("solr.docker.imageTag", "SOLR_DOCKER_IMAGE_TAG", "${version}")
def dockerImageName = propertyOrEnvOrDefault("solr.docker.imageName", "SOLR_DOCKER_IMAGE_NAME", "${dockerImageRepo}:${dockerImageTag}")
def baseDockerImage = propertyOrEnvOrDefault("solr.docker.baseImage", "SOLR_DOCKER_BASE_IMAGE", 'openjdk:11-jre-slim')
def githubUrlOrMirror = propertyOrEnvOrDefault("solr.docker.githubUrl", "SOLR_DOCKER_GITHUB_URL", 'github.com')
-docker {
- name = dockerImageName
- files file('include')
- buildArgs(['BASE_IMAGE' : baseDockerImage, 'SOLR_PACKAGE_IMAGE' : 'apache/solr-build:local-package', 'SOLR_VERSION': "${version}", 'GITHUB_URL': githubUrlOrMirror])
+// Build directory locations
+def dockerBuildDistribution = "$buildDir/distributions"
+def imageIdFile = "$buildDir/image-id"
+
+configurations {
+ packaging {
+ canBeResolved = true
+ }
+ dockerImage {
+ canBeResolved = true
+ }
}
-tasks.docker {
- // In order to create the solr docker image, the solr package image must be created first.
- dependsOn(dockerPackage.tasks.docker)
+dependencies {
+ packaging project(path: ":solr:packaging", configuration: 'archives')
+
+ dockerImage files(imageIdFile) {
+ builtBy 'dockerBuild'
+ }
+}
+
+task dockerTar(type: Tar) {
+ group = 'Docker'
+ description = 'Package docker context to prepare for docker build'
+
+ dependsOn configurations.packaging
+ into('scripts') {
+ from file('scripts')
+ fileMode 755
+ }
+ into('releases') {
+ from configurations.packaging
+ include '*.tgz'
+ }
+ from file('Dockerfile')
+ destinationDirectory = file(dockerBuildDistribution)
+ extension 'tgz'
+ compression = Compression.GZIP
+}
+
+task dockerBuild(dependsOn: tasks.dockerTar) {
+ group = 'Docker'
+ description = 'Build Solr docker image'
+
+ // Ensure that the docker image is rebuilt on build-arg changes or changes in the docker context
+ inputs.properties([
+ baseDockerImage: baseDockerImage,
+ githubUrlOrMirror: githubUrlOrMirror,
+ version: version
+ ])
+ inputs.dir(dockerBuildDistribution)
+
+ // Docker build must run after the docker context has been created & tarred
+ mustRunAfter tasks.dockerTar
Review comment:
is this redundant with the dependsOn declaration above? I'm surely missing some subtlety.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene-solr] dweiss commented on a change in pull request #2197: SOLR-15075: Solr docker gradle improvements
Posted by GitBox <gi...@apache.org>.
dweiss commented on a change in pull request #2197:
URL: https://github.com/apache/lucene-solr/pull/2197#discussion_r556633807
##########
File path: gradle/help.gradle
##########
@@ -30,7 +30,7 @@ configure(rootProject) {
["Git", "help/git.txt", "Git assistance and guides."],
["ValidateLogCalls", "help/validateLogCalls.txt", "How to use logging calls efficiently."],
["IDEs", "help/IDEs.txt", "IDE support."],
- ["Docker", "help/docker.txt", "Building Solr Docker images."],
Review comment:
Ok, fine.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene-solr] HoustonPutman commented on a change in pull request #2197: SOLR-15075: Solr docker gradle improvements
Posted by GitBox <gi...@apache.org>.
HoustonPutman commented on a change in pull request #2197:
URL: https://github.com/apache/lucene-solr/pull/2197#discussion_r557510176
##########
File path: solr/docker/build.gradle
##########
@@ -18,106 +18,203 @@
import com.google.common.base.Preconditions
import com.google.common.base.Strings
-apply plugin: 'base'
-apply plugin: 'com.palantir.docker'
-
-subprojects {
- apply plugin: 'base'
- apply plugin: 'com.palantir.docker'
-}
-
description = 'Solr Docker image'
-def dockerPackage = project(':solr:docker:package')
-
-dependencies {
- docker dockerPackage
-}
+apply plugin: 'base'
+// Solr Docker inputs
def dockerImageRepo = propertyOrEnvOrDefault("solr.docker.imageRepo", "SOLR_DOCKER_IMAGE_REPO", "apache/solr")
def dockerImageTag = propertyOrEnvOrDefault("solr.docker.imageTag", "SOLR_DOCKER_IMAGE_TAG", "${version}")
def dockerImageName = propertyOrEnvOrDefault("solr.docker.imageName", "SOLR_DOCKER_IMAGE_NAME", "${dockerImageRepo}:${dockerImageTag}")
def baseDockerImage = propertyOrEnvOrDefault("solr.docker.baseImage", "SOLR_DOCKER_BASE_IMAGE", 'openjdk:11-jre-slim')
def githubUrlOrMirror = propertyOrEnvOrDefault("solr.docker.githubUrl", "SOLR_DOCKER_GITHUB_URL", 'github.com')
-docker {
- name = dockerImageName
- files file('include')
- buildArgs(['BASE_IMAGE' : baseDockerImage, 'SOLR_PACKAGE_IMAGE' : 'apache/solr-build:local-package', 'SOLR_VERSION': "${version}", 'GITHUB_URL': githubUrlOrMirror])
+// Build directory locations
+def dockerBuildDistribution = "$buildDir/distributions"
+def imageIdFile = "$buildDir/image-id"
+
+configurations {
+ packaging {
+ canBeResolved = true
+ }
+ dockerImage {
+ canBeResolved = true
+ }
}
-tasks.docker {
- // In order to create the solr docker image, the solr package image must be created first.
- dependsOn(dockerPackage.tasks.docker)
+dependencies {
+ packaging project(path: ":solr:packaging", configuration: 'archives')
+
+ dockerImage files(imageIdFile) {
+ builtBy 'dockerBuild'
+ }
+}
+
+task dockerTar(type: Tar) {
+ group = 'Docker'
+ description = 'Package docker context to prepare for docker build'
+
+ dependsOn configurations.packaging
+ into('scripts') {
+ from file('scripts')
+ fileMode 755
+ }
+ into('releases') {
+ from configurations.packaging
+ include '*.tgz'
+ }
+ from file('Dockerfile')
+ destinationDirectory = file(dockerBuildDistribution)
+ extension 'tgz'
+ compression = Compression.GZIP
+}
+
+task dockerBuild(dependsOn: tasks.dockerTar) {
+ group = 'Docker'
+ description = 'Build Solr docker image'
+
+ // Ensure that the docker image is rebuilt on build-arg changes or changes in the docker context
+ inputs.properties([
+ baseDockerImage: baseDockerImage,
+ githubUrlOrMirror: githubUrlOrMirror,
+ version: version
+ ])
+ inputs.dir(dockerBuildDistribution)
+
+ // Docker build must run after the docker context has been created & tarred
+ mustRunAfter tasks.dockerTar
Review comment:
Thanks, I am not very familiar with gradle, so that's exactly the kind of information I need.
We will keep it as is then, and if people complain or it is causing mass confusion, then we can make the necessary changes in the future.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene-solr] dsmiley commented on a change in pull request #2197: SOLR-15075: Solr docker gradle improvements
Posted by GitBox <gi...@apache.org>.
dsmiley commented on a change in pull request #2197:
URL: https://github.com/apache/lucene-solr/pull/2197#discussion_r560534771
##########
File path: solr/docker/build.gradle
##########
@@ -18,106 +18,192 @@
import com.google.common.base.Preconditions
import com.google.common.base.Strings
-apply plugin: 'base'
-apply plugin: 'com.palantir.docker'
-
-subprojects {
- apply plugin: 'base'
- apply plugin: 'com.palantir.docker'
-}
-
description = 'Solr Docker image'
-def dockerPackage = project(':solr:docker:package')
-
-dependencies {
- docker dockerPackage
-}
+apply plugin: 'base'
+// Solr Docker inputs
def dockerImageRepo = propertyOrEnvOrDefault("solr.docker.imageRepo", "SOLR_DOCKER_IMAGE_REPO", "apache/solr")
def dockerImageTag = propertyOrEnvOrDefault("solr.docker.imageTag", "SOLR_DOCKER_IMAGE_TAG", "${version}")
def dockerImageName = propertyOrEnvOrDefault("solr.docker.imageName", "SOLR_DOCKER_IMAGE_NAME", "${dockerImageRepo}:${dockerImageTag}")
def baseDockerImage = propertyOrEnvOrDefault("solr.docker.baseImage", "SOLR_DOCKER_BASE_IMAGE", 'openjdk:11-jre-slim')
def githubUrlOrMirror = propertyOrEnvOrDefault("solr.docker.githubUrl", "SOLR_DOCKER_GITHUB_URL", 'github.com')
-docker {
- name = dockerImageName
- files file('include')
- buildArgs(['BASE_IMAGE' : baseDockerImage, 'SOLR_PACKAGE_IMAGE' : 'apache/solr-build:local-package', 'SOLR_VERSION': "${version}", 'GITHUB_URL': githubUrlOrMirror])
+// Build directory locations
+def dockerBuildDistribution = "$buildDir/distributions"
+def imageIdFile = "$buildDir/image-id"
+
+configurations {
+ packaging {
+ canBeResolved = true
+ }
+ dockerImage {
+ canBeResolved = true
+ }
+}
+
+dependencies {
+ packaging project(path: ":solr:packaging", configuration: 'archives')
+
+ dockerImage files(imageIdFile) {
+ builtBy 'dockerBuild'
+ }
}
-tasks.docker {
- // In order to create the solr docker image, the solr package image must be created first.
- dependsOn(dockerPackage.tasks.docker)
+task dockerTar(type: Tar) {
Review comment:
In our container centric world right now... I think it would not be a big deal if our tgz included a Dockerfile at the root. I'm not sure I want to push for that at the moment (would others hate it?), but throwing it out there. Re-building *any* official tgz our project produces would be marvelously simple then :-). (curl some official tgz then pipe right into docker build :-)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene-solr] dsmiley commented on pull request #2197: SOLR-15075: Solr docker gradle improvements
Posted by GitBox <gi...@apache.org>.
dsmiley commented on pull request #2197:
URL: https://github.com/apache/lucene-solr/pull/2197#issuecomment-759773315
I'm glad you payed attention to task inputs/outputs. I have shelved changes to this gradle build to improve this because changes to the Dockerfile weren't seen as changing the output! (it's sort of the most important input). I can now remove my WIP because you've overhauled the whole thing, which I'm glad to see :-)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene-solr] HoustonPutman commented on a change in pull request #2197: SOLR-15075: Solr docker gradle improvements
Posted by GitBox <gi...@apache.org>.
HoustonPutman commented on a change in pull request #2197:
URL: https://github.com/apache/lucene-solr/pull/2197#discussion_r564598544
##########
File path: solr/docker/build.gradle
##########
@@ -18,106 +18,192 @@
import com.google.common.base.Preconditions
import com.google.common.base.Strings
-apply plugin: 'base'
-apply plugin: 'com.palantir.docker'
-
-subprojects {
- apply plugin: 'base'
- apply plugin: 'com.palantir.docker'
-}
-
description = 'Solr Docker image'
-def dockerPackage = project(':solr:docker:package')
-
-dependencies {
- docker dockerPackage
-}
+apply plugin: 'base'
+// Solr Docker inputs
def dockerImageRepo = propertyOrEnvOrDefault("solr.docker.imageRepo", "SOLR_DOCKER_IMAGE_REPO", "apache/solr")
def dockerImageTag = propertyOrEnvOrDefault("solr.docker.imageTag", "SOLR_DOCKER_IMAGE_TAG", "${version}")
def dockerImageName = propertyOrEnvOrDefault("solr.docker.imageName", "SOLR_DOCKER_IMAGE_NAME", "${dockerImageRepo}:${dockerImageTag}")
def baseDockerImage = propertyOrEnvOrDefault("solr.docker.baseImage", "SOLR_DOCKER_BASE_IMAGE", 'openjdk:11-jre-slim')
def githubUrlOrMirror = propertyOrEnvOrDefault("solr.docker.githubUrl", "SOLR_DOCKER_GITHUB_URL", 'github.com')
-docker {
- name = dockerImageName
- files file('include')
- buildArgs(['BASE_IMAGE' : baseDockerImage, 'SOLR_PACKAGE_IMAGE' : 'apache/solr-build:local-package', 'SOLR_VERSION': "${version}", 'GITHUB_URL': githubUrlOrMirror])
+// Build directory locations
+def dockerBuildDistribution = "$buildDir/distributions"
+def imageIdFile = "$buildDir/image-id"
+
+configurations {
+ packaging {
+ canBeResolved = true
+ }
+ dockerImage {
+ canBeResolved = true
+ }
+}
+
+dependencies {
+ packaging project(path: ":solr:packaging", configuration: 'archives')
+
+ dockerImage files(imageIdFile) {
+ builtBy 'dockerBuild'
+ }
}
-tasks.docker {
- // In order to create the solr docker image, the solr package image must be created first.
- dependsOn(dockerPackage.tasks.docker)
+task dockerTar(type: Tar) {
Review comment:
@dsmiley I still think that your idea is something that we should explore. I think that this PR (that I just merged) is just a step in the right direction. We might as well have something that is working much better now, and add final touches later.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene-solr] dweiss commented on a change in pull request #2197: SOLR-15075: Solr docker gradle improvements
Posted by GitBox <gi...@apache.org>.
dweiss commented on a change in pull request #2197:
URL: https://github.com/apache/lucene-solr/pull/2197#discussion_r556635582
##########
File path: solr/docker/build.gradle
##########
@@ -18,106 +18,187 @@
import com.google.common.base.Preconditions
import com.google.common.base.Strings
-apply plugin: 'base'
-apply plugin: 'com.palantir.docker'
-
-subprojects {
- apply plugin: 'base'
- apply plugin: 'com.palantir.docker'
-}
-
description = 'Solr Docker image'
-def dockerPackage = project(':solr:docker:package')
-
-dependencies {
- docker dockerPackage
-}
+apply plugin: 'base'
+// Solr Docker inputs
def dockerImageRepo = propertyOrEnvOrDefault("solr.docker.imageRepo", "SOLR_DOCKER_IMAGE_REPO", "apache/solr")
def dockerImageTag = propertyOrEnvOrDefault("solr.docker.imageTag", "SOLR_DOCKER_IMAGE_TAG", "${version}")
def dockerImageName = propertyOrEnvOrDefault("solr.docker.imageName", "SOLR_DOCKER_IMAGE_NAME", "${dockerImageRepo}:${dockerImageTag}")
def baseDockerImage = propertyOrEnvOrDefault("solr.docker.baseImage", "SOLR_DOCKER_BASE_IMAGE", 'openjdk:11-jre-slim')
def githubUrlOrMirror = propertyOrEnvOrDefault("solr.docker.githubUrl", "SOLR_DOCKER_GITHUB_URL", 'github.com')
-docker {
- name = dockerImageName
- files file('include')
- buildArgs(['BASE_IMAGE' : baseDockerImage, 'SOLR_PACKAGE_IMAGE' : 'apache/solr-build:local-package', 'SOLR_VERSION': "${version}", 'GITHUB_URL': githubUrlOrMirror])
+// Build directory locations
+def dockerBuildDistribution = "$buildDir/distributions"
+def imageIdFile = "$buildDir/image-id"
+
+configurations {
+ packaging {
+ canBeResolved = true
+ }
+ dockerImage {
+ canBeResolved = true
+ }
+}
+
+dependencies {
+ packaging project(path: ":solr:packaging", configuration: 'archives')
+
+ dockerImage files(imageIdFile) {
+ builtBy 'dockerBuild'
+ }
+}
+
+task dockerTar(type: Tar) {
+ group = 'Docker'
+ description = 'Package docker context to prepare for docker build'
+
+ dependsOn configurations.packaging
+ into('scripts') {
+ from file('scripts')
+ fileMode 755
+ }
+ into('releases') {
+ from configurations.packaging
+ include '*.tgz'
+ }
+ from file('Dockerfile')
+ destinationDirectory = file(dockerBuildDistribution)
+ extension 'tgz'
+ compression = Compression.GZIP
}
-tasks.docker {
- // In order to create the solr docker image, the solr package image must be created first.
- dependsOn(dockerPackage.tasks.docker)
+task dockerBuild(dependsOn: tasks.dockerTar) {
+ group = 'Docker'
+ description = 'Build Solr docker image'
+
+ // Ensure that the docker image is rebuilt on build-arg changes or changes in the docker context
+ inputs.properties([
+ baseDockerImage: baseDockerImage,
+ githubUrlOrMirror: githubUrlOrMirror,
+ version: version
+ ])
+ inputs.dir(dockerBuildDistribution)
+
+ doLast {
+ exec {
+ standardInput = tasks.dockerTar.outputs.files.singleFile.newDataInputStream()
+ commandLine "docker", "build",
+ "--iidfile", imageIdFile,
+ "--build-arg", "BASE_IMAGE=${inputs.properties.baseDockerImage}",
+ "--build-arg", "SOLR_VERSION=${version}",
+ "--build-arg", "GITHUB_URL=${inputs.properties.githubUrlOrMirror}",
+ "-"
+ }
+ }
// Print information on the image after it has been created
doLast {
+ def dockerImageId = file(imageIdFile).text
project.logger.lifecycle("Solr Docker Image Created")
- project.logger.lifecycle("\tName: $dockerImageName")
- project.logger.lifecycle("\tBase Image: $baseDockerImage")
+ project.logger.lifecycle("\tID: \t$dockerImageId")
+ project.logger.lifecycle("\tBase Image: \t$baseDockerImage")
+ project.logger.lifecycle("\tSolr Version: \t$version")
}
+
+ outputs.files(imageIdFile)
}
-abstract class DockerTestSuite extends DefaultTask {
- private String solrImageName = null;
- private List<String> tests = new ArrayList<>();
- private List<String> ignore = new ArrayList<>();
+task dockerTag(dependsOn: tasks.dockerBuild) {
+ group = 'Docker'
+ description = 'Tag Solr docker image'
- @OutputDirectory
- abstract DirectoryProperty getOutputDir()
+ // Ensure that the docker image is re-tagged if the image ID or desired tag changes
+ inputs.properties([
+ dockerImageName: dockerImageName,
+ ])
+ inputs.file(imageIdFile)
- public void setSolrImageName(String solrImageName) {
- this.solrImageName = solrImageName
+ doLast {
+ exec {
+ commandLine "docker", "tag", tasks.dockerBuild.outputs.files.singleFile.text, inputs.properties.dockerImageName
+ }
}
- public String getSolrImageName() {
- Preconditions.checkArgument(!Strings.isNullOrEmpty(solrImageName), "solrImageName is a required dockerTests configuration item.")
- return solrImageName
+ // Print information on the image after it has been created
+ doLast {
+ def dockerImageId = tasks.dockerBuild.outputs.files.singleFile.text
+ project.logger.lifecycle("Solr Docker Image Tagged")
+ project.logger.lifecycle("\tID: \t$dockerImageId")
+ project.logger.lifecycle("\tTag: \t$dockerImageName")
}
+}
- @Option(option = "tests", description = "Only run these specified tests, comma separated.")
- public void setTests(List<String> tests) {
- this.tests = tests;
- }
+task dockerPush(dependsOn: tasks.dockerTag) {
+ group = 'Docker'
+ description = 'Push Solr docker image'
- @Input
- public List<String> getTests() {
- return tests;
- }
+ // Ensure that the docker image is re-pushed if the image ID or tag changes
+ inputs.properties([
+ dockerImageName: dockerImageName,
+ ])
+ inputs.file(imageIdFile)
- @Option(option = "ignore", description = "Ignore these tests, comma separated.")
- public void setIgnore(List<String> ignore) {
- this.ignore = ignore;
+ doLast {
+ exec {
+ commandLine "docker", "push", dockerImageName
+ }
}
- @Input
- public List<String> getIgnore() {
- return ignore;
+ // Print information on the image after it has been created
+ doLast {
+ project.logger.lifecycle("Solr Docker Image Pushed: \t$dockerImageName")
}
+}
+
+
+task testDocker(dependsOn: tasks.dockerBuild) {
+ group = 'Docker'
+ description = 'Test Solr docker image'
+
+ def inputDir = "tests/cases"
+ def outputDir = "$buildDir/tmp/tests"
+
+ // Ensure that the docker image is re-tested if the image ID changes or the test files change
+ inputs.properties([
Review comment:
I think Gradle tries to serialize these for caches, hence the question. I don't know myself - that's why I asked.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org