You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@lucene.apache.org by dw...@apache.org on 2019/12/08 20:50:07 UTC
[lucene-solr] branch gradle-master updated (c3bb81f -> eeb1c9a)
This is an automated email from the ASF dual-hosted git repository.
dweiss pushed a change to branch gradle-master
in repository https://gitbox.apache.org/repos/asf/lucene-solr.git.
from c3bb81f Merge remote-tracking branch 'origin/master' into gradle-master
new 1021f04 Add some support for -Ptests.verbose mode when streams are dumped to the console. This is constrained by gradle's runner but is better than nothing.
new 4d30402 Add initial guidelines concerning dependency management.
new 02c79dd Add testOpts task and info about it in tests.txt
new eeb1c9a Only print the slowest tests at the end of a successful run. Correct verbose mode to parse string switch correctly.
The 4 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
gradle/help.gradle | 3 +-
gradle/testing/defaults-tests.gradle | 56 ++++++++++++---
gradle/testing/fail-on-no-tests.gradle | 6 +-
gradle/testing/randomization.gradle | 7 +-
gradle/testing/slowest-tests-at-end.gradle | 4 +-
help/dependencies.txt | 106 +++++++++++++++++++++++++++++
help/tests.txt | 24 ++++---
7 files changed, 180 insertions(+), 26 deletions(-)
create mode 100644 help/dependencies.txt
[lucene-solr] 04/04: Only print the slowest tests at the end of a
successful run. Correct verbose mode to parse string switch correctly.
Posted by dw...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
dweiss pushed a commit to branch gradle-master
in repository https://gitbox.apache.org/repos/asf/lucene-solr.git
commit eeb1c9abf6adf1237bfed4b53bb195a37a51e305
Author: Dawid Weiss <da...@carrotsearch.com>
AuthorDate: Sun Dec 8 19:06:01 2019 +0100
Only print the slowest tests at the end of a successful run. Correct verbose mode to parse string switch correctly.
---
gradle/testing/defaults-tests.gradle | 2 +-
gradle/testing/slowest-tests-at-end.gradle | 4 ++--
2 files changed, 3 insertions(+), 3 deletions(-)
diff --git a/gradle/testing/defaults-tests.gradle b/gradle/testing/defaults-tests.gradle
index 3dcf76c..f42bdba 100644
--- a/gradle/testing/defaults-tests.gradle
+++ b/gradle/testing/defaults-tests.gradle
@@ -5,7 +5,7 @@ def verboseModeHookInstalled = false
allprojects {
plugins.withType(JavaPlugin) {
- def verboseMode = propertyOrDefault("tests.verbose", "false")
+ def verboseMode = Boolean.parseBoolean(propertyOrDefault("tests.verbose", "false"))
project.ext {
testsWorkDir = file("${buildDir}/tmp/tests-cwd")
diff --git a/gradle/testing/slowest-tests-at-end.gradle b/gradle/testing/slowest-tests-at-end.gradle
index b3e6d20..5c26da9 100644
--- a/gradle/testing/slowest-tests-at-end.gradle
+++ b/gradle/testing/slowest-tests-at-end.gradle
@@ -15,8 +15,8 @@ allprojects {
}
}
-gradle.buildFinished {
- if (allTests) {
+gradle.buildFinished { result ->
+ if (allTests && result.getFailure() == null) {
def slowest = allTests
.sort { a, b -> b.duration.compareTo(a.duration) }
.take(10)
[lucene-solr] 03/04: Add testOpts task and info about it in
tests.txt
Posted by dw...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
dweiss pushed a commit to branch gradle-master
in repository https://gitbox.apache.org/repos/asf/lucene-solr.git
commit 02c79dd2112d9dfa5c7f97553b0583dbbc6fb43c
Author: Dawid Weiss <da...@carrotsearch.com>
AuthorDate: Sun Dec 8 18:45:41 2019 +0100
Add testOpts task and info about it in tests.txt
---
gradle/testing/randomization.gradle | 7 +++----
help/tests.txt | 10 +++++++++-
2 files changed, 12 insertions(+), 5 deletions(-)
diff --git a/gradle/testing/randomization.gradle b/gradle/testing/randomization.gradle
index 5983a72..84523df 100644
--- a/gradle/testing/randomization.gradle
+++ b/gradle/testing/randomization.gradle
@@ -63,7 +63,7 @@ allprojects {
// asserts, debug output.
[propName: 'tests.asserts', value: "true", description: "Enables or disables assertions mode."],
[propName: 'tests.verbose', value: false, description: "Emit verbose debug information from tests."],
- [propName: 'tests.infostream', value: null, description: "Enables or disables infostream logs."],
+ [propName: 'tests.infostream', value: false, description: "Enables or disables infostream logs."],
[propName: 'tests.leaveTemporary', value: null, description: "Leave temporary directories after tests complete."],
[propName: 'tests.useSecurityManager', value: false, description: "Enable security manager in tests.", buildOnly: true],
// component randomization
@@ -118,7 +118,6 @@ allprojects {
}
// These are not official options or dynamically seed-derived options.
- testOptionsResolved['tests.infostream'] = propertyOrDefault('tests.infostream', testOptionsResolved['tests.verbose'])
if (testOptionsResolved['tests.file.encoding'] == 'random') {
testOptionsResolved['tests.file.encoding'] = RandomPicks.randomFrom(
new Random(projectSeedLong), [
@@ -188,7 +187,7 @@ allprojects {
// and descriptions.
allprojects {
plugins.withType(JavaPlugin) {
- task helpTestOpts() {
+ task testOpts() {
group = 'Help (developer guides and hints)'
description = "Display values of randomization settings for a given seed"
@@ -196,7 +195,7 @@ allprojects {
println "Test options for project ${project.path} and seed \"${rootSeed}\":"
testOptions.sort { a, b -> a.propName.compareTo(b.propName) }.each { opt ->
- def defValue = opt.value
+ def defValue = Objects.toString(opt.value, null)
def value = testOptionsResolved[opt.propName]
println String.format(Locale.ROOT,
"%s%-23s = %-8s # %s",
diff --git a/help/tests.txt b/help/tests.txt
index 09d8c35..9f7a1f8 100644
--- a/help/tests.txt
+++ b/help/tests.txt
@@ -30,10 +30,18 @@ gradlew -p lucene/core check # By folder designation + task
Randomization
-------------
-Run tests with the given starting seed:
+To run tests with the given starting seed pass 'tests.seed'
+property:
gradlew :lucene:misc:test -Ptests.seed=DEADBEEF
+There are a lot of other test randomization properties
+available. To list them, their defaults and current values
+run the testOpts task against a project that has tests.
+For example:
+
+gradlew -p lucene/core testOpts
+
Filtering
---------
[lucene-solr] 01/04: Add some support for -Ptests.verbose mode when
streams are dumped to the console. This is constrained by gradle's runner
but is better than nothing.
Posted by dw...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
dweiss pushed a commit to branch gradle-master
in repository https://gitbox.apache.org/repos/asf/lucene-solr.git
commit 1021f04d1a7c4176d6ff0d437266f4c3fd9c1c4d
Author: Dawid Weiss <da...@carrotsearch.com>
AuthorDate: Sat Dec 7 14:53:13 2019 +0100
Add some support for -Ptests.verbose mode when streams are dumped to the console. This is constrained by gradle's runner but is better than nothing.
---
gradle/testing/defaults-tests.gradle | 56 ++++++++++++++++++++++++++++------
gradle/testing/fail-on-no-tests.gradle | 6 +++-
help/tests.txt | 14 ++++-----
3 files changed, 58 insertions(+), 18 deletions(-)
diff --git a/gradle/testing/defaults-tests.gradle b/gradle/testing/defaults-tests.gradle
index c454b5c..3dcf76c 100644
--- a/gradle/testing/defaults-tests.gradle
+++ b/gradle/testing/defaults-tests.gradle
@@ -1,8 +1,12 @@
import org.apache.tools.ant.taskdefs.condition.Os
import org.gradle.api.tasks.testing.logging.*
+def verboseModeHookInstalled = false
+
allprojects {
plugins.withType(JavaPlugin) {
+ def verboseMode = propertyOrDefault("tests.verbose", "false")
+
project.ext {
testsWorkDir = file("${buildDir}/tmp/tests-cwd")
testsTmpDir = file("${buildDir}/tmp/tests-tmp")
@@ -10,13 +14,33 @@ allprojects {
commonSolrDir = project(":solr").projectDir
}
+ // If we're running in verbose mode and:
+ // 1) worker count > 1
+ // 2) number of 'test' tasks in the build is > 1
+ // then the output would very likely be mangled on the
+ // console. Fail and let the user know what to do.
+ if (verboseMode && !verboseModeHookInstalled) {
+ verboseModeHookInstalled = true
+ if (gradle.startParameter.maxWorkerCount > 1) {
+ gradle.taskGraph.whenReady { graph ->
+ def testTasks = graph.allTasks.findAll { task -> task instanceof Test }
+ if (testTasks.size() > 1) {
+ throw new GradleException("Run your tests in verbose mode only with --max-workers=1 option passed to gradle.")
+ }
+ }
+ }
+ }
+
test {
- workingDir testsWorkDir
+ if (verboseMode) {
+ maxParallelForks = 1
+ } else {
+ maxParallelForks = propertyOrDefault("tests.jvms", (int) Math.max(1, Math.min(Runtime.runtime.availableProcessors() / 2.0, 4.0)))
+ }
+ workingDir testsWorkDir
useJUnit()
- maxParallelForks = propertyOrDefault("tests.jvms", (int) Math.max(1, Math.min(Runtime.runtime.availableProcessors() / 2.0, 4.0)))
-
minHeapSize = "256m"
maxHeapSize = "512m"
@@ -44,12 +68,26 @@ allprojects {
}
// Set up logging.
- testLogging {
- events TestLogEvent.FAILED
- exceptionFormat TestExceptionFormat.FULL
- showExceptions true
- showCauses true
- showStackTraces true
+ if (verboseMode) {
+ testLogging {
+ events TestLogEvent.FAILED
+ exceptionFormat TestExceptionFormat.FULL
+ showExceptions true
+ showCauses true
+ showStackTraces true
+ showStandardStreams = false
+ }
+ onOutput { td, event ->
+ print event.message
+ }
+ } else {
+ testLogging {
+ events TestLogEvent.FAILED
+ exceptionFormat TestExceptionFormat.FULL
+ showExceptions true
+ showCauses true
+ showStackTraces true
+ }
}
doFirst {
diff --git a/gradle/testing/fail-on-no-tests.gradle b/gradle/testing/fail-on-no-tests.gradle
index dd148d1..f037725 100644
--- a/gradle/testing/fail-on-no-tests.gradle
+++ b/gradle/testing/fail-on-no-tests.gradle
@@ -22,8 +22,12 @@ gradle.taskGraph.whenReady { graph ->
// ... and there are some test tasks in the execution graph.
if (!testTasks.isEmpty()) {
def executedTests = 0
+ def executedTasks = 0
testTasks.each { task ->
+ task.doFirst {
+ executedTasks++
+ }
task.afterSuite { desc, result ->
executedTests += result.testCount
}
@@ -31,7 +35,7 @@ gradle.taskGraph.whenReady { graph ->
// After the build is finished, check the test count.
gradle.buildFinished {
- if (executedTests == 0) {
+ if (executedTests == 0 && executedTasks > 0) {
throw new GradleException("No tests found for the given filters?")
}
}
diff --git a/help/tests.txt b/help/tests.txt
index 686113e..09d8c35 100644
--- a/help/tests.txt
+++ b/help/tests.txt
@@ -94,13 +94,11 @@ cleanTest task:
gradlew -p lucene/core cleanTest test -Ptests.seed=deadbeef
+Verbose mode and debugging
+--------------------------
+The "tests.verbose" mode switch enables standard streams from tests
+to be dumped directly to the console. Run your verbose tests explicitly
+specifying the project and test task or a fully qualified task path. Example:
-
-
-
-
-
-
-
-
+gradlew -p lucene/core test -Ptests.verbose=true --tests "TestDemo"
[lucene-solr] 02/04: Add initial guidelines concerning dependency
management.
Posted by dw...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
dweiss pushed a commit to branch gradle-master
in repository https://gitbox.apache.org/repos/asf/lucene-solr.git
commit 4d3040235ef3588dccc0748f85b396a4a50f2df9
Author: Dawid Weiss <da...@carrotsearch.com>
AuthorDate: Sun Dec 8 18:34:12 2019 +0100
Add initial guidelines concerning dependency management.
---
gradle/help.gradle | 3 +-
help/dependencies.txt | 106 ++++++++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 108 insertions(+), 1 deletion(-)
diff --git a/gradle/help.gradle b/gradle/help.gradle
index e5f8014..e526389 100644
--- a/gradle/help.gradle
+++ b/gradle/help.gradle
@@ -5,8 +5,9 @@ configure(rootProject) {
["Workflow", "help/workflow.txt", "Typical workflow commands."],
["Ant", "help/ant.txt", "Ant-gradle migration help."],
["Tests", "help/tests.txt", "Tests, filtering, beasting, etc."],
+ ["Deps", "help/dependencies.txt", "Declaring, inspecting and excluding dependencies."],
["ForbiddenApis", "help/forbiddenApis.txt", "How to add/apply rules for forbidden APIs."],
- ["LocalSettings", "help/localSettings.txt", "Local settings, overrides and build performance tweaks."]
+ ["LocalSettings", "help/localSettings.txt", "Local settings, overrides and build performance tweaks."],
]
helpFiles.each { section, path, sectionInfo ->
diff --git a/help/dependencies.txt b/help/dependencies.txt
new file mode 100644
index 0000000..b68750c
--- /dev/null
+++ b/help/dependencies.txt
@@ -0,0 +1,106 @@
+Dependencies
+============
+
+Each gradle project can have multiple (named) "configurations"
+and each configuration can have dependencies attached to it.
+
+There are some standard conventions so, for example, the Java plugin
+adds standard configurations such as "api", "implementation",
+"testImplementation" and others. These configurations can also inherit
+from each other; more about this typic can be found here:
+
+[TODO: add conigurations link, java plugin configs]
+
+For the needs of Lucene and Solr we will typically focus on three
+configurations and attach project dependencies to them:
+
+api - makes a dependency available for main classes, tests and any
+ other modules importing the project (exportable dependency),
+
+implementation - makes a dependency available for main classes, tests
+ but will *not* export the dependency for other modules (so their
+ compilation classpath won't contain it).
+
+testImplementation - makes a dependency only available for test
+ classes.
+
+
+Adding a library dependency
+---------------------------
+
+Let's say we wish to add a dependency on library "foo.bar:baz" in
+version 1.2 to :lucene:core. Let's assume this library is only
+used internally by the project. The :lucene:core project is configured
+by lucene/core/build.gradle and we would add (or modify) the dependency
+block as follows:
+
+dependencies {
+ implementation "foo.bar:baz"
+}
+
+The "implementation" here is a named configuration; we don't need to declare
+it because it is declared for us by the java-library plugin.
+
+In "normal" gradle the version of the dependency would be present
+directly inside the declaration but we use a plugin
+(palantir-consistent-versions) to manage all dependency versions
+from the top-level (so that conflicts can be resolved globally).
+
+If this is the first time "foo.bar:baz" is added to the project, we'd have
+to add its version to "versions.props" file at the top level of the
+checkout:
+
+foo.bar:baz=1.2
+
+and then regenerate the "versions.lock" file using the following
+command:
+
+gradlew --write-locks
+
+IMPORTANT: The versions.lock file will contain the actual version
+of the dependency picked based on other project dependencies and
+their transitive dependencies.
+
+Once the dependency is added it always makes sense to see the
+tree of all module dependencies and maybe exclude transitive
+dependencies of foo.bar:baz that we won't need.
+
+
+Inspecting current dependencies
+-------------------------------
+
+The tree of dependencies of a project (in all configurations) can
+be dumped by the following command (example):
+
+gradlew -p lucene\analysis\icu dependencies
+
+But this can be a bit overwhelming; we will most likely be interested
+in just the "publicly visible" and "classpath-visible" configurations.
+
+The publicly visible project dependencies (classes shared by other
+modules importing our module) can be displayed with:
+
+gradlew -p lucene\analysis\icu dependencies --configuration api
+
+And the "private" set of dependencies (real classpath) can be dumped
+with:
+
+gradlew -p lucene\analysis\icu dependencies --configuration runtimeClasspath
+
+
+Excluding a transitive dependency
+---------------------------------
+
+Let's say "foo.bar:baz" has a transitive dependency on project
+"foo.bar:irrelevant" and we know the transitive dependency is not
+crucial for the functioning of "foo.bar:baz". We can exclude it
+by adding an exclusion block to the original declaration:
+
+dependencies {
+ implementation("foo.bar:baz", {
+ exclude group: "foo.bar", module: "irrelevant"
+ })
+}
+
+Note the brackets - they are important and prevent accidental
+mistakes of applying the exclusion to the wrong scope.
\ No newline at end of file