[GitHub] [cassandra-website] jacek-lewandowski commented on a change in pull request #79: build and test documentation improvements

[GitBox <gi...@apache.org>]

jacek-lewandowski commented on a change in pull request #79:
URL: https://github.com/apache/cassandra-website/pull/79#discussion_r740738708



##########
File path: site-content/source/modules/ROOT/pages/development/testing.adoc
##########
@@ -34,56 +86,460 @@ public void testBatchAndList() throws Throwable
 }
 ----
 
-Unit tests can be run using the command `ant test`. Both test suites and
-individual tests can be executed.
+=== Running JVM unit tests
+
+To run the unit tests, simply call:
 
-Test suite:
-[source, plaintext]
+[source,none]
 ----
-ant test -Dtest.name=<simple_classname>
+ant test
 ----
-For example, replace `<simple_classname>` with `SimpleQueryTest` to test all the methods in `org.apache.cassandra.cql3.SimpleQueryTest`.
 
-Individual test:
-[source, plaintext]
+However, this is probably not what you would usually want to do. That
+command would run all the unit tests (those from `test/unit`). It would
+take about an hour or more to finish.
+
+To run the specific test class or even a method, use the following
+command:
+
+[source,none]
 ----
-ant testsome -Dtest.name=<FQCN> -Dtest.methods=<testmethod1>[,testmethod2]
+ant testsome -Dtest.name=<TestClassName> -Dtest.method=<testMethodName>
 ----
 
-For example, replace `<FQCN>` with `org.apache.cassandra.cql3.SimpleQueryTest`
-and `<testmethod1>` with `testStaticCompactTables` to test just the one method.
+- `test.name` property is for either a simple or fully qualified class
+name
+- `test.method` property is optional; if not specified, all test cases
+from the specified class are executed. Though, you can also specify
+multiple methods separating them by comma
+
+You can also use IDE to run the tests - when you generate IDE files and
+properly import the Cassandra project, you should be able to run the
+tests by right-clicking on the test class or package name. Remember that
+it is not enough to compile with IDE for some tests, and you need to
+call `ant jar` to build the distribution artifacts. It is required when
+the test runs some tool as an external process, and that tool expects
+Cassandra artifacts to be in the build directory.
+
+Note that those commands apply to the tests in the `test/unit`
+directory. There are, however, some other test categories that have
+tests in individual directories:
 
+- `test/burn` - to run them, first build the uber jar with
+`ant burn-test-jar`, and then to run the tests call `ant test-burn` or
+`ant burn-testsome`
+- `test/long` - to run them, call `ant long-test` or `ant long-testsome`
+- `test/memory` - to run them, call `ant test-memory`
+- `test/microbench` discussed in <<microbenchmarks>>
+- `test/distributed` discussed in <<jvm_distributed_tests>>
 
-If you get the following error for a unit test, install the `ant-optional` package
-because you need the `JUnitTask` class:
+[TIP]
+.Hint
+====
+If you get the error similar to the one below, install the
+`ant-optional` package
+because you need the `JUnitTask` class
+(see xref:development/ide.adoc[prerequisites]).
 
 [source,none]
 ----
 Throws: cassandra-trunk/build.xml:1134: taskdef A class needed by class org.krummas.junit.JStackJUnitTask cannot be found:
 org/apache/tools/ant/taskdefs/optional/junit/JUnitTask  using the classloader
 AntClassLoader[/.../cassandra-trunk/lib/jstackjunit-0.0.1.jar]
 ----
+====
+
+=== Stress and FQLTool tests
+
+_Stress_ and _FQLTool_ are separate modules located under the `tools`
+directory in the Cassandra project. They have their own source code and
+unit tests. To run the tests for those tools, first, build jar artifacts
+for them but calling:
+
+[source,shell script]
+----
+ant fqltool-build fqltool-build-test
+ant stress-build stress-build-test
+----
+
+Then you can execute the tests with either one of the commands:
+
+[source,shell script]
+----
+ant fqltool-test
+ant stress-test
+and stress-test-some
+----
+
+or using your IDE.
+
+[#jvm_distributed_tests]
+=== JVM distributed tests
+
+JVM distributed tests can run a cluster of nodes inside a single JVM -
+they utilize a particular framework (that can be found at
+https://github.com/apache/cassandra-in-jvm-dtest-api[apache/cassandra-in-jvm-dtest-api])
+for that purpose. Those tests are intended to test features that require
+more started nodes or verify specific behaviors when the nodes get
+restarted, including upgrading them from one version to another. The
+tests are located at the `test/distributed` directory of the Cassandra
+project; however, only `org.apache.cassandra.distributed.test` and
+`org.apache.cassandra.upgrade` packages contain the actual tests. The
+rest of the files are various utilities related to the distributed tests
+framework.
+
+The distributed tests can be run in few ways. `ant test-jvm-dtest`
+command runs all the distributed JVM tests. It is not very useful; thus,
+we have also `ant test-jvm-dtest-some`, which allows specifying test
+class and test name in the same way as we could do that for the
+`ant testsome` command (see the JVM unit tests section). Distributed
+tests can also be run using IDE (in fact, I could successfully run JVM
+distributed tests in IntelliJ and even debug them).
+
+==== Upgrade tests
+
+JVM upgrade tests can be run precisely in the same way as any other JVM
+distributed tests. However, running them requires some preparation -
+for example, if a test verifies the upgrade from Cassandra 3.11 to the
+current version (say Cassandra 4.0), we need to have prepared dtest uber
+jars for all involved versions. To do this, check out Cassandra 3.11
+based branch you want to test the upgrade from into some other directory
+and run:
+
+[source,shell script]
+----
+ant dtest-jar
+----
+
+It creates `build/dtest-3.11.x.jar`, which has to be copied to the build
+directory of your current Cassandra project. Once you have dtest jars of
+all the involved versions for the upgrade test you want to run, you can
+finally execute the test using your favorite method.
+
+=== Running multiple tests
+
+It is possible to define a list of test classes that would run with a
+single command. To do that, you need to define a text file, by default
+called `testlist.txt`, and put it into your project directory. Such a
+file may look like the following:
+
+[source,none]
+----
+org/apache/cassandra/db/ReadCommandTest.java
+org/apache/cassandra/db/ReadCommandVerbHandlerTest.java
+----
+
+Essentially, you list the paths to the class files of the tests you want
+to run. Then you just call `ant testclasslist` - it takes that text file
+and runs the tests listed there. Note that - by default, it applies to
+the tests under the `test/unit` directory and takes the `testlist.txt`
+file, but this behavior can be modified by providing additional
+parameters:
+
+[source,shell script]
+----
+ant testclasslist -Dtest.classlistprefix=<category> -Dtest.classlistfile=<class list file>
+----
+
+For example, if we want to run the distributed tests this way, and say
+we listed our tests in the `distributed-tests-set.txt` file (paths to
+test classes relative to `test/distributed` directory), we can do that
+by calling:
+
+[source,shell script]
+----
+ant testclasslist -Dtest.classlistprefix=distributed -Dtest.classlistfile=distributed-tests-set.txt
+----
+
+=== Running coverage analysis
+
+Coverage reports from the executed JVM tests can be obtained in two ways
+- through IDE - for example, IntelliJ supports running tests with
+coverage analysis - it is just another run button next to the one for
+running in debug mode.
+
+The other way is to run Ant target `codecoverage`. Basically, it works
+for all the ways mentioned above of running JVM tests - the only
+difference is that instead of specifying the target directly, we pass it
+as a property called `taskname`. For example - given the original test
+command is:
+
+[source,shell script]
+----
+ant testsome -Dtest.name=org.apache.cassandra.utils.concurrent.AccumulatorTest
+----
+
+to run it with coverage analysis, we do:
+
+[source,shell script]
+----
+ant codecoverage -Dtaskname=testsome -Dtest.name=org.apache.cassandra.utils.concurrent.AccumulatorTest
+----
+
+It applies to all the targets like `test`, `testsome`, `test-long`,
+etc., even `testclasslist`. You can find the coverage report in
+`build/jacoco` (`index.html` is the entry point for the HTML version,
+but there are also XML and CSV reports).
+
+Note that if you run various tests that way, the coverage information is
+added to the previously collected runs. That is, you get the cumulative
+coverage from all runs unless you clean up the project or at least clean
+up the recorded coverage information by executing the command
+`ant jacoco-cleanup`.
+
+[#microbenchmarks]
+=== Micro-benchmarks
+
+To run micro-benchmarks, you first need to build the uber jar for the
+JMH framework. It can be accomplished by invoking:
+
+[source,shell script]
+----
+ant build-jmh
+----
+
+Then, you can run either all benchmarks (from the `test/microbench`
+directory) or the tests matching the name specified by the
+`benchmark.name` property when executing the `ant microbench` command.
+Whether you run all benchmarks or just a selected one, only classes
+under the `microbench` package are selected. Actually the class
+selection pattern is as `.\*microbench.*${benchmark.name}`. For example,
+in order to run `org.apache.cassandra.test.microbench.ChecksumBench`,
+you may execute:
+
+[source,shell script]
+----
+ant microbench -Dbenchmark.name=ChecksumBench
+----
+
+The `ant microbench` command runs the benchmarks with default parameters
+as defined in the `build.xml` file (see the `microbench` target
+definition). If you want to run JMH with custom parameters, you may
+consider using the `test/bin/jmh` script. Apart from allowing you to
+customize JMH options, it also sets up the environment and JVM options
+by running Cassandra init script (`conf/cassandra-env.sh`), so the
+environment for running the tests is more similar to the production
+environment. For example:
+
+[source,shell script]
+----
+test/bin/jmh -gc true org.apache.cassandra.test.microbench.CompactionBench.compactTest
+----
+
+You may also find it useful to run the command to list all the tests:
+`test/bin/jmh -l` or `test/bin/jmh -lp` (also showing the default
+parameters). The list of all options can be shown by running
+`test/bin/jmh -h`
+
+== Python tests
+
+=== Docker
+
+It is not entirely required to run Python tests in Docker. Still, it is
+recommended to get repeatable behavior, especially if you want to test
+something in the same environment as it is tested on the official
+Cassandra CI.
+
+==== Setup Docker
 
-Tests that consume a significant amount of time during execution can be found
-in the `test/long` directory.
-They can be executed as a regular JUnit test or standalone program.
-Except for the execution time, there’s nothing
-really special about them, but `ant` will only execute these test with the
-`ant long-test` target.
+Make sure you have Docker Desktop enabled if you want to use it - in the
+case of Docker Desktop, you need to specify the amount of resource you
+want to give the underlying virtual machine. Most distributed tests are
+not that demanding - usually, 4G / 2 cores should be enough to start one
+node. However, some tests really run multiple nodes, and some of them
+are automatically skipped if your machine has less than 32G - there is
+a way to force running them anyway. We will go back to that later. I
+usually use 8G / 4 cores out of 32G / 12 cores total available on my Mac.
 
-=== DTests
+==== Pull the Docker image
+
+The Docker image used on the official Cassandra CI can be found in
+https://github.com/apache/cassandra-builds[this] repository.
+You can use either
+https://github.com/apache/cassandra-builds/blob/trunk/docker/testing/ubuntu2004_j11.docker[docker/testing/ubuntu2004_j11.docker]
+or
+https://github.com/apache/cassandra-builds/blob/trunk/docker/testing/ubuntu2004_j11_w_dependencies.docker[docker/testing/ubuntu2004_j11_w_dependencies.docker]
+- the latter one has additional dependencies for upgrade testing (read
+more about that in the next section). Those images can be either built
+locally (as per instructions in the GitHub repo) or pulled from the
+Docker Hub - see
+https://hub.docker.com/search?q=apache%2Fcassandra-testing&type=image[here].
+So, firstly pull the image from Docker Hub (it will either fetch or
+update the image you previously fetched):
+
+[source, shell script]
+----
+docker pull apache/cassandra-testing-ubuntu2004-java11-w-dependencies
+----
+
+==== Start the container

Review comment:
       thanks




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org