You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@commons.apache.org by ah...@apache.org on 2019/10/23 16:57:04 UTC
[commons-rng] branch master updated (a7ec057 -> 4679139)
This is an automated email from the ASF dual-hosted git repository.
aherbert pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/commons-rng.git.
from a7ec057 Moved test for a different RNG output to RandomAssert.
new b63f5f0 Add hex seed option to stress command.
new d0033bb Add test for round trip of seeds from bytes to chars.
new 7fa3ded Added patch for PractRand v0.93 to flush stdout after each incremental result.
new 2170c04 Updated stress command to allow application threads argument.
new 9307eac Updated stress test documentation for PractRand.
new 0610ecd Update stdin2testu01 raw modes to have a maximum output argument.
new 3c6751a Update endianness documentation.
new ede9cf4 Update stress test documentation.
new 4679139 Updated PractRand patch files to fix linux compilation.
The 9 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.gitattributes | 2 +
commons-rng-examples/examples-stress/endianness.md | 110 +++++++++++---
.../examples-stress/src/main/c/stdin2testu01.c | 30 ++--
.../commons/rng/examples/stress/OutputCommand.java | 2 +-
.../rng/examples/stress/StressTestCommand.java | 76 +++++++++-
.../main/resources/patches/PractRand_0.93.patch | 33 +++++
.../main/resources/patches/PractRand_0.94.patch | 53 +++++++
.../commons/rng/examples/stress/HexTest.java | 119 +++++++++++++++
.../examples-stress/stress_test.md | 162 +++++++++++++++++----
9 files changed, 521 insertions(+), 66 deletions(-)
create mode 100644 .gitattributes
create mode 100644 commons-rng-examples/examples-stress/src/main/resources/patches/PractRand_0.93.patch
create mode 100644 commons-rng-examples/examples-stress/src/main/resources/patches/PractRand_0.94.patch
create mode 100644 commons-rng-examples/examples-stress/src/test/java/org/apache/commons/rng/examples/stress/HexTest.java
[commons-rng] 02/09: Add test for round trip of seeds from bytes to
chars.
Posted by ah...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
aherbert pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/commons-rng.git
commit d0033bbc16ec3421ae5aa62bbdeb958353064f1c
Author: aherbert <ah...@apache.org>
AuthorDate: Wed Oct 23 11:55:58 2019 +0100
Add test for round trip of seeds from bytes to chars.
---
.../commons/rng/examples/stress/HexTest.java | 119 +++++++++++++++++++++
1 file changed, 119 insertions(+)
diff --git a/commons-rng-examples/examples-stress/src/test/java/org/apache/commons/rng/examples/stress/HexTest.java b/commons-rng-examples/examples-stress/src/test/java/org/apache/commons/rng/examples/stress/HexTest.java
new file mode 100644
index 0000000..1fe5c58
--- /dev/null
+++ b/commons-rng-examples/examples-stress/src/test/java/org/apache/commons/rng/examples/stress/HexTest.java
@@ -0,0 +1,119 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.commons.rng.examples.stress;
+
+import org.junit.Assert;
+import org.junit.Test;
+
+import java.util.Arrays;
+import java.util.Locale;
+import java.util.concurrent.ThreadLocalRandom;
+
+/**
+ * Tests for {@link Hex}.
+ */
+public class HexTest {
+ /** Upper-case hex digits. */
+ private static final char[] DIGITS = {
+ '0', '1', '2', '3', '4', '5', '6', '7', '8', '9', 'A', 'B', 'C', 'D', 'E', 'F'
+ };
+
+ /**
+ * Test a round-trip hex encode and decode of a byte[] seed.
+ *
+ * <p>This test ensures that a generated byte[] seed recorded by the stress command
+ * in the results file can be used to create the same seed.
+ */
+ @Test
+ public void testHexEncodeAndDecode() {
+ // Empty bytes
+ for (int size = 0; size < 10; size++) {
+ assertHexEncodeAndDecode(new byte[size]);
+ }
+ // Random bytes
+ for (int size : new int[] {3, 4, 5, 8, 16, 31}) {
+ final byte[] bytes = new byte[size];
+ for (int i = 0; i < 5; i++) {
+ ThreadLocalRandom.current().nextBytes(bytes);
+ assertHexEncodeAndDecode(bytes);
+ }
+ }
+ }
+
+ /**
+ * Assert the bytes can be encoded and decoded.
+ *
+ * @param bytes the bytes
+ */
+ private static void assertHexEncodeAndDecode(byte[] bytes) {
+ // Use a StringBuilder to append the chars as this is the method used in the stress command.
+ final StringBuilder sb = new StringBuilder();
+ sb.append(Hex.encodeHex(bytes));
+ final byte[] decoded = Hex.decodeHex(sb);
+ Assert.assertArrayEquals(bytes, decoded);
+ }
+
+ /**
+ * Test a round-trip hex decode and encode of a char[] seed.
+ *
+ * <p>This test ensures that a user input random hex seed passed to the stress command
+ * can be converted to bytes and then recorded in the results file.
+ */
+ @Test
+ public void testHexDecodeAndEncode() {
+ // Note: char[] must be an even length.
+ // Empty chars.
+ for (int size = 0; size < 10; size++) {
+ final char[] chars = new char[size * 2];
+ Arrays.fill(chars, '0');
+ assertHexDecodeAndEncode(chars);
+ }
+ // Random bytes
+ for (int size : new int[] {3, 4, 5, 8, 16, 31}) {
+ final char[] chars = new char[size * 2];
+ for (int i = 0; i < 5; i++) {
+ // Encode upper case
+ for (int j = 0; j < chars.length; j++) {
+ chars[j] = DIGITS[ThreadLocalRandom.current().nextInt(16)];
+ }
+ assertHexDecodeAndEncode(chars);
+ }
+ }
+ }
+
+ /**
+ * Assert the characters can be decoded and encoded.
+ *
+ * @param chars the chars
+ */
+ private static void assertHexDecodeAndEncode(char[] chars) {
+ final String text = String.valueOf(chars);
+ final byte[] decoded = Hex.decodeHex(text);
+ // Test the encoding is lower case
+ Assert.assertArrayEquals(text.toLowerCase(Locale.US).toCharArray(), Hex.encodeHex(decoded));
+ }
+
+ @Test(expected = IllegalArgumentException.class)
+ public void testHexThrowsWithOddNumberOfCharacters() {
+ Hex.decodeHex("0");
+ }
+
+ @Test(expected = IllegalArgumentException.class)
+ public void testHexThrowsWithIllegalHexCharacters() {
+ Hex.decodeHex("0g");
+ }
+}
[commons-rng] 03/09: Added patch for PractRand v0.93 to flush
stdout after each incremental result.
Posted by ah...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
aherbert pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/commons-rng.git
commit 7fa3ded5dae92cf8579b343c4bb1ee82026fadfb
Author: aherbert <ah...@apache.org>
AuthorDate: Wed Oct 23 12:05:36 2019 +0100
Added patch for PractRand v0.93 to flush stdout after each incremental result.
---
.../src/main/resources/patches/RNG_test_0.93.patch | 10 ++++++++++
1 file changed, 10 insertions(+)
diff --git a/commons-rng-examples/examples-stress/src/main/resources/patches/RNG_test_0.93.patch b/commons-rng-examples/examples-stress/src/main/resources/patches/RNG_test_0.93.patch
new file mode 100644
index 0000000..a71705c
--- /dev/null
+++ b/commons-rng-examples/examples-stress/src/main/resources/patches/RNG_test_0.93.patch
@@ -0,0 +1,10 @@
+--- tools/RNG_test.cpp 2019-10-23 12:01:43.168306859 +0100
++++ tools/RNG_test.new.cpp 2019-10-23 12:02:32.696759305 +0100
+@@ -500,6 +500,7 @@
+ else
+ std::printf(" ...and %d test result(s) without anomalies\n", int(results.size() - marked.size()));
+ std::printf("\n");
++ std::fflush(stdout);
+ if (end_on_failure && biggest_decimal_suspicion > 8.5) std::exit(0);
+ }
+ double interpret_length(const std::string &lengthstr, bool normal_mode) {
[commons-rng] 04/09: Updated stress command to allow application
threads argument.
Posted by ah...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
aherbert pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/commons-rng.git
commit 2170c04c7ba07bb3f31f6acb5e5d247d296a36b9
Author: aherbert <ah...@apache.org>
AuthorDate: Wed Oct 23 13:39:59 2019 +0100
Updated stress command to allow application threads argument.
This supports smart computation of concurrent tasks for multi-threaded
test applications such as PractRand.
---
.../rng/examples/stress/StressTestCommand.java | 49 ++++++++++++++++++----
1 file changed, 42 insertions(+), 7 deletions(-)
diff --git a/commons-rng-examples/examples-stress/src/main/java/org/apache/commons/rng/examples/stress/StressTestCommand.java b/commons-rng-examples/examples-stress/src/main/java/org/apache/commons/rng/examples/stress/StressTestCommand.java
index 53b0ec0..6ed4dd5 100644
--- a/commons-rng-examples/examples-stress/src/main/java/org/apache/commons/rng/examples/stress/StressTestCommand.java
+++ b/commons-rng-examples/examples-stress/src/main/java/org/apache/commons/rng/examples/stress/StressTestCommand.java
@@ -117,11 +117,23 @@ class StressTestCommand implements Callable<Void> {
"Use for parallel tests with the same output prefix."})
private int trialOffset;
- /** The number of concurrent tasks. */
- @Option(names = {"-n", "--tasks"},
- description = {"Number of concurrent tasks (default: ${DEFAULT-VALUE}).",
- "Two threads are required per task."})
- private int taskCount = Math.max(1, Runtime.getRuntime().availableProcessors() / 2);
+ /** The number of available processors. */
+ @Option(names = {"-p", "--processors"},
+ description = {"Number of available processors (default: ${DEFAULT-VALUE}).",
+ "Number of concurrent tasks = ceil(processors / threadsPerTask)",
+ "threadsPerTask = applicationThreads + (ignoreJavaThread ? 0 : 1)"})
+ private int processors = Math.max(1, Runtime.getRuntime().availableProcessors());
+
+ /** The number of threads to use for each test task. */
+ @Option(names = {"--ignore-java-thread"},
+ description = {"Ignore the java RNG thread when computing concurrent tasks."})
+ private boolean ignoreJavaThread;
+
+ /** The number of threads to use for each testing application. */
+ @Option(names = {"--threads"},
+ description = {"Number of threads to use for each application (default: ${DEFAULT-VALUE}).",
+ "Total threads per task includes an optional java thread."})
+ private int applicationThreads = 1;
/** The size of the byte buffer for the binary data. */
@Option(names = {"--buffer-size"},
@@ -350,11 +362,13 @@ class StressTestCommand implements Callable<Void> {
final String basePath = fileOutputPrefix.getAbsolutePath();
checkExistingOutputFiles(basePath, stressTestData);
- final ProgressTracker progressTracker = new ProgressTracker(taskCount);
+ final int parallelTasks = getParallelTasks();
+
+ final ProgressTracker progressTracker = new ProgressTracker(parallelTasks);
final List<Runnable> tasks = createTasks(command, basePath, stressTestData, progressTracker);
// Run tasks with parallel execution.
- final ExecutorService service = Executors.newFixedThreadPool(taskCount);
+ final ExecutorService service = Executors.newFixedThreadPool(parallelTasks);
LogUtils.info("Running stress test ...");
LogUtils.info("Shutdown by creating stop file: %s", stopFile);
@@ -433,6 +447,27 @@ class StressTestCommand implements Callable<Void> {
}
/**
+ * Gets the number of parallel tasks. This uses the number of available processors and
+ * the number of threads to use per task.
+ *
+ * <pre>
+ * threadsPerTask = applicationThreads + (ignoreJavaThread ? 0 : 1)
+ * parallelTasks = ceil(processors / threadsPerTask)
+ * </pre>
+ *
+ * @return the parallel tasks
+ */
+ private int getParallelTasks() {
+ // Avoid zeros in the fraction numberator and denominator
+ final int availableProcessors = Math.max(1, processors);
+ final int threadsPerTask = Math.max(1, applicationThreads + (ignoreJavaThread ? 0 : 1));
+ final int parallelTasks = (int) Math.ceil((double) availableProcessors / threadsPerTask);
+ LogUtils.debug("Parallel tasks = %d (%d / %d)",
+ parallelTasks, availableProcessors, threadsPerTask);
+ return parallelTasks;
+ }
+
+ /**
* Create the tasks for the test data. The output file for the sub-process will be
* constructed using the base path, the test identifier and the trial number.
*
[commons-rng] 06/09: Update stdin2testu01 raw modes to have a
maximum output argument.
Posted by ah...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
aherbert pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/commons-rng.git
commit 0610ecd89e27b9ddcef43dc63afb62ca4e3f9866
Author: aherbert <ah...@apache.org>
AuthorDate: Wed Oct 23 14:56:54 2019 +0100
Update stdin2testu01 raw modes to have a maximum output argument.
---
.../examples-stress/src/main/c/stdin2testu01.c | 30 +++++++++++++++-------
1 file changed, 21 insertions(+), 9 deletions(-)
diff --git a/commons-rng-examples/examples-stress/src/main/c/stdin2testu01.c b/commons-rng-examples/examples-stress/src/main/c/stdin2testu01.c
index fc0ffb7..0b4d4da 100644
--- a/commons-rng-examples/examples-stress/src/main/c/stdin2testu01.c
+++ b/commons-rng-examples/examples-stress/src/main/c/stdin2testu01.c
@@ -131,7 +131,7 @@ void printLong(uint64_t value)
putchar(' ');
printByte((uint8_t)( value & 0xff));
/* Write the unsigned and signed int value */
- printf(" %20llu %20lld\n", value, (int64_t) value);
+ printf(" %20lu %20ld\n", value, (int64_t) value);
}
/*
@@ -254,28 +254,40 @@ void deleteStdinReader(unif01_Gen *gen) {
util_Free(gen);
}
+unsigned long getCount(int argc, char **argv) {
+ if (argc < 3) {
+ /* 2^64 - 1 == Unlimited. */
+ return -1;
+ }
+ return strtoul(argv[2], 0, 0);
+}
+
int main(int argc,
char **argv) {
- unif01_Gen *gen = createStdinReader();
- char *spec = argv[1];
-
if (argc < 2) {
printf("[ERROR] Specify test suite: '%s', '%s' or '%s'\n", TU_S, TU_C, TU_B);
exit(1);
- } else if (strcmp(spec, TU_S) == 0) {
+ }
+
+ unif01_Gen *gen = createStdinReader();
+ char *spec = argv[1];
+
+ if (strcmp(spec, TU_S) == 0) {
bbattery_SmallCrush(gen);
} else if (strcmp(spec, TU_C) == 0) {
bbattery_Crush(gen);
} else if (strcmp(spec, TU_B) == 0) {
bbattery_BigCrush(gen);
} else if (strcmp(spec, T_RAW_32) == 0) {
- /* Print to stdout until stdin closes. */
- while (1) {
+ unsigned long count = getCount(argc, argv);
+ /* Print to stdout until stdin closes or count is reached. */
+ while (count--) {
printInt(nextInt(0, gen->state));
}
} else if (strcmp(spec, T_RAW_64) == 0) {
- /* Print to stdout until stdin closes. Use dedicated 64-bit reader. */
- while (1) {
+ unsigned long count = getCount(argc, argv);
+ /* Print to stdout until stdin closes or count is reached. Use dedicated 64-bit reader. */
+ while (count--) {
printLong(nextLong(gen->state));
}
} else {
[commons-rng] 05/09: Updated stress test documentation for
PractRand.
Posted by ah...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
aherbert pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/commons-rng.git
commit 9307eacc97d9f54299afda41639349b343eaac9e
Author: aherbert <ah...@apache.org>
AuthorDate: Wed Oct 23 14:18:46 2019 +0100
Updated stress test documentation for PractRand.
---
.../examples-stress/stress_test.md | 140 +++++++++++++++++----
1 file changed, 117 insertions(+), 23 deletions(-)
diff --git a/commons-rng-examples/examples-stress/stress_test.md b/commons-rng-examples/examples-stress/stress_test.md
index dbb0d90..898d7a8 100644
--- a/commons-rng-examples/examples-stress/stress_test.md
+++ b/commons-rng-examples/examples-stress/stress_test.md
@@ -19,7 +19,8 @@ Apache Commons RNG Stress Test Example
======================================
The stress test module contains an application for calling external tools that perform stringent
-uniformity tests. The following shows an example of how to run **Dieharder** and **TestU01**.
+uniformity tests. The following shows an example of how to run **Dieharder**, **TestU01** and
+**PractRand**.
Installation on Linux/MacOS
---------------------------
@@ -32,20 +33,20 @@ This assumes that the source code has been checked out and the current working d
This can be installed from [TestU01](http://simul.iro.umontreal.ca/testu01/tu01.html) or using
the available packages (linux):
- > apt-get install libtestu01 libtestu01-dev testu01-bin
+ apt-get install libtestu01 libtestu01-dev testu01-bin
If installing **TestU01** from source then it is recommended to do this using an install directory
to avoid header clashes as the install puts headers directly into `<install directory>/include`:
- > ./configure --prefix=/usr/local/testu01
- > make
- > make install
+ ./configure --prefix=/usr/local/testu01
+ make
+ make install
A simple bridge is provided to read the integers from `stdin` and pass them to the **TestU01** library.
This can be compiled using one of the following (depending on the library names):
- > gcc src/main/c/stdin2testu01.c -o stdin2testu01 -ltestu01 -lprobdist -lmylib -lm
- > gcc src/main/c/stdin2testu01.c -o stdin2testu01 -ltestu01 \
+ gcc src/main/c/stdin2testu01.c -o stdin2testu01 -ltestu01 -lprobdist -lmylib -lm
+ gcc src/main/c/stdin2testu01.c -o stdin2testu01 -ltestu01 \
-ltestu01probdist -ltestu01mylib -lm
Note: If **TestU01** was installed from source then the `stdin2testu01.c` may have to be modified to
@@ -55,7 +56,7 @@ use different headers. Open the source and update to:
Then compile with the `<install directory>/include` and `lib` directories:
- > gcc src/main/c/stdin2testu01.c -o stdin2testu01 \
+ gcc src/main/c/stdin2testu01.c -o stdin2testu01 \
-I/usr/local/testu01/include \
-L/usr/local/testu01/lib \
-ltestu01 -lprobdist -lmylib -lm
@@ -65,12 +66,40 @@ Then compile with the `<install directory>/include` and `lib` directories:
This can be installed from [Dieharder](http://webhome.phy.duke.edu/~rgb/General/dieharder.php) or
using the available packages:
- > apt-get install dieharder
- > brew install dieharder
+ apt-get install dieharder
+ brew install dieharder
The `dieharder` executable can read integers from `stdin` for analysis so no additional steps are
required.
+### PractRand
+
+This must be installed using the source from [PractRand](http://pracrand.sourceforge.net/). These
+instructions apply to `v0.93` on linux. The latest stable build `v0.94` does not compile the
+`RNG_test` source.
+
+`PractRand` prints results to standard output. Version 0.93 did not flush the output after
+reporting test results. This prevents following the latest results and can cause
+missing results if the test aborts. This can happen due to memory allocation failures on long
+running tests which can use gigabytes of memory. A patch file is provided
+[RNG_test_0.93.patch](./src/main/resources/patch/RNG_test_0.93.patch) to flush the output.
+This can be applied from the root directory of the PractRand v0.93 source archive:
+
+ patch -p0 < RNG_test_0.93.patch
+
+Flushing of test results to output has been corrected in version 0.94+.
+
+Building **PractRand** for multi-threaded support:
+
+ g++ -std=c++14 -c src/*.cpp src/RNGs/*.cpp src/RNGs/other/*.cpp -O3 -Iinclude -pthread
+ ar rcs libPractRand.a *.o
+ rm -f *.o
+ g++ -std=c++14 -o RNG_test tools/RNG_test.cpp libPractRand.a -O3 -Iinclude -pthread
+
+Note: Earlier compilers may substitute `-std=c++14` for `-std=c++11`.
+
+The `RNG_test` tool is used to accept raw bits via standard input for testing.
+
Test platform Endianness
------------------------
@@ -84,6 +113,12 @@ The **Dieharder** application and `stdin2testu01` bridge application for the **T
test suites read binary data using a c-library function that requires the data to be in the native
byte order. These test suites will be supported automatically.
+The **PractRand** application will read binary data using the `stdin32` or `stdin64` arguments
+for 32-bit or 64-bit data. **PractRand** defaults use smart testing based on the RNG metadata to
+put special emphasis on testing the least significant bits of the input data and so 64-bit
+generators should be tested using `stdin64`. This also requires that the Commons RNG stress test
+application is run in 64-bit output mode.
+
If using a alternative test suite the endian format expected by the test suite must be verified.
More details of how to do this are provided in the [endianness](./endianness.md) page.
@@ -92,19 +127,21 @@ Running on Linux/MacOS
Build the `examples-stress` jar file:
- > mvn package -P examples-stress
+ mvn package -P examples-stress
This will create a single jar file with all the dependencies in the `target` directory.
The jar file is executable and provides usage information with the `--help` or `-h` option:
- > java -jar target/examples-stress.jar -h
+ java -jar target/examples-stress.jar -h
The list of generators that can be tested can be customised. The default is to use all known
generators that do not require arguments in addition to a random seed. To create a template list
use the `list` command:
- > java -jar target/examples-stress.jar list
+ java -jar target/examples-stress.jar list
+
+This will produce a list of generators:
# ID RandomSource trials [constructor arguments ...]
1 JDK 1
@@ -119,32 +156,89 @@ To run the stress test use the `stress` command and provide the following argume
| Argument | Description | Example |
| --------- | ----------- | ------- |
-| executable | The test tool | dieharder, stdin2testu01 |
-| ... | Arguments for the test tool | dieharder: -a -g 200 -Y 1 -k 2 <br/> stdin2testu01: SmallCrush, Crush, BigCrush |
+| executable | The test tool | dieharder, stdin2testu01, PractRand |
+| ... | Arguments for the test tool | dieharder: -a -g 200 -Y 1 -k 2 <br/> stdin2testu01: SmallCrush, Crush, BigCrush <br/> RNG_test stdin32 -tf 1 -te 0 -tlmin 1KB -tlmax 4TB |
-The command sets defaults for other options that can be changed, for example the output results
-file prefix, the number of concurrent tasks, byte-order or the number of trials per generator.
+The `stress` command defaults for other options that can be changed, for example the output
+results file prefix, the number of concurrent tasks, byte-order or the number of trials per
+generator.
Use the `--help` option to show the available options.
### TestU01
- > java -jar target/examples-stress.jar stress \
+ java -jar target/examples-stress.jar stress \
--prefix target/tu_ \
./stdin2testu01 \
BigCrush
### Dieharder
- > java -jar target/examples-stress.jar stress \
+ java -jar target/examples-stress.jar stress \
--prefix target/dh_ \
/usr/bin/dieharder \
-a -g 200 -Y 1 -k 2
+### PractRand
+
+**PractRand** must be run separately for 32-bit or 64-bit generators. This requires a subset of
+each to be specified in a list. For example to perform 3 trials of each generator:
+
+ java -jar target/examples-stress.jar list -t 3 \
+ --provider int > rng32.list
+ java -jar target/examples-stress.jar stress \
+ -l rng32.list \
+ --prefix target/pr_ \
+ --threads 3 \
+ RNG_test stdin32 \
+ -tf 1 -te 0 -tlmin 1 KB -tl max 4TB -multithreaded
+
+ java -jar target/examples-stress.jar list -t 3 \
+ --provider long > rng64.list
+ java -jar target/examples-stress.jar stress \
+ -l rng64.list \
+ --raw64
+ --prefix target/pr_ \
+ --threads 3 \
+ RNG_test stdin64 \
+ -tf 1 -te 0 -tlmin 1 KB -tl max 4TB -multithreaded
+
+Note the use of `--raw64` and ``threads 3` for the `stress` command and the `stdin32` or `stdin64` and `-multithreaded` arguments to **PractRand**.
+
+**PractRand** runs for a long time as the RNG output is tested in doubling sizes starting from 1 KB.
+The default maximum is 32 TB of RNG output. At each doubling of data an interim test analysis is
+performed. The tests compare the current aggregated output to previous tests. Thus memory
+consumption grows with the test duration. Trials have shown the following:
+
+| Argument | Max memory (KB) |
+| --------- | ---------- |
+| stdin32 -tf 1 -te 0 -tlmax 1TB | 732792 |
+| stdin32 -tf 2 -te 0 -tlmax 1TB | 1271424 |
+| stdin64 -tf 1 -te 0 -tlmax 1TB | 800148 |
+| stdin64 -tf 2 -te 0 -tlmax 1TB | 1304116 |
+| stdin32 -tf 1 -te 0 -tlmax 4TB | ? |
+| stdin32 -tf 2 -te 0 -tlmax 4TB | 2314232 |
+| stdin64 -tf 1 -te 0 -tlmax 4TB | ? |
+| stdin64 -tf 2 -te 0 -tlmax 4TB | ? |
+
+Given the large memory consumption on long running tests the machine resources may be better used
+when RAM limited by running **PractRand** multi-threaded instead of running more single-threaded
+concurrent tasks.
+**PractRand** -multithreaded mode has an approximate 3-fold speed up although thread use fluctuates
+from 1 to 5 threads. Allowing for an extra thread to control the java RNG this requires
+approximately 3 threads per **PractRand** and 4 in total per test task. By default the `stress`
+command will use 1 thread per test application. The number of threads can be increased using the
+``--threads` option. Note that the `stress` command cannot allocate threads to the process.
+Instead it adjusts the number of concurrent tasks it will execute as
+ceil(processors / threadsPerTask).
+
+### Viewing results
+
The output results can be viewed using the `results` command:
- > java -jar target/examples-stress.jar results \
+ java -jar target/examples-stress.jar results \
target/tu_* \
- target/dh_*
+ target/dh_* \
+ target/pr_* \
Various formats are available. Use the `--help` option to show the available options.
@@ -160,13 +254,13 @@ byte-reversed if desired.
The `list` command can output available generators by provider type. For example to output the
64-bit providers to a `rng64.list` file:
- > java -jar target/examples-stress.jar list \
+ java -jar target/examples-stress.jar list \
--provider long > rng64.list
The list can then be used to test 64-bit providers, for example to test the lower 32-bits in reverse
order using **BigCrush**:
- > java -jar target/examples-stress.jar stress \
+ java -jar target/examples-stress.jar stress \
--list rng64.list \
--low-bits \
--reverse-bits \
[commons-rng] 07/09: Update endianness documentation.
Posted by ah...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
aherbert pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/commons-rng.git
commit 3c6751af94d1221a3f04222bd85b5a0d2ddb04a6
Author: aherbert <ah...@apache.org>
AuthorDate: Wed Oct 23 15:29:07 2019 +0100
Update endianness documentation.
---
commons-rng-examples/examples-stress/endianness.md | 110 +++++++++++++++++----
1 file changed, 90 insertions(+), 20 deletions(-)
diff --git a/commons-rng-examples/examples-stress/endianness.md b/commons-rng-examples/examples-stress/endianness.md
index be4e570..56567f8 100644
--- a/commons-rng-examples/examples-stress/endianness.md
+++ b/commons-rng-examples/examples-stress/endianness.md
@@ -18,16 +18,20 @@
Apache Commons RNG Stress Test Endianness
=========================================
-The stress test application will output raw binary data for generated integers. An integer is 4
-bytes and thus the byte order or [Endianness](https://en.wikipedia.org/wiki/Endianness) of the data
+The stress test application will output raw binary data for generated numbers. An integer is 4
+bytes and a long is 8 bytes and thus the byte order or
+[Endianness](https://en.wikipedia.org/wiki/Endianness) of the data
must be correct for the test application. The stress test application can support either big-endian
or little-endian format. The application will auto-detect the platform and will default to output
binary data using the native byte order.
You can determine the native platform format using the stress test application `endian` command:
- > mvn package -P examples-stress
- > java -jar target/examples-stress.jar endian
+ mvn package -P examples-stress
+ java -jar target/examples-stress.jar endian
+
+This will output:
+
[INFO] LITTLE_ENDIAN
The **Dieharder** application and `stdin2testu01` bridge application for the **TestU01**
@@ -36,7 +40,9 @@ byte order. These test suites will be supported automatically.
The following sections show examples of verifying the endianness for a test application. The first
uses **Dieharder** which can read test data in both text and binary format. The second uses the
-`stdin2testu01` bridge application to verify data passed to an application using a C API is correct.
+`stdin2testu01` bridge application to verify data passed to an application using a C API is
+correct. The third verifies that any data tests using the `output` command are applicable to
+the `stress` command.
Testing using file output
-------------------------
@@ -47,17 +53,17 @@ application to demonstrate the result.
Build the `examples-stress` jar file:
- > mvn package -P examples-stress
+ mvn package -P examples-stress
This will create a single jar file with all the dependencies in the `target` directory. The program
contains a simple command to output data from a random generator. To output data in both text
and binary format use the following commands:
- > java -jar target/examples-stress.jar output SPLIT_MIX_64 -s 1 -n 100000 \
+ java -jar target/examples-stress.jar output SPLIT_MIX_64 -s 1 -n 100000 \
-f DIEHARDER -o test.dh
- > java -jar target/examples-stress.jar output SPLIT_MIX_64 -s 1 -n 100000 \
+ java -jar target/examples-stress.jar output SPLIT_MIX_64 -s 1 -n 100000 \
-f BINARY -o test.big -b big_endian
- > java -jar target/examples-stress.jar output SPLIT_MIX_64 -s 1 -n 100000 \
+ java -jar target/examples-stress.jar output SPLIT_MIX_64 -s 1 -n 100000 \
-f BINARY -o test.little -b little_endian
This should produce the following output files:
@@ -70,9 +76,9 @@ This should produce the following output files:
The data can then be used to run a test within **Dieharder**:
- > dieharder -g 202 -d 0 -f test.dh
- > dieharder -g 201 -d 0 -f test.little
- > dieharder -g 201 -d 0 -f test.big
+ dieharder -g 202 -d 0 -f test.dh
+ dieharder -g 201 -d 0 -f test.little
+ dieharder -g 201 -d 0 -f test.big
Only one of the raw binary files should match the result from the text file. This is the correct
endianness.
@@ -83,17 +89,17 @@ written direct to dieharder. For the birthdays test (`-d 0` option) 20,000,000 s
In the following example the stress test application directly writes to `stdout` which is then
piped to the `dieharder` application which reads using `stdin` (`-g 200` option):
- > java -jar target/examples-stress.jar output SPLIT_MIX_64 -s 1 -n 20000000 \
+ java -jar target/examples-stress.jar output SPLIT_MIX_64 -s 1 -n 20000000 \
-f DIEHARDER -o test.dh
- > dieharder -g 202 -d 0 -f test.dh
- > java -jar target/examples-stress.jar output SPLIT_MIX_64 -s 1 -n 20000000 \
+ dieharder -g 202 -d 0 -f test.dh
+ java -jar target/examples-stress.jar output SPLIT_MIX_64 -s 1 -n 20000000 \
-f BINARY -b little_endian | dieharder -g 200 -d 0
If the results are not the same then the second command can be repeated with `-b big_endian`.
Remember to clean up the output files:
- > rm -f test.dh test.big test.little
+ rm -f test.dh test.big test.little
Testing using the Java to c bridge
----------------------------------
@@ -105,7 +111,7 @@ with a `c` language API.
Build the `examples-stress` jar file:
- > mvn package -P examples-stress
+ mvn package -P examples-stress
This will create a single jar file with all the dependencies in the `target` directory. The program
contains a simple test for the bridge between the Java application and a sub-process. This will
@@ -119,7 +125,7 @@ application are provided in the [stress test](./stress_test.md) page.
The platform endianess is auto-detected by the Java application. To run the bridge test
use the following command:
- > java -jar target/examples-stress.jar bridge ./stdin2testu01 raw32
+ java -jar target/examples-stress.jar bridge ./stdin2testu01 raw32
This should produce the following output files:
@@ -149,7 +155,7 @@ If the data has been correctly read the `bridge.data` and `bridge.out` should ma
If the endianess is incorrect then the data sent by the Java application will not match the
data read by the sub-process. For example to swap the endianness use the `-b` option:
- > java -jar target/examples-stress.jar bridge -b BIG_ENDIAN ./stdin2testu01 raw32
+ java -jar target/examples-stress.jar bridge -b BIG_ENDIAN ./stdin2testu01 raw32
In this case the little-endian plaform has been sent big-endian data and the contents of the
`bridge.out` file begin with the reverse byte order:
@@ -164,4 +170,68 @@ values do not match; the sub-process has incorrectly received the data.
Remember to clean up the output files:
- > rm -f bridge.data bridge.out bridge.err
+ rm -f bridge.data bridge.out bridge.err
+
+Testing the stress command
+--------------------------
+
+The `stress` command will open the provided application and send binary data. The data transfer
+modes should match the equivalent modes in the `output` command. However the `output` command
+can be used to output more formats and used to test data transfer to a test application that
+accepts different input (such as the **Dieharder** text and binary format).
+
+To verify that the `stress` command is sending the same data as the `output` command requires
+piping data from each to a program. This example uses `stdin2testu01` with the raw mode.
+
+Create a list for the `stress` command:
+
+ java -jar target/examples-stress.jar list | grep SPLIT_MIX_64 > rng.list
+
+Run the `stress` command:
+
+ java -jar target/examples-stress.jar stress -l rng.list \
+ -x 012345 \
+ --prefix stress_ \
+ -o overwrite \
+ ./stdin2testu01 raw32 10
+
+The `overwrite` mode will replace existing files and is used here to allow repeat testing.
+
+Run the equivalent `output` command:
+
+ java -jar target/examples-stress.jar output SPLIT_MIX_64 \
+ -x 012345 \
+ -f binary \
+ -b little_endian \
+ | ./stdin2testu01 raw32 10 > output.data
+
+Note the use of the same hex seed with the `-x` option.
+
+Compare the data in `stress_10_1` and `output.data`.
+The number data in the file should be identical if the endianness was correctly configured.
+
+ diff stress_10_1 output.data
+
+The only difference should be the the header and footer added by the `stress` command
+to the results file. If the endianness is incorrect for the `output` command then the number
+data will not match. Note that the `output` command by default uses big endian for consistency
+across all platforms.
+
+An equivalent 64-bit output would use the `--raw64` option for each command:
+
+ java -jar target/examples-stress.jar stress -l rng.list \
+ -x 012345 \
+ --prefix stress_ \
+ -o overwrite \
+ --raw64 \
+ ./stdin2testu01 raw64 10
+ java -jar target/examples-stress.jar output SPLIT_MIX_64 \
+ -x 012345 \
+ -f binary \
+ -b little_endian \
+ --raw64 \
+ | ./stdin2testu01 raw64 10 > output.data
+
+Remember to clean up the output files:
+
+ rm -f rng.list stress_10_1 output.data
[commons-rng] 01/09: Add hex seed option to stress command.
Posted by ah...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
aherbert pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/commons-rng.git
commit b63f5f0d8082f880cb94b9ed3dbd0249554767e9
Author: aherbert <ah...@apache.org>
AuthorDate: Wed Oct 23 11:26:28 2019 +0100
Add hex seed option to stress command.
Allows the output command and stress command to stream the same data.
Also allows tests to be repeated.
---
.../commons/rng/examples/stress/OutputCommand.java | 2 +-
.../rng/examples/stress/StressTestCommand.java | 27 +++++++++++++++++++++-
2 files changed, 27 insertions(+), 2 deletions(-)
diff --git a/commons-rng-examples/examples-stress/src/main/java/org/apache/commons/rng/examples/stress/OutputCommand.java b/commons-rng-examples/examples-stress/src/main/java/org/apache/commons/rng/examples/stress/OutputCommand.java
index b27b197..305f957 100644
--- a/commons-rng-examples/examples-stress/src/main/java/org/apache/commons/rng/examples/stress/OutputCommand.java
+++ b/commons-rng-examples/examples-stress/src/main/java/org/apache/commons/rng/examples/stress/OutputCommand.java
@@ -97,7 +97,7 @@ class OutputCommand implements Callable<Void> {
/** The random seed as a byte[]. */
@Option(names = {"-x", "--hex-seed"},
description = {"The hex-encoded random seed.",
- "Bytes for other primitives use little-endian format.",
+ "Seed conversion for multi-byte primitives use little-endian format.",
"Over-rides the --seed parameter."})
private String byteSeed;
diff --git a/commons-rng-examples/examples-stress/src/main/java/org/apache/commons/rng/examples/stress/StressTestCommand.java b/commons-rng-examples/examples-stress/src/main/java/org/apache/commons/rng/examples/stress/StressTestCommand.java
index a963ede..53b0ec0 100644
--- a/commons-rng-examples/examples-stress/src/main/java/org/apache/commons/rng/examples/stress/StressTestCommand.java
+++ b/commons-rng-examples/examples-stress/src/main/java/org/apache/commons/rng/examples/stress/StressTestCommand.java
@@ -160,6 +160,13 @@ class StressTestCommand implements Callable<Void> {
"generators sequentially, each appropriately byte reversed for the platform."})
private boolean raw64;
+ /** The random seed as a byte[]. */
+ @Option(names = {"-x", "--hex-seed"},
+ description = {"The hex-encoded random seed.",
+ "Seed conversion for multi-byte primitives use little-endian format.",
+ "Use to repeat tests. Not recommended for batch testing."})
+ private String byteSeed;
+
/**
* Flag to indicate the output should be combined with a hashcode from a new object.
* This is a method previously used in the
@@ -456,7 +463,7 @@ class StressTestCommand implements Callable<Void> {
}
}
// Create the generator. Explicitly create a seed so it can be recorded.
- final byte[] seed = testData.getRandomSource().createSeed();
+ final byte[] seed = createSeed(testData.getRandomSource());
UniformRandomProvider rng = testData.createRNG(seed);
// Upper or lower bits from 64-bit generators must be created first.
@@ -498,6 +505,24 @@ class StressTestCommand implements Callable<Void> {
}
/**
+ * Creates the seed. This will call {@link RandomSource#createSeed()} unless a hex seed has
+ * been explicitly specified on the command line.
+ *
+ * @param randomSource Random source.
+ * @return the seed
+ */
+ private byte[] createSeed(RandomSource randomSource) {
+ if (byteSeed != null) {
+ try {
+ return Hex.decodeHex(byteSeed);
+ } catch (IllegalArgumentException ex) {
+ throw new ApplicationException("Invalid hex seed: " + ex.getMessage(), ex);
+ }
+ }
+ return randomSource.createSeed();
+ }
+
+ /**
* Submit the tasks to the executor service.
*
* @param service The executor service.
[commons-rng] 08/09: Update stress test documentation.
Posted by ah...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
aherbert pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/commons-rng.git
commit ede9cf4f18b242f51fe72eb7995f9e50ff9c4c1b
Author: aherbert <ah...@apache.org>
AuthorDate: Wed Oct 23 15:42:19 2019 +0100
Update stress test documentation.
Added example memory usage figures.
Updated 64-bit generator testing section.
---
commons-rng-examples/examples-stress/stress_test.md | 21 ++++++++++++++-------
1 file changed, 14 insertions(+), 7 deletions(-)
diff --git a/commons-rng-examples/examples-stress/stress_test.md b/commons-rng-examples/examples-stress/stress_test.md
index 898d7a8..054113a 100644
--- a/commons-rng-examples/examples-stress/stress_test.md
+++ b/commons-rng-examples/examples-stress/stress_test.md
@@ -207,7 +207,8 @@ Note the use of `--raw64` and ``threads 3` for the `stress` command and the `std
**PractRand** runs for a long time as the RNG output is tested in doubling sizes starting from 1 KB.
The default maximum is 32 TB of RNG output. At each doubling of data an interim test analysis is
performed. The tests compare the current aggregated output to previous tests. Thus memory
-consumption grows with the test duration. Trials have shown the following:
+consumption grows with the test duration. Trials have shown the following maximum memory consumption
+when watching the **PractRand** process:
| Argument | Max memory (KB) |
| --------- | ---------- |
@@ -217,7 +218,7 @@ consumption grows with the test duration. Trials have shown the following:
| stdin64 -tf 2 -te 0 -tlmax 1TB | 1304116 |
| stdin32 -tf 1 -te 0 -tlmax 4TB | ? |
| stdin32 -tf 2 -te 0 -tlmax 4TB | 2314232 |
-| stdin64 -tf 1 -te 0 -tlmax 4TB | ? |
+| stdin64 -tf 1 -te 0 -tlmax 4TB | 1324436 |
| stdin64 -tf 2 -te 0 -tlmax 4TB | ? |
Given the large memory consumption on long running tests the machine resources may be better used
@@ -245,11 +246,17 @@ Various formats are available. Use the `--help` option to show the available opt
Test 64-bit generators
----------------------
-The available random generators output either 32-bits or 64-bits per cycle. The test applications
-**Dieharder** and **TestU01** require 32-bit input. The standard method for a 64-bit generator is
-to use the upper and then lower 32-bits of each 64-bit output. The stress test application has
-options to use only the upper or the lower 32-bits for testing. These can then be bit-reversed or
-byte-reversed if desired.
+The available random generators output either 32-bits or 64-bits per cycle.
+
+Any application that supports 64-bit generators and should be tested using the `--raw64`
+output mode of the `stress` command. See the example for running tests using **PractRand**
+for details.
+
+Any application that supports 32-bit generators can be tested using different subsets of the
+64-bit output. For example the test applications **Dieharder** and **TestU01** require 32-bit input.
+The standard method for a 64-bit generator is to use the upper and then lower 32-bits of each
+64-bit output. The stress test application has options to use only the upper or the lower 32-bits
+for testing. These can then be bit-reversed or byte-reversed if desired.
The `list` command can output available generators by provider type. For example to output the
64-bit providers to a `rng64.list` file:
[commons-rng] 09/09: Updated PractRand patch files to fix linux
compilation.
Posted by ah...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
aherbert pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/commons-rng.git
commit 4679139fd899bf510dad7da12f5a7c92db1af5fe
Author: aherbert <ah...@apache.org>
AuthorDate: Wed Oct 23 17:53:01 2019 +0100
Updated PractRand patch files to fix linux compilation.
---
.gitattributes | 2 +
.../main/resources/patches/PractRand_0.93.patch | 33 ++++++++++++++
.../main/resources/patches/PractRand_0.94.patch | 53 ++++++++++++++++++++++
.../src/main/resources/patches/RNG_test_0.93.patch | 10 ----
.../examples-stress/stress_test.md | 25 ++++++----
5 files changed, 103 insertions(+), 20 deletions(-)
diff --git a/.gitattributes b/.gitattributes
new file mode 100644
index 0000000..db173de
--- /dev/null
+++ b/.gitattributes
@@ -0,0 +1,2 @@
+* text=auto
+*.patch -text
diff --git a/commons-rng-examples/examples-stress/src/main/resources/patches/PractRand_0.93.patch b/commons-rng-examples/examples-stress/src/main/resources/patches/PractRand_0.93.patch
new file mode 100644
index 0000000..c39491d
--- /dev/null
+++ b/commons-rng-examples/examples-stress/src/main/resources/patches/PractRand_0.93.patch
@@ -0,0 +1,33 @@
+--- tools/RNG_test.cpp 2019-10-23 12:01:43.168306859 +0100
++++ tools/RNG_test.new.cpp 2019-10-23 12:02:32.696759305 +0100
+@@ -500,6 +500,7 @@
+ else
+ std::printf(" ...and %d test result(s) without anomalies\n", int(results.size() - marked.size()));
+ std::printf("\n");
++ std::fflush(stdout);
+ if (end_on_failure && biggest_decimal_suspicion > 8.5) std::exit(0);
+ }
+ double interpret_length(const std::string &lengthstr, bool normal_mode) {
+--- tools/RNG_from_name.h.orig 2019-10-23 17:14:27.443993007 +0100
++++ tools/RNG_from_name.h 2019-10-23 17:30:21.011881702 +0100
+@@ -8,7 +8,7 @@
+ std::fprintf(stderr, "error reading standard input\n");
+ std::exit(0);
+ }
+- enum { BUFF_SIZE = 4096 / sizeof(Word) };
++ enum { BUFF_SIZE = 8192 / sizeof(Word) };
+ Word *pos, *end;
+ bool ended;
+ Word buffer[BUFF_SIZE];
+@@ -19,7 +19,10 @@
+ end = &buffer[n];
+ }
+ public:
+- _stdin_reader() : ended(false) { refill(); }
++ _stdin_reader() : ended(false) {
++ setvbuf(stdin, 0, _IOFBF, 8192);
++ refill();
++ }
+ Word read() { Word rv = *(pos++); if (pos == end) refill(); return rv; }
+ };
+ class RNG_stdin : public PractRand::RNGs::vRNG8 {
diff --git a/commons-rng-examples/examples-stress/src/main/resources/patches/PractRand_0.94.patch b/commons-rng-examples/examples-stress/src/main/resources/patches/PractRand_0.94.patch
new file mode 100644
index 0000000..5904782
--- /dev/null
+++ b/commons-rng-examples/examples-stress/src/main/resources/patches/PractRand_0.94.patch
@@ -0,0 +1,53 @@
+--- src/test_batteries.cpp.orig 2019-10-23 16:21:03.850440174 +0100
++++ src/test_batteries.cpp 2019-10-23 16:21:12.538517226 +0100
+@@ -25,8 +25,8 @@
+ #include "PractRand/Tests/CoupGap.h"
+ #include "PractRand/Tests/BRank.h"
+ #include "PractRand/Tests/mod3.h"
+-#include "PractRand/Tests/NearSeq.h"
+-#include "PractRand/Tests/Coup16.h"
++#include "PractRand/Tests/nearseq.h"
++#include "PractRand/Tests/coup16.h"
+ #include "PractRand/Tests/DistFreq4.h"
+ #include "PractRand/Tests/transforms.h"
+
+--- tools/RNG_test.cpp.orig 2019-10-23 16:24:21.040189344 +0100
++++ tools/RNG_test.cpp 2019-10-23 16:24:50.280448774 +0100
+@@ -453,10 +453,10 @@
+ return true;
+ }
+
+-#include "PractRand/tests/Birthday.h"
+-#include "PractRand/tests/FPMulti.h"
+-#include "PractRand/tests/DistFreq4.h"
+-#include "PractRand/tests/Gap16.h"
++#include "PractRand/Tests/birthday.h"
++#include "PractRand/Tests/FPMulti.h"
++#include "PractRand/Tests/DistFreq4.h"
++#include "PractRand/Tests/Gap16.h"
+ PractRand::Tests::ListOfTests testset_BirthdaySystematic() {
+ //return PractRand::Tests::ListOfTests(new PractRand::Tests::FPMulti(3,0));
+ //return PractRand::Tests::ListOfTests(new PractRand::Tests::BirthdayAlt(10), new PractRand::Tests::Birthday32());
+--- tools/RNG_from_name.h.orig 2019-10-23 17:42:13.570076059 +0100
++++ tools/RNG_from_name.h 2019-10-23 17:42:33.774253293 +0100
+@@ -8,7 +8,7 @@
+ std::fprintf(stderr, "error reading standard input\n");
+ std::exit(0);
+ }
+- enum { BUFF_SIZE = 4096 / sizeof(Word) };
++ enum { BUFF_SIZE = 8096 / sizeof(Word) };
+ Word *pos, *end;
+ bool ended;
+ Word buffer[BUFF_SIZE];
+@@ -20,7 +20,10 @@
+ end = &buffer[n];
+ }
+ public:
+- _stdin_reader() : ended(false) { refill(); }
++ _stdin_reader() : ended(false) {
++ setvbuf(stdin, 0, _IOFBF, 8192);
++ refill();
++ }
+ Word read() { if (pos == end) refill(); return *(pos++); }
+ };
+ class RNG_stdin : public PractRand::RNGs::vRNG8 {
diff --git a/commons-rng-examples/examples-stress/src/main/resources/patches/RNG_test_0.93.patch b/commons-rng-examples/examples-stress/src/main/resources/patches/RNG_test_0.93.patch
deleted file mode 100644
index a71705c..0000000
--- a/commons-rng-examples/examples-stress/src/main/resources/patches/RNG_test_0.93.patch
+++ /dev/null
@@ -1,10 +0,0 @@
---- tools/RNG_test.cpp 2019-10-23 12:01:43.168306859 +0100
-+++ tools/RNG_test.new.cpp 2019-10-23 12:02:32.696759305 +0100
-@@ -500,6 +500,7 @@
- else
- std::printf(" ...and %d test result(s) without anomalies\n", int(results.size() - marked.size()));
- std::printf("\n");
-+ std::fflush(stdout);
- if (end_on_failure && biggest_decimal_suspicion > 8.5) std::exit(0);
- }
- double interpret_length(const std::string &lengthstr, bool normal_mode) {
diff --git a/commons-rng-examples/examples-stress/stress_test.md b/commons-rng-examples/examples-stress/stress_test.md
index 054113a..3d0b451 100644
--- a/commons-rng-examples/examples-stress/stress_test.md
+++ b/commons-rng-examples/examples-stress/stress_test.md
@@ -75,19 +75,24 @@ required.
### PractRand
This must be installed using the source from [PractRand](http://pracrand.sourceforge.net/). These
-instructions apply to `v0.93` on linux. The latest stable build `v0.94` does not compile the
-`RNG_test` source.
+instructions apply on linux using the v0.93 and v0.94 versions. Patch files have been generated
+to correct issues with the source for compilation using `g++`. The patches also includes an
+increase to the buffer size for reading from standard input to match the buffer size used by the
+`stress` command.
-`PractRand` prints results to standard output. Version 0.93 did not flush the output after
-reporting test results. This prevents following the latest results and can cause
-missing results if the test aborts. This can happen due to memory allocation failures on long
-running tests which can use gigabytes of memory. A patch file is provided
-[RNG_test_0.93.patch](./src/main/resources/patch/RNG_test_0.93.patch) to flush the output.
-This can be applied from the root directory of the PractRand v0.93 source archive:
+Version 0.93 did not flush the standard output after reporting test results. This prevents
+following the latest results and can cause missing results if the test aborts. This can be
+corrected using [PractRand_0.93.patch](./src/main/resources/patch/PractRand_0.93.patch)
+from the root directory of the PractRand v0.93 source archive:
- patch -p0 < RNG_test_0.93.patch
+ patch -p0 < PractRand_0.93.patch
-Flushing of test results to output has been corrected in version 0.94+.
+Version 0.94 does not compile on linux due to mismatch between include statements and filenames.
+Case insensitive include filenames are allowed using windows compilers but not linux compilers.
+This can be corrected using [PractRand_0.94.patch](./src/main/resources/patch/PractRand_0.94.patch)
+from the root directory of the PractRand v0.94 source archive:
+
+ patch -p0 < PractRand_0.94.patch
Building **PractRand** for multi-threaded support: