You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by GitBox <gi...@apache.org> on 2022/04/20 20:12:53 UTC
[GitHub] [nifi] exceptionfactory opened a new pull request, #5984: NIFI-9942 Remove load test from TestPutUDP
exceptionfactory opened a new pull request, #5984:
URL: https://github.com/apache/nifi/pull/5984
#### Description of PR
NIFI-9942 Removes the load test method from `TestPutUDP` to avoid intermittent failures on automated builds. Additional changes include refactoring the test to use JUnit 5 and standardizing test method names.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [X] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [X] Does your PR title start with **NIFI-XXXX** where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
- [X] Has your PR been rebased against the latest commit within the target branch (typically `main`)?
- [X] Is your initial contribution a single, squashed commit? _Additional commits in response to PR reviewer feedback should be made on this branch and pushed to allow change tracking. Do not `squash` or use `--force` when pushing to allow for clean monitoring of changes._
### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via `mvn -Pcontrib-check clean install` at the root `nifi` folder?
- [X] Have you written or updated unit tests to verify your changes?
- [ ] Have you verified that the full build is successful on JDK 8?
- [ ] Have you verified that the full build is successful on JDK 11?
- [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE` file, including the main `LICENSE` file under `nifi-assembly`?
- [ ] If applicable, have you updated the `NOTICE` file, including the main `NOTICE` file found under `nifi-assembly`?
- [ ] If adding new Properties, have you added `.displayName` in addition to .name (programmatic access) for each of the new properties?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in which it is rendered?
### Note:
Please ensure that once the PR is submitted, you check GitHub Actions CI for build issues and submit an update to your PR as soon as possible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] greyp9 closed pull request #5984: NIFI-9942 Remove load test from TestPutUDP
Posted by GitBox <gi...@apache.org>.
greyp9 closed pull request #5984: NIFI-9942 Remove load test from TestPutUDP
URL: https://github.com/apache/nifi/pull/5984
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] greyp9 commented on a diff in pull request #5984: NIFI-9942 Remove load test from TestPutUDP
Posted by GitBox <gi...@apache.org>.
greyp9 commented on code in PR #5984:
URL: https://github.com/apache/nifi/pull/5984#discussion_r854534196
##########
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/TestPutUDP.java:
##########
@@ -26,152 +26,107 @@
import org.apache.nifi.remote.io.socket.NetworkUtils;
import org.apache.nifi.util.TestRunner;
import org.apache.nifi.util.TestRunners;
-import org.junit.After;
-import org.junit.Before;
-import org.junit.Test;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.Timeout;
import java.net.InetAddress;
import java.nio.charset.Charset;
import java.nio.charset.StandardCharsets;
import java.util.concurrent.BlockingQueue;
import java.util.concurrent.LinkedBlockingQueue;
-import static org.junit.Assert.assertArrayEquals;
-import static org.junit.Assert.assertNotNull;
-import static org.junit.Assert.assertNull;
+import static org.junit.jupiter.api.Assertions.assertArrayEquals;
+import static org.junit.jupiter.api.Assertions.assertNotNull;
+import static org.junit.jupiter.api.Assertions.assertNull;
+@Timeout(10)
public class TestPutUDP {
private final static String UDP_SERVER_ADDRESS = "127.0.0.1";
- private final static String SERVER_VARIABLE = "ALKJAFLKJDFLSKJSDFLKJSDF";
- private final static String UDP_SERVER_ADDRESS_EL = "${" + SERVER_VARIABLE + "}";
+ private final static String SERVER_VARIABLE = "SERVER";
private static final String DELIMITER = "\n";
private static final Charset CHARSET = StandardCharsets.UTF_8;
private final static int MAX_FRAME_LENGTH = 32800;
private final static int VALID_LARGE_FILE_SIZE = 32768;
- private final static int VALID_SMALL_FILE_SIZE = 64;
private final static int INVALID_LARGE_FILE_SIZE = 1_000_000;
- private final static int LOAD_TEST_ITERATIONS = 500;
- private final static int LOAD_TEST_THREAD_COUNT = 1;
- private final static int DEFAULT_ITERATIONS = 1;
- private final static int DEFAULT_THREAD_COUNT = 1;
private final static char CONTENT_CHAR = 'x';
private final static int DATA_WAIT_PERIOD = 50;
- private final static int DEFAULT_TEST_TIMEOUT_PERIOD = 10000;
- private final static int LONG_TEST_TIMEOUT_PERIOD = 30000;
+ private final static String[] EMPTY_FILE = { "" };
+ private final static String[] VALID_FILES = { "FIRST", "SECOND", "12345678", "343424222", "!@£$%^&*()_+:|{}[];\\" };
private TestRunner runner;
private int port;
private EventServer eventServer;
private BlockingQueue<ByteArrayMessage> messages;
-
- // Test Data
- private final static String[] EMPTY_FILE = { "" };
- private final static String[] VALID_FILES = { "abcdefghijklmnopqrstuvwxyz", "zyxwvutsrqponmlkjihgfedcba", "12345678", "343424222", "!@£$%^&*()_+:|{}[];\\" };
-
- @Before
+ @BeforeEach
public void setup() throws Exception {
runner = TestRunners.newTestRunner(PutUDP.class);
runner.setVariable(SERVER_VARIABLE, UDP_SERVER_ADDRESS);
port = NetworkUtils.getAvailableUdpPort();
createTestServer(port, VALID_LARGE_FILE_SIZE);
}
- private void createTestServer(final int port, final int frameSize) throws Exception {
- messages = new LinkedBlockingQueue<>();
- final byte[] delimiter = DELIMITER.getBytes(CHARSET);
- final InetAddress listenAddress = InetAddress.getByName(UDP_SERVER_ADDRESS);
- NettyEventServerFactory serverFactory = new ByteArrayMessageNettyEventServerFactory(
- runner.getLogger(), listenAddress, port, TransportProtocol.UDP, delimiter, frameSize, messages);
- serverFactory.setSocketReceiveBuffer(MAX_FRAME_LENGTH);
- serverFactory.setShutdownQuietPeriod(ShutdownQuietPeriod.QUICK.getDuration());
- serverFactory.setShutdownTimeout(ShutdownTimeout.QUICK.getDuration());
- eventServer = serverFactory.getEventServer();
- }
-
- @After
+ @AfterEach
public void cleanup() {
runner.shutdown();
removeTestServer();
}
- private void removeTestServer() {
- if (eventServer != null) {
- eventServer.shutdown();
- eventServer = null;
- }
- }
-
- @Test(timeout = DEFAULT_TEST_TIMEOUT_PERIOD)
- public void testValidFiles() throws Exception {
- configureProperties(UDP_SERVER_ADDRESS);
- sendTestData(VALID_FILES);
- checkReceivedAllData(VALID_FILES);
- checkInputQueueIsEmpty();
- }
-
- @Test(timeout = DEFAULT_TEST_TIMEOUT_PERIOD)
- public void testValidFilesEL() throws Exception {
- configureProperties(UDP_SERVER_ADDRESS_EL);
- sendTestData(VALID_FILES);
- checkReceivedAllData(VALID_FILES);
- checkInputQueueIsEmpty();
+ @Test
+ public void testSend() throws Exception {
+ configureProperties();
+ sendMessages(VALID_FILES);
+ assertMessagesReceived(VALID_FILES);
+ runner.assertQueueEmpty();
}
- @Test(timeout = DEFAULT_TEST_TIMEOUT_PERIOD)
- public void testEmptyFile() throws Exception {
- configureProperties(UDP_SERVER_ADDRESS);
- sendTestData(EMPTY_FILE);
+ @Test
+ public void testSendEmptyFile() throws Exception {
+ configureProperties();
+ sendMessages(EMPTY_FILE);
checkRelationships(EMPTY_FILE.length, 0);
Review Comment:
maybe this should just be:
```
checkRelationships(0, 0);
```
##########
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/TestPutUDP.java:
##########
@@ -26,152 +26,107 @@
import org.apache.nifi.remote.io.socket.NetworkUtils;
import org.apache.nifi.util.TestRunner;
import org.apache.nifi.util.TestRunners;
-import org.junit.After;
-import org.junit.Before;
-import org.junit.Test;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.Timeout;
import java.net.InetAddress;
import java.nio.charset.Charset;
import java.nio.charset.StandardCharsets;
import java.util.concurrent.BlockingQueue;
import java.util.concurrent.LinkedBlockingQueue;
-import static org.junit.Assert.assertArrayEquals;
-import static org.junit.Assert.assertNotNull;
-import static org.junit.Assert.assertNull;
+import static org.junit.jupiter.api.Assertions.assertArrayEquals;
+import static org.junit.jupiter.api.Assertions.assertNotNull;
+import static org.junit.jupiter.api.Assertions.assertNull;
+@Timeout(10)
public class TestPutUDP {
private final static String UDP_SERVER_ADDRESS = "127.0.0.1";
- private final static String SERVER_VARIABLE = "ALKJAFLKJDFLSKJSDFLKJSDF";
- private final static String UDP_SERVER_ADDRESS_EL = "${" + SERVER_VARIABLE + "}";
+ private final static String SERVER_VARIABLE = "SERVER";
private static final String DELIMITER = "\n";
private static final Charset CHARSET = StandardCharsets.UTF_8;
private final static int MAX_FRAME_LENGTH = 32800;
private final static int VALID_LARGE_FILE_SIZE = 32768;
- private final static int VALID_SMALL_FILE_SIZE = 64;
private final static int INVALID_LARGE_FILE_SIZE = 1_000_000;
- private final static int LOAD_TEST_ITERATIONS = 500;
- private final static int LOAD_TEST_THREAD_COUNT = 1;
- private final static int DEFAULT_ITERATIONS = 1;
- private final static int DEFAULT_THREAD_COUNT = 1;
private final static char CONTENT_CHAR = 'x';
private final static int DATA_WAIT_PERIOD = 50;
- private final static int DEFAULT_TEST_TIMEOUT_PERIOD = 10000;
- private final static int LONG_TEST_TIMEOUT_PERIOD = 30000;
+ private final static String[] EMPTY_FILE = { "" };
+ private final static String[] VALID_FILES = { "FIRST", "SECOND", "12345678", "343424222", "!@£$%^&*()_+:|{}[];\\" };
private TestRunner runner;
private int port;
private EventServer eventServer;
private BlockingQueue<ByteArrayMessage> messages;
-
- // Test Data
- private final static String[] EMPTY_FILE = { "" };
- private final static String[] VALID_FILES = { "abcdefghijklmnopqrstuvwxyz", "zyxwvutsrqponmlkjihgfedcba", "12345678", "343424222", "!@£$%^&*()_+:|{}[];\\" };
-
- @Before
+ @BeforeEach
public void setup() throws Exception {
runner = TestRunners.newTestRunner(PutUDP.class);
runner.setVariable(SERVER_VARIABLE, UDP_SERVER_ADDRESS);
port = NetworkUtils.getAvailableUdpPort();
createTestServer(port, VALID_LARGE_FILE_SIZE);
}
- private void createTestServer(final int port, final int frameSize) throws Exception {
- messages = new LinkedBlockingQueue<>();
- final byte[] delimiter = DELIMITER.getBytes(CHARSET);
- final InetAddress listenAddress = InetAddress.getByName(UDP_SERVER_ADDRESS);
- NettyEventServerFactory serverFactory = new ByteArrayMessageNettyEventServerFactory(
- runner.getLogger(), listenAddress, port, TransportProtocol.UDP, delimiter, frameSize, messages);
- serverFactory.setSocketReceiveBuffer(MAX_FRAME_LENGTH);
- serverFactory.setShutdownQuietPeriod(ShutdownQuietPeriod.QUICK.getDuration());
- serverFactory.setShutdownTimeout(ShutdownTimeout.QUICK.getDuration());
- eventServer = serverFactory.getEventServer();
- }
-
- @After
+ @AfterEach
public void cleanup() {
runner.shutdown();
removeTestServer();
}
- private void removeTestServer() {
- if (eventServer != null) {
- eventServer.shutdown();
- eventServer = null;
- }
- }
-
- @Test(timeout = DEFAULT_TEST_TIMEOUT_PERIOD)
- public void testValidFiles() throws Exception {
- configureProperties(UDP_SERVER_ADDRESS);
- sendTestData(VALID_FILES);
- checkReceivedAllData(VALID_FILES);
- checkInputQueueIsEmpty();
- }
-
- @Test(timeout = DEFAULT_TEST_TIMEOUT_PERIOD)
- public void testValidFilesEL() throws Exception {
Review Comment:
Trying to decide if this EL case is useful.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] exceptionfactory commented on a diff in pull request #5984: NIFI-9942 Remove load test from TestPutUDP
Posted by GitBox <gi...@apache.org>.
exceptionfactory commented on code in PR #5984:
URL: https://github.com/apache/nifi/pull/5984#discussion_r854545529
##########
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/TestPutUDP.java:
##########
@@ -26,152 +26,107 @@
import org.apache.nifi.remote.io.socket.NetworkUtils;
import org.apache.nifi.util.TestRunner;
import org.apache.nifi.util.TestRunners;
-import org.junit.After;
-import org.junit.Before;
-import org.junit.Test;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.Timeout;
import java.net.InetAddress;
import java.nio.charset.Charset;
import java.nio.charset.StandardCharsets;
import java.util.concurrent.BlockingQueue;
import java.util.concurrent.LinkedBlockingQueue;
-import static org.junit.Assert.assertArrayEquals;
-import static org.junit.Assert.assertNotNull;
-import static org.junit.Assert.assertNull;
+import static org.junit.jupiter.api.Assertions.assertArrayEquals;
+import static org.junit.jupiter.api.Assertions.assertNotNull;
+import static org.junit.jupiter.api.Assertions.assertNull;
+@Timeout(10)
public class TestPutUDP {
private final static String UDP_SERVER_ADDRESS = "127.0.0.1";
- private final static String SERVER_VARIABLE = "ALKJAFLKJDFLSKJSDFLKJSDF";
- private final static String UDP_SERVER_ADDRESS_EL = "${" + SERVER_VARIABLE + "}";
+ private final static String SERVER_VARIABLE = "SERVER";
private static final String DELIMITER = "\n";
private static final Charset CHARSET = StandardCharsets.UTF_8;
private final static int MAX_FRAME_LENGTH = 32800;
private final static int VALID_LARGE_FILE_SIZE = 32768;
- private final static int VALID_SMALL_FILE_SIZE = 64;
private final static int INVALID_LARGE_FILE_SIZE = 1_000_000;
- private final static int LOAD_TEST_ITERATIONS = 500;
- private final static int LOAD_TEST_THREAD_COUNT = 1;
- private final static int DEFAULT_ITERATIONS = 1;
- private final static int DEFAULT_THREAD_COUNT = 1;
private final static char CONTENT_CHAR = 'x';
private final static int DATA_WAIT_PERIOD = 50;
- private final static int DEFAULT_TEST_TIMEOUT_PERIOD = 10000;
- private final static int LONG_TEST_TIMEOUT_PERIOD = 30000;
+ private final static String[] EMPTY_FILE = { "" };
+ private final static String[] VALID_FILES = { "FIRST", "SECOND", "12345678", "343424222", "!@£$%^&*()_+:|{}[];\\" };
private TestRunner runner;
private int port;
private EventServer eventServer;
private BlockingQueue<ByteArrayMessage> messages;
-
- // Test Data
- private final static String[] EMPTY_FILE = { "" };
- private final static String[] VALID_FILES = { "abcdefghijklmnopqrstuvwxyz", "zyxwvutsrqponmlkjihgfedcba", "12345678", "343424222", "!@£$%^&*()_+:|{}[];\\" };
-
- @Before
+ @BeforeEach
public void setup() throws Exception {
runner = TestRunners.newTestRunner(PutUDP.class);
runner.setVariable(SERVER_VARIABLE, UDP_SERVER_ADDRESS);
port = NetworkUtils.getAvailableUdpPort();
createTestServer(port, VALID_LARGE_FILE_SIZE);
}
- private void createTestServer(final int port, final int frameSize) throws Exception {
- messages = new LinkedBlockingQueue<>();
- final byte[] delimiter = DELIMITER.getBytes(CHARSET);
- final InetAddress listenAddress = InetAddress.getByName(UDP_SERVER_ADDRESS);
- NettyEventServerFactory serverFactory = new ByteArrayMessageNettyEventServerFactory(
- runner.getLogger(), listenAddress, port, TransportProtocol.UDP, delimiter, frameSize, messages);
- serverFactory.setSocketReceiveBuffer(MAX_FRAME_LENGTH);
- serverFactory.setShutdownQuietPeriod(ShutdownQuietPeriod.QUICK.getDuration());
- serverFactory.setShutdownTimeout(ShutdownTimeout.QUICK.getDuration());
- eventServer = serverFactory.getEventServer();
- }
-
- @After
+ @AfterEach
public void cleanup() {
runner.shutdown();
removeTestServer();
}
- private void removeTestServer() {
- if (eventServer != null) {
- eventServer.shutdown();
- eventServer = null;
- }
- }
-
- @Test(timeout = DEFAULT_TEST_TIMEOUT_PERIOD)
- public void testValidFiles() throws Exception {
- configureProperties(UDP_SERVER_ADDRESS);
- sendTestData(VALID_FILES);
- checkReceivedAllData(VALID_FILES);
- checkInputQueueIsEmpty();
- }
-
- @Test(timeout = DEFAULT_TEST_TIMEOUT_PERIOD)
- public void testValidFilesEL() throws Exception {
Review Comment:
I considered that, but running through the entire process seemed less than useful just to verify support for expression language. Other properties also support expression language, and are not checked, so it does not seem to add much value in this case. It is also a property defined on `AbstractPutEventProcessor`, so that would be a better place to implement expression language checks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org