You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by GitBox <gi...@apache.org> on 2021/09/13 09:50:32 UTC
[GitHub] [nifi-minifi-cpp] adam-markovics opened a new pull request #1172: MQTT tests updated
adam-markovics opened a new pull request #1172:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1172
Thank you for submitting a contribution to Apache NiFi - MiNiFi C++.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [ ] Does your PR title start with MINIFICPP-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
- [ ] Has your PR been rebased against the latest commit within the target branch (typically main)?
- [ ] Is your initial contribution a single, squashed commit?
### For code changes:
- [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the LICENSE file?
- [ ] If applicable, have you updated the NOTICE file?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in which it is rendered?
### Note:
Please ensure that once the PR is submitted, you check GitHub Actions CI results for build issues and submit an update to your PR as soon as possible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi-minifi-cpp] adam-markovics commented on a change in pull request #1172: MQTT tests added
Posted by GitBox <gi...@apache.org>.
adam-markovics commented on a change in pull request #1172:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1172#discussion_r714539249
##########
File path: docker/test/integration/features/mqtt.feature
##########
@@ -0,0 +1,66 @@
+Feature: Sending data to using MQTT streaming platform using PublishMQTT
+ In order to send and receive data via MQTT
+ As a user of MiNiFi
+ I need to have PublishMQTT and ConsumeMQTT processors
+
+ Background:
+ Given the content of "/tmp/output" is monitored
+
+ Scenario: A MiNiFi instance transfers data to an MQTT broker
+ Given a GetFile processor with the "Input Directory" property set to "/tmp/input"
+ And a file with the content "test" is present in "/tmp/input"
+ And a PublishMQTT processor set up to communicate with an MQTT broker instance
+ And a PutFile processor with the "Directory" property set to "/tmp/output"
+ And the "success" relationship of the GetFile processor is connected to the PublishMQTT
+ And the "success" relationship of the PublishMQTT processor is connected to the PutFile
+
+ And an MQTT broker is set up in correspondence with the PublishMQTT
+
+ When both instances start up
+ Then a flowfile with the content "test" is placed in the monitored directory in less than 60 seconds
+ And the MQTT broker has 1 log line matching "Received PUBLISH from .*testtopic.*\\(4 bytes\\)"
+
+ Scenario: A MiNiFi instance tries to transfer data to a non-existent MQTT broker
Review comment:
No, it is because PublishMQTT will do nothing and wait for the broker to appear. Then it will send it. I'm open to discussion if this is a desired behavior.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi-minifi-cpp] adam-markovics commented on a change in pull request #1172: MQTT tests added
Posted by GitBox <gi...@apache.org>.
adam-markovics commented on a change in pull request #1172:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1172#discussion_r711909662
##########
File path: docker/test/integration/resources/mqtt_broker/mosquitto_test.conf
##########
@@ -0,0 +1,874 @@
+# Config file for mosquitto
Review comment:
This config is a slightly modified version of the base image, eclipse-mosquitto:2.0.11. Maybe I could apply some small diff on that instead in the Dockerfile in a RUN section.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi-minifi-cpp] lordgamez commented on a change in pull request #1172: MQTT tests added
Posted by GitBox <gi...@apache.org>.
lordgamez commented on a change in pull request #1172:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1172#discussion_r711912077
##########
File path: docker/test/integration/minifi/core/MqttBrokerContainer.py
##########
@@ -0,0 +1,23 @@
+import logging
+from .Container import Container
+
+
+class MqttBrokerContainer(Container):
+ def __init__(self, name, vols, network, image_store):
+ super().__init__(name, 'mqtt-broker', vols, network, image_store)
+
+ def get_startup_finished_log_entry(self):
+ return "mosquitto version 2.0.11 running"
Review comment:
If that's not a huge effort to implement regex check here, that would okay.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi-minifi-cpp] lordgamez commented on a change in pull request #1172: MQTT tests added
Posted by GitBox <gi...@apache.org>.
lordgamez commented on a change in pull request #1172:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1172#discussion_r714563909
##########
File path: docker/test/integration/steps/steps.py
##########
@@ -556,9 +556,9 @@ def step_impl(context, log_message, duration):
context.test.check_minifi_log_contents(log_message, timeparse(duration))
# MQTT
Review comment:
This line fails flake8 test locally because 2 blank lines are expected:
`steps.py:559:1: E302 expected 2 blank lines, found 1`
Somehow it does not fail in the CI which should be checked.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi-minifi-cpp] fgerlits commented on a change in pull request #1172: MQTT tests added
Posted by GitBox <gi...@apache.org>.
fgerlits commented on a change in pull request #1172:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1172#discussion_r716498144
##########
File path: docker/test/integration/features/mqtt.feature
##########
@@ -0,0 +1,66 @@
+Feature: Sending data to using MQTT streaming platform using PublishMQTT
+ In order to send and receive data via MQTT
+ As a user of MiNiFi
+ I need to have PublishMQTT and ConsumeMQTT processors
+
+ Background:
+ Given the content of "/tmp/output" is monitored
+
+ Scenario: A MiNiFi instance transfers data to an MQTT broker
+ Given a GetFile processor with the "Input Directory" property set to "/tmp/input"
+ And a file with the content "test" is present in "/tmp/input"
+ And a PublishMQTT processor set up to communicate with an MQTT broker instance
+ And a PutFile processor with the "Directory" property set to "/tmp/output"
+ And the "success" relationship of the GetFile processor is connected to the PublishMQTT
+ And the "success" relationship of the PublishMQTT processor is connected to the PutFile
+
+ And an MQTT broker is set up in correspondence with the PublishMQTT
+
+ When both instances start up
+ Then a flowfile with the content "test" is placed in the monitored directory in less than 60 seconds
+ And the MQTT broker has 1 log line matching "Received PUBLISH from .*testtopic.*\\(4 bytes\\)"
+
+ Scenario: A MiNiFi instance tries to transfer data to a non-existent MQTT broker
Review comment:
OK, I get it -- if the processor can't connect to the broker, it won't even pick up any flow files from the queue. I would change the title of this scenario to better explain this, e.g. "If the MQTT broker does not exist, then no flow files are processed".
This approach could be improved (e.g. with a configurable timeout), but definitely not in this pull request.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi-minifi-cpp] adam-markovics commented on a change in pull request #1172: MQTT tests added
Posted by GitBox <gi...@apache.org>.
adam-markovics commented on a change in pull request #1172:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1172#discussion_r714595142
##########
File path: docker/test/integration/steps/steps.py
##########
@@ -556,9 +556,9 @@ def step_impl(context, log_message, duration):
context.test.check_minifi_log_contents(log_message, timeparse(duration))
# MQTT
Review comment:
I checked before submitting and nothing failed for me either. Strange. My local version is:
```
adam@amarkovics-Linux:~/work/minifi-cpp/build$ flake8 --version
3.7.9 (mccabe: 0.6.1, pycodestyle: 2.5.0, pyflakes: 2.1.1) CPython 3.8.10 on Linux
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi-minifi-cpp] lordgamez commented on a change in pull request #1172: MQTT tests added
Posted by GitBox <gi...@apache.org>.
lordgamez commented on a change in pull request #1172:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1172#discussion_r710981237
##########
File path: docker/test/integration/features/mqtt.feature
##########
@@ -0,0 +1,66 @@
+Feature: Sending data to using MQTT streaming platform using PublishMQTT
Review comment:
The word 'using' is not needed here
##########
File path: docker/test/integration/resources/mqtt_broker/mosquitto_test.conf
##########
@@ -0,0 +1,874 @@
+# Config file for mosquitto
Review comment:
There seems to be only 3 lines not commented out in this configuration. Could we only leave those in this test config? I think it would be more clear at glance what is actually set.
##########
File path: docker/test/integration/minifi/core/ImageStore.py
##########
@@ -23,21 +24,28 @@ def cleanup(self):
self.client.images.remove(image.id, force=True)
def get_image(self, container_engine):
+ if container_engine in self.images:
Review comment:
Good improvement. With this change I think we should also rename the `__get_.*_image` functions to `__build_.*image`
##########
File path: docker/test/integration/minifi/core/DockerTestCluster.py
##########
@@ -54,13 +55,16 @@ def get_app_log(self, container_name):
return container.status, None
- def wait_for_app_logs(self, container_name, log_entry, timeout_seconds, count=1):
+ def wait_for_app_logs(self, container_name, log_entry, timeout_seconds, count=1, use_regex=False):
Review comment:
I would prefer having a separate function for log matching and regex matching, the surrounding code could may be extracted as a decorator.
##########
File path: docker/test/integration/minifi/core/MqttBrokerContainer.py
##########
@@ -0,0 +1,23 @@
+import logging
+from .Container import Container
+
+
+class MqttBrokerContainer(Container):
+ def __init__(self, name, vols, network, image_store):
+ super().__init__(name, 'mqtt-broker', vols, network, image_store)
+
+ def get_startup_finished_log_entry(self):
+ return "mosquitto version 2.0.11 running"
Review comment:
Could this be changed somehow not to be version dependent?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi-minifi-cpp] fgerlits commented on a change in pull request #1172: MQTT tests added
Posted by GitBox <gi...@apache.org>.
fgerlits commented on a change in pull request #1172:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1172#discussion_r712841843
##########
File path: docker/test/integration/MiNiFi_integration_test_driver.py
##########
@@ -196,3 +196,6 @@ def check_minifi_log_contents(self, line, timeout_seconds=60):
def check_query_results(self, postgresql_container_name, query, number_of_rows, timeout_seconds):
assert self.cluster.check_query_results(postgresql_container_name, query, number_of_rows, timeout_seconds)
+
+ def check_mosquitto_logs(self, broker_name, log_pattern, timeout_seconds, count=1, use_regex=False):
Review comment:
This is not really specific to mosquitto; I would rename it to something like "wait_for_container_logs" (and rename "broker_name" to "container_name").
##########
File path: docker/test/integration/features/mqtt.feature
##########
@@ -0,0 +1,66 @@
+Feature: Sending data to using MQTT streaming platform using PublishMQTT
+ In order to send and receive data via MQTT
+ As a user of MiNiFi
+ I need to have PublishMQTT and ConsumeMQTT processors
+
+ Background:
+ Given the content of "/tmp/output" is monitored
+
+ Scenario: A MiNiFi instance transfers data to an MQTT broker
+ Given a GetFile processor with the "Input Directory" property set to "/tmp/input"
+ And a file with the content "test" is present in "/tmp/input"
+ And a PublishMQTT processor set up to communicate with an MQTT broker instance
+ And a PutFile processor with the "Directory" property set to "/tmp/output"
+ And the "success" relationship of the GetFile processor is connected to the PublishMQTT
+ And the "success" relationship of the PublishMQTT processor is connected to the PutFile
+
+ And an MQTT broker is set up in correspondence with the PublishMQTT
+
+ When both instances start up
+ Then a flowfile with the content "test" is placed in the monitored directory in less than 60 seconds
+ And the MQTT broker has 1 log line matching "Received PUBLISH from .*testtopic.*\\(4 bytes\\)"
+
+ Scenario: A MiNiFi instance tries to transfer data to a non-existent MQTT broker
+ Given a GetFile processor with the "Input Directory" property set to "/tmp/input"
+ And a file with the content "test" is present in "/tmp/input"
+ And a PublishMQTT processor set up to communicate with an MQTT broker instance
+ And a PutFile processor with the "Directory" property set to "/tmp/output"
+ And the "success" relationship of the GetFile processor is connected to the PublishMQTT
+ And the "success" relationship of the PublishMQTT processor is connected to the PutFile
+ And the "failure" relationship of the PublishMQTT processor is connected to the PutFile
+
+ When the MiNiFi instance starts up
+ Then no files are placed in the monitored directory in 30 seconds of running time
+
+ Scenario: Verify delivery of message when MQTT broker is unstable
+ Given a GetFile processor with the "Input Directory" property set to "/tmp/input"
+ And a file with the content "test" is present in "/tmp/input"
+ And a PublishMQTT processor set up to communicate with an MQTT broker instance
+ And a PutFile processor with the "Directory" property set to "/tmp/output"
+ And the "success" relationship of the GetFile processor is connected to the PublishMQTT
+ And the "success" relationship of the PublishMQTT processor is connected to the PutFile
+
+ When the MiNiFi instance starts up
+
+ Then no files are placed in the monitored directory in 30 seconds of running time
+ And an MQTT broker is deployed in correspondence with the PublishMQTT
+ And a flowfile with the content "test" is placed in the monitored directory in less than 60 seconds
+ And the MQTT broker has 1 log line matching "Received PUBLISH from .*testtopic.*\\(4 bytes\\)"
+
+ Scenario: A MiNiFi instance publishes and consumes data to/from an MQTT broker
+ Given a GetFile processor with the "Input Directory" property set to "/tmp/input"
+ And a file with the content "test" is present in "/tmp/input"
+ And a PublishMQTT processor set up to communicate with an MQTT broker instance
+ And the "success" relationship of the GetFile processor is connected to the PublishMQTT
+
+ And a ConsumeMQTT processor set up to communicate with an MQTT broker instance
+ And a PutFile processor with the "Directory" property set to "/tmp/output"
+ And "ConsumeMQTT" processor is a start node
+ And the "success" relationship of the ConsumeMQTT processor is connected to the PutFile
+
+ And an MQTT broker is set up in correspondence with the PublishMQTT and ConsumeMQTT
+
+ When both instances start up
+ Then a flowfile with the content "test" is placed in the monitored directory in less than 60 seconds
+ And the MQTT broker has 1 log line matching "Received PUBLISH from .*testtopic.*\\(4 bytes\\)"
+ And the MQTT broker has 1 log line matching "Received SUBSCRIBE from"
Review comment:
missing newline at the end
##########
File path: docker/test/integration/features/mqtt.feature
##########
@@ -0,0 +1,66 @@
+Feature: Sending data to using MQTT streaming platform using PublishMQTT
+ In order to send and receive data via MQTT
+ As a user of MiNiFi
+ I need to have PublishMQTT and ConsumeMQTT processors
+
+ Background:
+ Given the content of "/tmp/output" is monitored
+
+ Scenario: A MiNiFi instance transfers data to an MQTT broker
+ Given a GetFile processor with the "Input Directory" property set to "/tmp/input"
+ And a file with the content "test" is present in "/tmp/input"
+ And a PublishMQTT processor set up to communicate with an MQTT broker instance
+ And a PutFile processor with the "Directory" property set to "/tmp/output"
+ And the "success" relationship of the GetFile processor is connected to the PublishMQTT
+ And the "success" relationship of the PublishMQTT processor is connected to the PutFile
+
+ And an MQTT broker is set up in correspondence with the PublishMQTT
+
+ When both instances start up
+ Then a flowfile with the content "test" is placed in the monitored directory in less than 60 seconds
+ And the MQTT broker has 1 log line matching "Received PUBLISH from .*testtopic.*\\(4 bytes\\)"
+
+ Scenario: A MiNiFi instance tries to transfer data to a non-existent MQTT broker
Review comment:
Is the test file not routed to failure because PublishMQTT will throw in `onSchedule()`? If so, then the name of this scenario could be better, eg. "If the MQTT broker is not available, the PublishMQTT processor will not start".
##########
File path: docker/test/integration/features/mqtt.feature
##########
@@ -0,0 +1,66 @@
+Feature: Sending data to using MQTT streaming platform using PublishMQTT
+ In order to send and receive data via MQTT
+ As a user of MiNiFi
+ I need to have PublishMQTT and ConsumeMQTT processors
+
+ Background:
+ Given the content of "/tmp/output" is monitored
+
+ Scenario: A MiNiFi instance transfers data to an MQTT broker
+ Given a GetFile processor with the "Input Directory" property set to "/tmp/input"
+ And a file with the content "test" is present in "/tmp/input"
+ And a PublishMQTT processor set up to communicate with an MQTT broker instance
+ And a PutFile processor with the "Directory" property set to "/tmp/output"
+ And the "success" relationship of the GetFile processor is connected to the PublishMQTT
+ And the "success" relationship of the PublishMQTT processor is connected to the PutFile
+
+ And an MQTT broker is set up in correspondence with the PublishMQTT
+
+ When both instances start up
+ Then a flowfile with the content "test" is placed in the monitored directory in less than 60 seconds
+ And the MQTT broker has 1 log line matching "Received PUBLISH from .*testtopic.*\\(4 bytes\\)"
+
+ Scenario: A MiNiFi instance tries to transfer data to a non-existent MQTT broker
+ Given a GetFile processor with the "Input Directory" property set to "/tmp/input"
+ And a file with the content "test" is present in "/tmp/input"
+ And a PublishMQTT processor set up to communicate with an MQTT broker instance
+ And a PutFile processor with the "Directory" property set to "/tmp/output"
+ And the "success" relationship of the GetFile processor is connected to the PublishMQTT
+ And the "success" relationship of the PublishMQTT processor is connected to the PutFile
+ And the "failure" relationship of the PublishMQTT processor is connected to the PutFile
+
+ When the MiNiFi instance starts up
+ Then no files are placed in the monitored directory in 30 seconds of running time
+
+ Scenario: Verify delivery of message when MQTT broker is unstable
+ Given a GetFile processor with the "Input Directory" property set to "/tmp/input"
+ And a file with the content "test" is present in "/tmp/input"
+ And a PublishMQTT processor set up to communicate with an MQTT broker instance
+ And a PutFile processor with the "Directory" property set to "/tmp/output"
+ And the "success" relationship of the GetFile processor is connected to the PublishMQTT
+ And the "success" relationship of the PublishMQTT processor is connected to the PutFile
+
+ When the MiNiFi instance starts up
+
+ Then no files are placed in the monitored directory in 30 seconds of running time
+ And an MQTT broker is deployed in correspondence with the PublishMQTT
Review comment:
moving the blank line may make this test clearer:
```suggestion
When the MiNiFi instance starts up
Then no files are placed in the monitored directory in 30 seconds of running time
And an MQTT broker is deployed in correspondence with the PublishMQTT
```
##########
File path: docker/test/integration/minifi/core/DockerTestCluster.py
##########
@@ -54,13 +55,16 @@ def get_app_log(self, container_name):
return container.status, None
- def wait_for_app_logs(self, container_name, log_entry, timeout_seconds, count=1):
+ def wait_for_app_logs(self, container_name, log_entry, timeout_seconds, count=1, use_regex=False):
wait_start_time = time.perf_counter()
while (time.perf_counter() - wait_start_time) < timeout_seconds:
logging.info('Waiting for app-logs `%s` in container `%s`', log_entry, container_name)
status, logs = self.get_app_log(container_name)
- if logs is not None and count <= logs.decode("utf-8").count(log_entry):
- return True
+ if logs is not None:
+ if not use_regex and count <= logs.decode("utf-8").count(log_entry):
+ return True
+ elif use_regex and len(re.findall(log_entry, logs.decode("utf-8"))) <= count:
Review comment:
this should be `>=`
(I would also change the non-regex case from `count <= something` to `something >= count` as I think that would read better)
##########
File path: docker/test/integration/steps/steps.py
##########
@@ -546,3 +554,14 @@ def step_impl(context, query, number_of_rows, timeout_seconds):
@then("the Minifi logs contain the following message: \"{log_message}\" in less than {duration}")
def step_impl(context, log_message, duration):
context.test.check_minifi_log_contents(log_message, timeparse(duration))
+
+# MQTT
+@then("the MQTT broker has 1 log line matching \"{log_pattern}\"")
Review comment:
this sounds like the test will only pass if there is exactly 1 matching log line, but in fact we check for at least 1 matching log line
```suggestion
@then("the MQTT broker has a log line matching \"{log_pattern}\"")
```
##########
File path: docker/test/integration/minifi/core/ImageStore.py
##########
@@ -12,6 +12,7 @@ class ImageStore:
def __init__(self):
self.client = docker.from_env()
self.images = dict()
+ self.test_dir = os.environ['PYTHONPATH'].split(':')[-1] # Based on DockerVerify.sh
Review comment:
This looks brittle, as someone may add another directory to the end of `PYTHONPATH` in the future. It would be better if eg. DockerVerify.sh exported a dedicated `TEST_DIRECTORY` environment variable which we could read here.
EDIT: I can see this same code is already used in a few other places; I would change those, too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi-minifi-cpp] adam-markovics commented on a change in pull request #1172: MQTT tests added
Posted by GitBox <gi...@apache.org>.
adam-markovics commented on a change in pull request #1172:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1172#discussion_r714539249
##########
File path: docker/test/integration/features/mqtt.feature
##########
@@ -0,0 +1,66 @@
+Feature: Sending data to using MQTT streaming platform using PublishMQTT
+ In order to send and receive data via MQTT
+ As a user of MiNiFi
+ I need to have PublishMQTT and ConsumeMQTT processors
+
+ Background:
+ Given the content of "/tmp/output" is monitored
+
+ Scenario: A MiNiFi instance transfers data to an MQTT broker
+ Given a GetFile processor with the "Input Directory" property set to "/tmp/input"
+ And a file with the content "test" is present in "/tmp/input"
+ And a PublishMQTT processor set up to communicate with an MQTT broker instance
+ And a PutFile processor with the "Directory" property set to "/tmp/output"
+ And the "success" relationship of the GetFile processor is connected to the PublishMQTT
+ And the "success" relationship of the PublishMQTT processor is connected to the PutFile
+
+ And an MQTT broker is set up in correspondence with the PublishMQTT
+
+ When both instances start up
+ Then a flowfile with the content "test" is placed in the monitored directory in less than 60 seconds
+ And the MQTT broker has 1 log line matching "Received PUBLISH from .*testtopic.*\\(4 bytes\\)"
+
+ Scenario: A MiNiFi instance tries to transfer data to a non-existent MQTT broker
Review comment:
No, it is because PublishMQTT will do nothing and wait for the broker to appear. Then it will forward it. I'm open to discussion if this is a desired behavior.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi-minifi-cpp] lordgamez commented on a change in pull request #1172: MQTT tests added
Posted by GitBox <gi...@apache.org>.
lordgamez commented on a change in pull request #1172:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1172#discussion_r714608322
##########
File path: docker/test/integration/steps/steps.py
##########
@@ -556,9 +556,9 @@ def step_impl(context, log_message, duration):
context.test.check_minifi_log_contents(log_message, timeparse(duration))
# MQTT
Review comment:
Must be the version difference (in CI as well) because my local version is 3.8.4
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi-minifi-cpp] fgerlits closed pull request #1172: MINIFICPP-977 MQTT tests added
Posted by GitBox <gi...@apache.org>.
fgerlits closed pull request #1172:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1172
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi-minifi-cpp] adam-markovics commented on a change in pull request #1172: MQTT tests added
Posted by GitBox <gi...@apache.org>.
adam-markovics commented on a change in pull request #1172:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1172#discussion_r711909662
##########
File path: docker/test/integration/resources/mqtt_broker/mosquitto_test.conf
##########
@@ -0,0 +1,874 @@
+# Config file for mosquitto
Review comment:
This config is a slightly modified version of the one in the base image, eclipse-mosquitto:2.0.11. Maybe I could apply some small diff on that instead in the Dockerfile in a RUN section.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi-minifi-cpp] lordgamez commented on a change in pull request #1172: MQTT tests added
Posted by GitBox <gi...@apache.org>.
lordgamez commented on a change in pull request #1172:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1172#discussion_r711911687
##########
File path: docker/test/integration/resources/mqtt_broker/mosquitto_test.conf
##########
@@ -0,0 +1,874 @@
+# Config file for mosquitto
Review comment:
That should be okay as well, or a `sed` command if that's a very trivial change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi-minifi-cpp] adam-markovics commented on a change in pull request #1172: MQTT tests added
Posted by GitBox <gi...@apache.org>.
adam-markovics commented on a change in pull request #1172:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1172#discussion_r711903849
##########
File path: docker/test/integration/minifi/core/MqttBrokerContainer.py
##########
@@ -0,0 +1,23 @@
+import logging
+from .Container import Container
+
+
+class MqttBrokerContainer(Container):
+ def __init__(self, name, vols, network, image_store):
+ super().__init__(name, 'mqtt-broker', vols, network, image_store)
+
+ def get_startup_finished_log_entry(self):
+ return "mosquitto version 2.0.11 running"
Review comment:
With regex matching I think. Would you second that?
Currently the output of starting mosquitto in verbose:
```
adam@amarkovics-Linux:~/work/minifi-cpp/build$ mosquitto -v
1632119285: mosquitto version 2.0.12 starting
1632119285: Using default config.
1632119285: Starting in local only mode. Connections will only be possible from clients running on this machine.
1632119285: Create a configuration file which defines a listener to allow remote access.
1632119285: For more details see https://mosquitto.org/documentation/authentication-methods/
1632119285: Opening ipv4 listen socket on port 1883.
1632119285: Opening ipv6 listen socket on port 1883.
1632119285: mosquitto version 2.0.12 running
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org