You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by "kevdoran (via GitHub)" <gi...@apache.org> on 2025/10/29 20:27:47 UTC
[PR] NIFI-15156 Add support for discovering Listen Ports [nifi]
kevdoran opened a new pull request, #10476:
URL: https://github.com/apache/nifi/pull/10476
<!-- Licensed to the Apache Software Foundation (ASF) under one or more -->
<!-- contributor license agreements. See the NOTICE file distributed with -->
<!-- this work for additional information regarding copyright ownership. -->
<!-- The ASF licenses this file to You under the Apache License, Version 2.0 -->
<!-- (the "License"); you may not use this file except in compliance with -->
<!-- the License. You may obtain a copy of the License at -->
<!-- http://www.apache.org/licenses/LICENSE-2.0 -->
<!-- Unless required by applicable law or agreed to in writing, software -->
<!-- distributed under the License is distributed on an "AS IS" BASIS, -->
<!-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -->
<!-- See the License for the specific language governing permissions and -->
<!-- limitations under the License. -->
# Summary
[NIFI-15156](https://issues.apache.org/jira/browse/NIFI-15156)
- Update Listen Processors and Controller Services to implement ListenComponent interface
- Update Listen Processors and Controller Services to identify ListenPortDefinition properties
- Add draft REST API Controller, Entity, and DTO for listing Listen Ports
- Refactor ListenSyslog so that the transport protocol for a given port property is static and no longer configurable at runtime while still allowing users to choose between UDP and TCP for the processor.
- Update Manifest serializers and deserializers to include ListenPortDefinition for PropertyDescriptors
# Tracking
Please complete the following tracking steps prior to pull request creation.
### Issue Tracking
- [x] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created
### Pull Request Tracking
- [x] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-00000`
- [x] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-00000`
### Pull Request Formatting
- [x] Pull Request based on current revision of the `main` branch
- [x] Pull Request refers to a feature branch with one commit containing changes
# Verification
Please indicate the verification steps performed prior to pull request creation.
### Build
- [ ] Build completed using `./mvnw clean install -P contrib-check`
- [ ] JDK 21
- [ ] JDK 25
### Licensing
- [x] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html)
- [x] New dependencies are documented in applicable `LICENSE` and `NOTICE` files
### Documentation
- [ ] Documentation formatting appears as expected in rendered files
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] NIFI-15156 Add support for discovering Listen Ports [nifi]
Posted by "kevdoran (via GitHub)" <gi...@apache.org>.
kevdoran commented on PR #10476:
URL: https://github.com/apache/nifi/pull/10476#issuecomment-3528009648
Thanks very much for the review @exceptionfactory! I agreed with your suggestions, and have pushed a commit that should address everything.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] NIFI-15156 Add support for discovering Listen Ports [nifi]
Posted by "kevdoran (via GitHub)" <gi...@apache.org>.
kevdoran commented on code in PR #10476:
URL: https://github.com/apache/nifi/pull/10476#discussion_r2523428187
##########
nifi-framework-bundle/nifi-framework/nifi-framework-components/src/main/java/org/apache/nifi/controller/flow/AbstractFlowManager.java:
##########
@@ -732,4 +734,24 @@ private void removeRuleViolationsForSubject(String identifier) {
ruleViolationsManager.removeRuleViolationsForSubject(identifier);
}
}
+
+ @Override
+ public Set<ComponentNode> getAllListenComponents() {
+
+ final Set<ComponentNode> allListenComponents = new HashSet<>();
+
+ // TODO improve performance by adding hooks to create/delete methods to keep track of listen components in a new hash map
Review Comment:
Yeah started to look into this yesterday, and it's complicated enough that it should be a follow up task. Will do.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] NIFI-15156 Add support for discovering Listen Ports [nifi]
Posted by "exceptionfactory (via GitHub)" <gi...@apache.org>.
exceptionfactory commented on code in PR #10476:
URL: https://github.com/apache/nifi/pull/10476#discussion_r2523662527
##########
nifi-extension-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ListenHTTP.java:
##########
@@ -422,6 +429,47 @@ public void shutdownHttpServer() {
shutdownHttpServer(toShutdown);
}
+ @Override
+ public List<ListenPort> getListenPorts(final ConfigurationContext context) {
+
+ final List<ListenPort> ports = new ArrayList<>();
+
+ final Integer primaryPortNumber = context.getProperty(PORT).evaluateAttributeExpressions().asInteger();
+ final Integer healthCheckPortNumber = context.getProperty(HEALTH_CHECK_PORT).evaluateAttributeExpressions().asInteger();
+ final SSLContextProvider sslContextProvider = context.getProperty(SSL_CONTEXT_SERVICE).asControllerService(SSLContextProvider.class);
+ final HttpProtocolStrategy httpProtocolStrategy = sslContextProvider == null
+ ? HttpProtocolStrategy.valueOf(HTTP_PROTOCOL_STRATEGY.getDefaultValue())
+ : context.getProperty(HTTP_PROTOCOL_STRATEGY).asAllowableValue(HttpProtocolStrategy.class);
+ final List<String> applicationProtocols = switch (httpProtocolStrategy) {
+ case H2 -> List.of("h2");
+ case HTTP_1_1 -> List.of("http/1.1");
+ case H2_HTTP_1_1 -> List.of("h2", "http/1.1");
+ case null -> List.of("h2", "http/1.1");
+ };
+
+ if (primaryPortNumber != null) {
Review Comment:
Thanks for clarifying the expected behavior.
##########
nifi-manifest/nifi-extension-manifest-model/src/main/java/org/apache/nifi/extension/manifest/ListenPortDefinition.java:
##########
@@ -0,0 +1,54 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.extension.manifest;
+
+import io.swagger.v3.oas.annotations.media.Schema;
+import jakarta.xml.bind.annotation.XmlAccessType;
+import jakarta.xml.bind.annotation.XmlAccessorType;
+import jakarta.xml.bind.annotation.XmlElement;
+import jakarta.xml.bind.annotation.XmlElementWrapper;
+
+import java.util.List;
+
+@XmlAccessorType(XmlAccessType.FIELD)
+public class ListenPortDefinition {
+
+ private TransportProtocol transportProtocol;
+
+ @XmlElementWrapper
+ @XmlElement(name = "applicationProtocol")
Review Comment:
Thanks for providing the example, that makes sense.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] NIFI-15156 Add support for discovering Listen Ports [nifi]
Posted by "exceptionfactory (via GitHub)" <gi...@apache.org>.
exceptionfactory commented on code in PR #10476:
URL: https://github.com/apache/nifi/pull/10476#discussion_r2519862886
##########
nifi-extension-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ListenHTTP.java:
##########
@@ -422,6 +429,47 @@ public void shutdownHttpServer() {
shutdownHttpServer(toShutdown);
}
+ @Override
+ public List<ListenPort> getListenPorts(final ConfigurationContext context) {
+
+ final List<ListenPort> ports = new ArrayList<>();
+
+ final Integer primaryPortNumber = context.getProperty(PORT).evaluateAttributeExpressions().asInteger();
+ final Integer healthCheckPortNumber = context.getProperty(HEALTH_CHECK_PORT).evaluateAttributeExpressions().asInteger();
+ final SSLContextProvider sslContextProvider = context.getProperty(SSL_CONTEXT_SERVICE).asControllerService(SSLContextProvider.class);
+ final HttpProtocolStrategy httpProtocolStrategy = sslContextProvider == null
+ ? HttpProtocolStrategy.valueOf(HTTP_PROTOCOL_STRATEGY.getDefaultValue())
+ : context.getProperty(HTTP_PROTOCOL_STRATEGY).asAllowableValue(HttpProtocolStrategy.class);
+ final List<String> applicationProtocols = switch (httpProtocolStrategy) {
+ case H2 -> List.of("h2");
+ case HTTP_1_1 -> List.of("http/1.1");
+ case H2_HTTP_1_1 -> List.of("h2", "http/1.1");
+ case null -> List.of("h2", "http/1.1");
+ };
+
+ if (primaryPortNumber != null) {
Review Comment:
The primary port is required, should this ever be `null`?
##########
nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/NiFiServiceFacade.java:
##########
@@ -3126,4 +3127,17 @@ ControllerServiceReferencingComponentsEntity updateControllerServiceReferencingC
*/
Set<String> filterComponents(String groupId, Function<ProcessGroup, Set<String>> getComponents);
+ // ----------------------------------------
+ // Listen Port methods
+ // ----------------------------------------
+
+ /**
+ * Get all dynamically defined data ingress ports provided by Listen Components (e.g., Processors and Controller Services)
+ *
+ * @param user the user performing the lookup
+ * @return the list of listen Ports accessible to the current user
+ */
+ Set<ListenPortDTO> getListenPorts(NiFiUser user);
+
+
Review Comment:
Empty lines can be removed.
##########
nifi-framework-bundle/nifi-framework/nifi-client-dto/src/main/java/org/apache/nifi/web/api/dto/ListenPortDTO.java:
##########
@@ -0,0 +1,136 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.web.api.dto;
+
+import io.swagger.v3.oas.annotations.media.Schema;
+import jakarta.xml.bind.annotation.XmlType;
+
+import java.util.List;
+
+@XmlType(name = "listenPort")
+public class ListenPortDTO {
+
+ // Port definition
+ private String portName;
+ private int portNumber;
+ private String transportProtocol;
+ private List<String> applicationProtocols;
+
+ // Contextual information about the component providing the port, and the PG containing the component
+ private String componentType;
+ private String componentId;
+ private String componentName;
+ private String componentClass;
+ private String parentGroupId;
+ private String parentGroupName;
+
+ public String getPortName() {
+ return portName;
+ }
+
+ @Schema(description = "The name of the the listen port. Useful context for components that provide multiple ports.")
Review Comment:
It looks like this annotation should be on `getPortName()` instead of `setPortName()`
##########
nifi-framework-bundle/nifi-framework/nifi-framework-components/src/main/java/org/apache/nifi/controller/flow/AbstractFlowManager.java:
##########
@@ -732,4 +734,24 @@ private void removeRuleViolationsForSubject(String identifier) {
ruleViolationsManager.removeRuleViolationsForSubject(identifier);
}
}
+
+ @Override
+ public Set<ComponentNode> getAllListenComponents() {
+
+ final Set<ComponentNode> allListenComponents = new HashSet<>();
Review Comment:
For deterministic ordering, recommend using `LinkedHashSet`:
```suggestion
final Set<ComponentNode> allListenComponents = new LinkedHashSet<>();
```
##########
nifi-manifest/nifi-extension-manifest-model/src/main/java/org/apache/nifi/extension/manifest/ListenPortDefinition.java:
##########
@@ -0,0 +1,54 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.extension.manifest;
+
+import io.swagger.v3.oas.annotations.media.Schema;
+import jakarta.xml.bind.annotation.XmlAccessType;
+import jakarta.xml.bind.annotation.XmlAccessorType;
+import jakarta.xml.bind.annotation.XmlElement;
+import jakarta.xml.bind.annotation.XmlElementWrapper;
+
+import java.util.List;
+
+@XmlAccessorType(XmlAccessType.FIELD)
+public class ListenPortDefinition {
+
+ private TransportProtocol transportProtocol;
+
+ @XmlElementWrapper
+ @XmlElement(name = "applicationProtocol")
Review Comment:
Is there a reason for renaming the element to avoid the plural?
##########
nifi-manifest/nifi-extension-manifest-model/src/main/java/org/apache/nifi/extension/manifest/ListenPortDefinition.java:
##########
@@ -0,0 +1,54 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.extension.manifest;
+
+import io.swagger.v3.oas.annotations.media.Schema;
+import jakarta.xml.bind.annotation.XmlAccessType;
+import jakarta.xml.bind.annotation.XmlAccessorType;
+import jakarta.xml.bind.annotation.XmlElement;
+import jakarta.xml.bind.annotation.XmlElementWrapper;
+
+import java.util.List;
+
+@XmlAccessorType(XmlAccessType.FIELD)
+public class ListenPortDefinition {
+
+ private TransportProtocol transportProtocol;
+
+ @XmlElementWrapper
+ @XmlElement(name = "applicationProtocol")
+ private List<String> applicationProtocols;
+
+ @Schema(description = "The transport protocol used by this listen port")
+ public TransportProtocol getTransportProtocol() {
+ return transportProtocol;
+ }
+
+ public void setTransportProtocol(final TransportProtocol transportProtocol) {
+ this.transportProtocol = transportProtocol;
+ }
+
+ @Schema(description = "The application protocol(s) that this listen port could support (if any)")
Review Comment:
```suggestion
@Schema(description = "The application protocols that this listen port could support (if any)")
```
##########
nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/controller/ControllerFacade.java:
##########
@@ -1853,6 +1859,62 @@ public SearchResultsDTO search(final String searchLiteral, final String activeGr
return results;
}
+ /**
+ * Get all user-defined data ingress ports provided by Listen Components (e.g., Processors and Controller Services)
+ *
+ * @param user the user performing the lookup
+ * @return the set of listen Ports accessible to the current user
+ */
+ public Set<ListenPortDTO> getListenPorts(final NiFiUser user) {
+
+ // Get all listen components for which the requesting user is authorized
+ final Set<ComponentNode> listenComponentNodes = flowController.getFlowManager().getAllListenComponents().stream()
+ .filter(componentNode -> componentNode.isAuthorized(authorizer, RequestAction.READ, user))
+ .collect(Collectors.toSet());
+
+ // If the current user doesn't have access to any listen components, return an empty result for ports
+ if (listenComponentNodes.isEmpty()) {
+ return Collections.emptySet();
+ }
+
+ // Now find all Listen Ports provided by the Listen Components. A listen component can provide multiple Listen Ports (e.g., ListenHTTP can have a data port and a health check port).
+ // The current Listen Ports for a component depend on configuration (e.g., port property value), so create a configuration context to provide ListenComponent.getListenPorts(context).
+ final Set<ListenPortDTO> listenPorts = new HashSet<>();
+ final ControllerServiceProvider controllerServiceProvider = flowController.getControllerServiceProvider();
+ for (final ComponentNode componentNode : listenComponentNodes) {
+ final ConfigurationContext configurationContext = new StandardConfigurationContext(componentNode, controllerServiceProvider, null);
+ final ConfigurableComponent component = componentNode.getComponent();
+ if (component instanceof ListenComponent listenComponent /* should always be true */) {
+ listenComponent.getListenPorts(configurationContext).forEach(listenPort -> {
+ final ListenPortDTO listenPortDTO = new ListenPortDTO();
+ listenPortDTO.setPortName(listenPort.getPortName());
+ listenPortDTO.setPortNumber(listenPort.getPortNumber());
+ listenPortDTO.setTransportProtocol(listenPort.getTransportProtocol().name());
+ listenPortDTO.setApplicationProtocols(listenPort.getApplicationProtocols());
+ listenPortDTO.setComponentClass(componentNode.getCanonicalClassName());
+ listenPortDTO.setComponentId(componentNode.getIdentifier());
+ listenPortDTO.setComponentName(componentNode.getName());
+ listenPortDTO.setParentGroupId(componentNode.getParentProcessGroup().map(ProcessGroup::getIdentifier).orElse(null));
+ listenPortDTO.setParentGroupName(componentNode.getParentProcessGroup().map(ProcessGroup::getName).orElse(null));
+
+ // TODO this next bit could be refined
Review Comment:
Recommend removing this `TODO`
##########
nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/controller/ControllerFacade.java:
##########
@@ -1853,6 +1859,62 @@ public SearchResultsDTO search(final String searchLiteral, final String activeGr
return results;
}
+ /**
+ * Get all user-defined data ingress ports provided by Listen Components (e.g., Processors and Controller Services)
+ *
+ * @param user the user performing the lookup
+ * @return the set of listen Ports accessible to the current user
+ */
+ public Set<ListenPortDTO> getListenPorts(final NiFiUser user) {
+
+ // Get all listen components for which the requesting user is authorized
+ final Set<ComponentNode> listenComponentNodes = flowController.getFlowManager().getAllListenComponents().stream()
+ .filter(componentNode -> componentNode.isAuthorized(authorizer, RequestAction.READ, user))
+ .collect(Collectors.toSet());
+
+ // If the current user doesn't have access to any listen components, return an empty result for ports
+ if (listenComponentNodes.isEmpty()) {
+ return Collections.emptySet();
+ }
+
+ // Now find all Listen Ports provided by the Listen Components. A listen component can provide multiple Listen Ports (e.g., ListenHTTP can have a data port and a health check port).
+ // The current Listen Ports for a component depend on configuration (e.g., port property value), so create a configuration context to provide ListenComponent.getListenPorts(context).
+ final Set<ListenPortDTO> listenPorts = new HashSet<>();
+ final ControllerServiceProvider controllerServiceProvider = flowController.getControllerServiceProvider();
+ for (final ComponentNode componentNode : listenComponentNodes) {
+ final ConfigurationContext configurationContext = new StandardConfigurationContext(componentNode, controllerServiceProvider, null);
+ final ConfigurableComponent component = componentNode.getComponent();
+ if (component instanceof ListenComponent listenComponent /* should always be true */) {
Review Comment:
For readability, recommend moving the comment above or below the line, instead of on the same line.
##########
nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/controller/ControllerFacade.java:
##########
@@ -1853,6 +1859,62 @@ public SearchResultsDTO search(final String searchLiteral, final String activeGr
return results;
}
+ /**
+ * Get all user-defined data ingress ports provided by Listen Components (e.g., Processors and Controller Services)
+ *
+ * @param user the user performing the lookup
+ * @return the set of listen Ports accessible to the current user
+ */
+ public Set<ListenPortDTO> getListenPorts(final NiFiUser user) {
+
+ // Get all listen components for which the requesting user is authorized
+ final Set<ComponentNode> listenComponentNodes = flowController.getFlowManager().getAllListenComponents().stream()
+ .filter(componentNode -> componentNode.isAuthorized(authorizer, RequestAction.READ, user))
+ .collect(Collectors.toSet());
+
+ // If the current user doesn't have access to any listen components, return an empty result for ports
+ if (listenComponentNodes.isEmpty()) {
+ return Collections.emptySet();
+ }
+
+ // Now find all Listen Ports provided by the Listen Components. A listen component can provide multiple Listen Ports (e.g., ListenHTTP can have a data port and a health check port).
+ // The current Listen Ports for a component depend on configuration (e.g., port property value), so create a configuration context to provide ListenComponent.getListenPorts(context).
+ final Set<ListenPortDTO> listenPorts = new HashSet<>();
+ final ControllerServiceProvider controllerServiceProvider = flowController.getControllerServiceProvider();
+ for (final ComponentNode componentNode : listenComponentNodes) {
+ final ConfigurationContext configurationContext = new StandardConfigurationContext(componentNode, controllerServiceProvider, null);
+ final ConfigurableComponent component = componentNode.getComponent();
+ if (component instanceof ListenComponent listenComponent /* should always be true */) {
+ listenComponent.getListenPorts(configurationContext).forEach(listenPort -> {
+ final ListenPortDTO listenPortDTO = new ListenPortDTO();
+ listenPortDTO.setPortName(listenPort.getPortName());
+ listenPortDTO.setPortNumber(listenPort.getPortNumber());
+ listenPortDTO.setTransportProtocol(listenPort.getTransportProtocol().name());
+ listenPortDTO.setApplicationProtocols(listenPort.getApplicationProtocols());
+ listenPortDTO.setComponentClass(componentNode.getCanonicalClassName());
+ listenPortDTO.setComponentId(componentNode.getIdentifier());
+ listenPortDTO.setComponentName(componentNode.getName());
+ listenPortDTO.setParentGroupId(componentNode.getParentProcessGroup().map(ProcessGroup::getIdentifier).orElse(null));
+ listenPortDTO.setParentGroupName(componentNode.getParentProcessGroup().map(ProcessGroup::getName).orElse(null));
+
+ // TODO this next bit could be refined
+ if (componentNode instanceof ProcessorNode) {
+ listenPortDTO.setComponentType("Processor");
+ } else if (componentNode instanceof ControllerServiceNode) {
+ listenPortDTO.setComponentType("ControllerService");
+ } else {
+ // Would we ever have anything other than a Processor or Controller Service providing a Listen Port?
Review Comment:
The question can be removed since the log indicates this is unexpected.
##########
nifi-framework-bundle/nifi-framework/nifi-framework-components/src/main/java/org/apache/nifi/controller/flow/AbstractFlowManager.java:
##########
@@ -732,4 +734,24 @@ private void removeRuleViolationsForSubject(String identifier) {
ruleViolationsManager.removeRuleViolationsForSubject(identifier);
}
}
+
+ @Override
+ public Set<ComponentNode> getAllListenComponents() {
+
+ final Set<ComponentNode> allListenComponents = new HashSet<>();
+
+ // TODO improve performance by adding hooks to create/delete methods to keep track of listen components in a new hash map
Review Comment:
Recommend removing this `TODO` and creating a Jira issue instead.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] NIFI-15156 Add support for discovering Listen Ports [nifi]
Posted by "kevdoran (via GitHub)" <gi...@apache.org>.
kevdoran commented on PR #10476:
URL: https://github.com/apache/nifi/pull/10476#issuecomment-3470242287
I simplified this branch to have the bare minimum for the following:
- Include Listen Port definitions in extension manifests, flow definitions, versioned flows, etc.
- REST API is able to report Listen Ports the framework can discover that the user is authorized for
- Framework is able to discover Listen Ports from components that implement new ListenComponent interface
- ListenHTTP is updated to support new interfaces for testing all of the above
I restructured the [commits](https://github.com/apache/nifi/pull/10476/commits) so that they are individually reviewable for each of the areas above.
To test the functionality:
1. Build this branch (skip contrib-check, I still have some cleanup to do_
2. Observe that the manifest file for ListenHTTP `nifi-extension-bundles/nifi-standard-bundle/nifi-standard-nar/target/META-INF/docs/extension-manifest.xml` includes the ListenPort definition in PropertyDescriptors:
```xml
# nifi-extension-bundles/nifi-standard-bundle/nifi-standard-nar/target/META-INF/docs/extension-manifest.xml
<property>
<name>Listening Port</name>
<displayName>Listening Port</displayName>
<description>The Port to listen on for incoming connections</description>
<required>true</required>
<sensitive>false</sensitive>
<expressionLanguageSupported>true</expressionLanguageSupported>
<expressionLanguageScope>ENVIRONMENT</expressionLanguageScope>
<dynamicallyModifiesClasspath>false</dynamicallyModifiesClasspath>
<dynamic>false</dynamic>
<listenPortDefinition>
<transportProtocol>TCP</transportProtocol>
<applicationProtocols>
<applicationProtocol>http/1.1</applicationProtocol>
<applicationProtocol>h2</applicationProtocol>
</applicationProtocols>
</listenPortDefinition>
</property>
<property>
<name>health-check-port</name>
<displayName>Listening Port for Health Check Requests</displayName>
<required>false</required>
<sensitive>false</sensitive>
<expressionLanguageSupported>true</expressionLanguageSupported>
<expressionLanguageScope>ENVIRONMENT</expressionLanguageScope>
<dynamicallyModifiesClasspath>false</dynamicallyModifiesClasspath>
<dynamic>false</dynamic>
<listenPortDefinition>
<transportProtocol>TCP</transportProtocol>
<applicationProtocols>
<applicationProtocol>http/1.1</applicationProtocol>
<applicationProtocol>h2</applicationProtocol>
</applicationProtocols>
</listenPortDefinition>
</property>
```
3. Start NiFi and go to NiFi UI in browser: https://localhost:8443/nifi
```
NIFI_DIR=nifi-assembly/target/nifi-2.7.0-SNAPSHOT-bin/nifi-2.7.0-SNAPSHOT && ./$NIFI_DIR/bin/nifi.sh set-single-user-credentials admin ***** && $NIFI_DIR/bin/nifi.sh start && tail -f $NIFI_DIR/logs/nifi-*.log
```
4. Add one or more ListenHTTP Processors to the canvas. Try nesting them in Process Groups. Configure each to have Port property and optionally and Health Check port as well.
5. In the same browser (for auth cookie reasons) go to this new NiFi REST API url: https://localhost:8443/nifi-api/flow/listen-ports - you should see something like the following:
```json
{
"listenPorts": [
{
"portNumber": 12345,
"transportProtocol": "TCP",
"applicationProtocols": [
"http/1.1"
],
"componentType": "Processor",
"componentId": "36cc4d72-019a-1000-bb5c-ca9e515d29f0",
"componentName": "ListenHTTP",
"componentClass": "org.apache.nifi.processors.standard.ListenHTTP",
"parentGroupId": "36cbfb7f-019a-1000-3fa2-66e24d7ab4af",
"parentGroupName": "NiFi Flow"
},
{
"portNumber": 23456,
"transportProtocol": "TCP",
"applicationProtocols": [
"http/1.1"
],
"componentType": "Processor",
"componentId": "36cc4d72-019a-1000-bb5c-ca9e515d29f0",
"componentName": "ListenHTTP",
"componentClass": "org.apache.nifi.processors.standard.ListenHTTP",
"parentGroupId": "36cbfb7f-019a-1000-3fa2-66e24d7ab4af",
"parentGroupName": "NiFi Flow"
},
{
"portNumber": 8888,
"transportProtocol": "TCP",
"applicationProtocols": [
"http/1.1"
],
"componentType": "Processor",
"componentId": "36ccbc61-019a-1000-eec1-67567effc4da",
"componentName": "Listen For Jira Webhook Events",
"componentClass": "org.apache.nifi.processors.standard.ListenHTTP",
"parentGroupId": "36cca021-019a-1000-3143-8c31afe40b9e",
"parentGroupName": "My Custom PG"
}
]
}
```
6. Download the flow definition JSON. Observe that the processor property descriptors contain the listen port definitions:
```json
"health-check-port": {
"name": "health-check-port",
"displayName": "Listening Port for Health Check Requests",
"identifiesControllerService": false,
"sensitive": false,
"dynamic": false,
"listenPortDefinition":
{
"transportProtocol": "TCP",
"applicationProtocols":
[
"http/1.1",
"h2"
]
}
},
"Listening Port": {
"name": "Listening Port",
"displayName": "Listening Port",
"identifiesControllerService": false,
"sensitive": false,
"dynamic": false,
"listenPortDefinition":
{
"transportProtocol": "TCP",
"applicationProtocols":
[
"http/1.1",
"h2"
]
}
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] NIFI-15156 Add support for discovering Listen Ports [nifi]
Posted by "kevdoran (via GitHub)" <gi...@apache.org>.
kevdoran commented on code in PR #10476:
URL: https://github.com/apache/nifi/pull/10476#discussion_r2523447025
##########
nifi-framework-bundle/nifi-framework/nifi-client-dto/src/main/java/org/apache/nifi/web/api/dto/ListenPortDTO.java:
##########
@@ -0,0 +1,136 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.web.api.dto;
+
+import io.swagger.v3.oas.annotations.media.Schema;
+import jakarta.xml.bind.annotation.XmlType;
+
+import java.util.List;
+
+@XmlType(name = "listenPort")
+public class ListenPortDTO {
+
+ // Port definition
+ private String portName;
+ private int portNumber;
+ private String transportProtocol;
+ private List<String> applicationProtocols;
+
+ // Contextual information about the component providing the port, and the PG containing the component
+ private String componentType;
+ private String componentId;
+ private String componentName;
+ private String componentClass;
+ private String parentGroupId;
+ private String parentGroupName;
+
+ public String getPortName() {
+ return portName;
+ }
+
+ @Schema(description = "The name of the the listen port. Useful context for components that provide multiple ports.")
Review Comment:
Good catch!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] NIFI-15156 Add support for discovering Listen Ports [nifi]
Posted by "exceptionfactory (via GitHub)" <gi...@apache.org>.
exceptionfactory merged PR #10476:
URL: https://github.com/apache/nifi/pull/10476
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] NIFI-15156 Add support for discovering Listen Ports [nifi]
Posted by "kevdoran (via GitHub)" <gi...@apache.org>.
kevdoran commented on code in PR #10476:
URL: https://github.com/apache/nifi/pull/10476#discussion_r2523568549
##########
nifi-manifest/nifi-extension-manifest-model/src/main/java/org/apache/nifi/extension/manifest/ListenPortDefinition.java:
##########
@@ -0,0 +1,54 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.extension.manifest;
+
+import io.swagger.v3.oas.annotations.media.Schema;
+import jakarta.xml.bind.annotation.XmlAccessType;
+import jakarta.xml.bind.annotation.XmlAccessorType;
+import jakarta.xml.bind.annotation.XmlElement;
+import jakarta.xml.bind.annotation.XmlElementWrapper;
+
+import java.util.List;
+
+@XmlAccessorType(XmlAccessType.FIELD)
+public class ListenPortDefinition {
+
+ private TransportProtocol transportProtocol;
+
+ @XmlElementWrapper
+ @XmlElement(name = "applicationProtocol")
Review Comment:
Yeah, that sets the xml name for the elements inside the element wrapper collection, ie:
```xml
<applicationProtocols>
<applicationProtocol>http/1.1</applicationProtocol>
<applicationProtocol>h2</applicationProtocol>
</applicationProtocols>
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] NIFI-15156 Add support for discovering Listen Ports [nifi]
Posted by "kevdoran (via GitHub)" <gi...@apache.org>.
kevdoran commented on code in PR #10476:
URL: https://github.com/apache/nifi/pull/10476#discussion_r2523399361
##########
nifi-extension-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ListenHTTP.java:
##########
@@ -422,6 +429,47 @@ public void shutdownHttpServer() {
shutdownHttpServer(toShutdown);
}
+ @Override
+ public List<ListenPort> getListenPorts(final ConfigurationContext context) {
+
+ final List<ListenPort> ports = new ArrayList<>();
+
+ final Integer primaryPortNumber = context.getProperty(PORT).evaluateAttributeExpressions().asInteger();
+ final Integer healthCheckPortNumber = context.getProperty(HEALTH_CHECK_PORT).evaluateAttributeExpressions().asInteger();
+ final SSLContextProvider sslContextProvider = context.getProperty(SSL_CONTEXT_SERVICE).asControllerService(SSLContextProvider.class);
+ final HttpProtocolStrategy httpProtocolStrategy = sslContextProvider == null
+ ? HttpProtocolStrategy.valueOf(HTTP_PROTOCOL_STRATEGY.getDefaultValue())
+ : context.getProperty(HTTP_PROTOCOL_STRATEGY).asAllowableValue(HttpProtocolStrategy.class);
+ final List<String> applicationProtocols = switch (httpProtocolStrategy) {
+ case H2 -> List.of("h2");
+ case HTTP_1_1 -> List.of("http/1.1");
+ case H2_HTTP_1_1 -> List.of("h2", "http/1.1");
+ case null -> List.of("h2", "http/1.1");
+ };
+
+ if (primaryPortNumber != null) {
Review Comment:
It does not have a default value, so it could be null before the processor is configured, ie, the invalid state after it has been added to the canvas but before it has been configured. With the current design, that means that Listen Port would not be discoverable until the port property is configured.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] NIFI-15156 Add support for discovering Listen Ports [nifi]
Posted by "kevdoran (via GitHub)" <gi...@apache.org>.
kevdoran commented on code in PR #10476:
URL: https://github.com/apache/nifi/pull/10476#discussion_r2523579202
##########
nifi-framework-bundle/nifi-framework/nifi-framework-components/src/main/java/org/apache/nifi/controller/flow/AbstractFlowManager.java:
##########
@@ -732,4 +734,24 @@ private void removeRuleViolationsForSubject(String identifier) {
ruleViolationsManager.removeRuleViolationsForSubject(identifier);
}
}
+
+ @Override
+ public Set<ComponentNode> getAllListenComponents() {
+
+ final Set<ComponentNode> allListenComponents = new HashSet<>();
+
+ // TODO improve performance by adding hooks to create/delete methods to keep track of listen components in a new hash map
Review Comment:
Apache Jira is degraded at the moment and I'm unable to create the issue, but I've made a not for myself to create one.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org