You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by GitBox <gi...@apache.org> on 2021/02/25 21:44:17 UTC
[GitHub] [nifi] exceptionfactory commented on a change in pull request #4846: NIFI-8260 Implement an Upload File capability in the NiFi UI for flow definitions.
exceptionfactory commented on a change in pull request #4846:
URL: https://github.com/apache/nifi/pull/4846#discussion_r583189069
##########
File path: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-components/src/main/java/org/apache/nifi/groups/StandardProcessGroup.java
##########
@@ -3766,7 +3765,7 @@ public void updateFlow(final VersionedFlowSnapshot proposedSnapshot, final Strin
final boolean updateDescendantVersionedFlows) {
writeLock.lock();
try {
- verifyCanUpdate(proposedSnapshot, true, verifyNotDirty);
+ verifyCanUpdate(proposedSnapshot, true, verifyNotDirty);
Review comment:
It looks like the changes in this file are just formatting related, should the changes be reverted?
##########
File path: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/StandardNiFiServiceFacade.java
##########
@@ -15,9 +15,35 @@
* limitations under the License.
*/
package org.apache.nifi.web;
-
import com.google.common.collect.Sets;
import io.prometheus.client.CollectorRegistry;
+import java.io.IOException;
Review comment:
Recommend reverting the changes to this file since it is only reordering imports.
##########
File path: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/ProcessGroupResource.java
##########
@@ -4173,6 +4179,227 @@ private void sanitizeRegistryInfo(final VersionedProcessGroup versionedProcessGr
}
}
+ /**
+ * Uploads the specified versioned flow file and adds it to a new process group.
+ *
+ * @param httpServletRequest request
+ * @param in The flow file stream
+ * @return A processGroupEntity
+ * @throws IOException if there is an error during deserialization of the InputStream
+ */
+ @POST
+ @Consumes(MediaType.MULTIPART_FORM_DATA)
+ @Produces(MediaType.APPLICATION_JSON)
+ @Path("{id}/process-groups/upload")
+ @ApiOperation(
+ value = "Uploads a versioned flow file and creates a process group",
+ response = ProcessGroupEntity.class,
+ authorizations = {
+ @Authorization(value = "Write - /process-groups/{uuid}")
+ }
+ )
+ @ApiResponses(
+ value = {
+ @ApiResponse(code = 400, message = "NiFi was unable to complete the request because it was invalid. The request should not be retried without modification."),
+ @ApiResponse(code = 401, message = "Client could not be authenticated."),
+ @ApiResponse(code = 403, message = "Client is not authorized to make this request."),
+ @ApiResponse(code = 404, message = "The specified resource could not be found."),
+ @ApiResponse(code = 409, message = "The request was valid but NiFi was not in the appropriate state to process it. Retrying the same request later may be successful.")
+ }
+ )
+ public Response uploadProcessGroup(
+ @Context final HttpServletRequest httpServletRequest,
+ @ApiParam(
+ value = "The process group id.",
+ required = true
+ )
+ @PathParam("id") final String groupId,
+ @ApiParam(
+ value = "The process group name.",
+ required = true
+ )
+ @FormDataParam("groupName") final String groupName,
+ @ApiParam(
+ value = "The process group X position.",
+ required = true
+ )
+ @FormDataParam("position-x") final Double positionX,
+ @ApiParam(
+ value = "The process group Y position.",
+ required = true
+ )
+ @FormDataParam("position-y") final Double positionY,
+ @ApiParam(
+ value = "The client id.",
+ required = true
+ )
+ @FormDataParam("clientId") final String clientId,
+ @ApiParam(
+ value = "Acknowledges that this node is disconnected to allow for mutable requests to proceed.",
+ required = false
+ )
+ @FormDataParam(DISCONNECTED_NODE_ACKNOWLEDGED) @DefaultValue("false") final Boolean disconnectedNodeAcknowledged,
+ @FormDataParam("file") final InputStream in) throws IOException {
+
+ // ensure the group name is specified
+ if (StringUtils.isBlank(groupName)) {
+ throw new IllegalArgumentException("The process group name is required.");
+ }
+
+ if (StringUtils.isBlank(groupId)) {
+ throw new IllegalArgumentException("The parent process group id must be the same as specified in the URI.");
+ }
+
+ if (positionX == null) {
+ throw new IllegalArgumentException("The x coordinate of the proposed position must be specified.");
+ }
+
+ if (positionY == null) {
+ throw new IllegalArgumentException("The y coordinate of the proposed position must be specified.");
+ }
+
+ if (StringUtils.isBlank(clientId)) {
+ throw new IllegalArgumentException("The client id must be specified");
+ }
+
+ // create a new process group
+ final ProcessGroupEntity newProcessGroupEntity = new ProcessGroupEntity();
+
+ // get the contents of the InputStream as a String
+ String stringContent = null;
+ if (in != null) {
+ stringContent = IOUtils.toString(in, StandardCharsets.UTF_8);
+ }
+
+ // deserialize content to a VersionedFlowSnapshot
+ VersionedFlowSnapshot deserializedSnapshot = null;
+ final ObjectMapper mapper = new ObjectMapper();
+
+ if (stringContent.length() > 0) {
+ try {
+ mapper.setSerializationInclusion(JsonInclude.Include.NON_NULL);
+ mapper.setDefaultPropertyInclusion(JsonInclude.Value.construct(JsonInclude.Include.NON_NULL, JsonInclude.Include.NON_NULL));
+ mapper.setAnnotationIntrospector(new JaxbAnnotationIntrospector(mapper.getTypeFactory()));
+ mapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);
+
+ deserializedSnapshot = mapper.readValue(stringContent, VersionedFlowSnapshot.class);
+ } catch (JsonParseException jpe) {
+ logger.warn("The file uploaded is not a valid JSON format: ", jpe);
+ String responseJson = "The specified file is not a valid JSON format.";
+ return Response.status(Response.Status.OK).entity(responseJson).type("application/json").build();
+ } catch (IOException e) {
Review comment:
This `IOException` is logged, but is not wrapped an thrown again. As a result, following method calls could fail, so this Exception should be wrapped and a new Exception should be thrown.
##########
File path: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/ProcessGroupResource.java
##########
@@ -4173,6 +4179,227 @@ private void sanitizeRegistryInfo(final VersionedProcessGroup versionedProcessGr
}
}
+ /**
+ * Uploads the specified versioned flow file and adds it to a new process group.
+ *
+ * @param httpServletRequest request
+ * @param in The flow file stream
+ * @return A processGroupEntity
+ * @throws IOException if there is an error during deserialization of the InputStream
+ */
+ @POST
+ @Consumes(MediaType.MULTIPART_FORM_DATA)
+ @Produces(MediaType.APPLICATION_JSON)
+ @Path("{id}/process-groups/upload")
+ @ApiOperation(
+ value = "Uploads a versioned flow file and creates a process group",
+ response = ProcessGroupEntity.class,
+ authorizations = {
+ @Authorization(value = "Write - /process-groups/{uuid}")
+ }
+ )
+ @ApiResponses(
+ value = {
+ @ApiResponse(code = 400, message = "NiFi was unable to complete the request because it was invalid. The request should not be retried without modification."),
+ @ApiResponse(code = 401, message = "Client could not be authenticated."),
+ @ApiResponse(code = 403, message = "Client is not authorized to make this request."),
+ @ApiResponse(code = 404, message = "The specified resource could not be found."),
+ @ApiResponse(code = 409, message = "The request was valid but NiFi was not in the appropriate state to process it. Retrying the same request later may be successful.")
+ }
+ )
+ public Response uploadProcessGroup(
+ @Context final HttpServletRequest httpServletRequest,
+ @ApiParam(
+ value = "The process group id.",
+ required = true
+ )
+ @PathParam("id") final String groupId,
+ @ApiParam(
+ value = "The process group name.",
+ required = true
+ )
+ @FormDataParam("groupName") final String groupName,
+ @ApiParam(
+ value = "The process group X position.",
+ required = true
+ )
+ @FormDataParam("position-x") final Double positionX,
+ @ApiParam(
+ value = "The process group Y position.",
+ required = true
+ )
+ @FormDataParam("position-y") final Double positionY,
+ @ApiParam(
+ value = "The client id.",
+ required = true
+ )
+ @FormDataParam("clientId") final String clientId,
+ @ApiParam(
+ value = "Acknowledges that this node is disconnected to allow for mutable requests to proceed.",
+ required = false
+ )
+ @FormDataParam(DISCONNECTED_NODE_ACKNOWLEDGED) @DefaultValue("false") final Boolean disconnectedNodeAcknowledged,
+ @FormDataParam("file") final InputStream in) throws IOException {
+
+ // ensure the group name is specified
+ if (StringUtils.isBlank(groupName)) {
+ throw new IllegalArgumentException("The process group name is required.");
+ }
+
+ if (StringUtils.isBlank(groupId)) {
+ throw new IllegalArgumentException("The parent process group id must be the same as specified in the URI.");
+ }
+
+ if (positionX == null) {
+ throw new IllegalArgumentException("The x coordinate of the proposed position must be specified.");
+ }
+
+ if (positionY == null) {
+ throw new IllegalArgumentException("The y coordinate of the proposed position must be specified.");
+ }
+
+ if (StringUtils.isBlank(clientId)) {
+ throw new IllegalArgumentException("The client id must be specified");
+ }
+
+ // create a new process group
+ final ProcessGroupEntity newProcessGroupEntity = new ProcessGroupEntity();
+
+ // get the contents of the InputStream as a String
+ String stringContent = null;
+ if (in != null) {
+ stringContent = IOUtils.toString(in, StandardCharsets.UTF_8);
+ }
+
+ // deserialize content to a VersionedFlowSnapshot
+ VersionedFlowSnapshot deserializedSnapshot = null;
+ final ObjectMapper mapper = new ObjectMapper();
+
+ if (stringContent.length() > 0) {
+ try {
+ mapper.setSerializationInclusion(JsonInclude.Include.NON_NULL);
+ mapper.setDefaultPropertyInclusion(JsonInclude.Value.construct(JsonInclude.Include.NON_NULL, JsonInclude.Include.NON_NULL));
+ mapper.setAnnotationIntrospector(new JaxbAnnotationIntrospector(mapper.getTypeFactory()));
+ mapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);
+
+ deserializedSnapshot = mapper.readValue(stringContent, VersionedFlowSnapshot.class);
+ } catch (JsonParseException jpe) {
+ logger.warn("The file uploaded is not a valid JSON format: ", jpe);
+ String responseJson = "The specified file is not a valid JSON format.";
+ return Response.status(Response.Status.OK).entity(responseJson).type("application/json").build();
+ } catch (IOException e) {
+ logger.warn("An error occurred while deserializing the flow version.", e);
+ }
+ } else {
+ logger.warn("The uploaded file was empty");
+ throw new IOException("The uploaded file was empty.");
+ }
+
+ sanitizeRegistryInfo(deserializedSnapshot.getFlowContents());
+
+ // resolve Bundle info
+ serviceFacade.discoverCompatibleBundles(deserializedSnapshot.getFlowContents());
+
+ // if there are any Controller Services referenced that are inherited from the parent group, resolve those to point to the appropriate Controller Service, if we are able to.
+ serviceFacade.resolveInheritedControllerServices(deserializedSnapshot, groupId, NiFiUserUtils.getNiFiUser());
+
+ // create a new ProcessGroupDTO
+ ProcessGroupDTO processGroupDTO = new ProcessGroupDTO();
+
+ processGroupDTO.setParentGroupId(groupId);
+ processGroupDTO.setName(groupName);
+
+ newProcessGroupEntity.setComponent(processGroupDTO);
+ newProcessGroupEntity.setVersionedFlowSnapshot(deserializedSnapshot);
+
+ // create a PositionDTO
+ PositionDTO positionDTO = new PositionDTO();
+ positionDTO.setX(positionX);
+ positionDTO.setY(positionY);
+ newProcessGroupEntity.getComponent().setPosition(positionDTO);
Review comment:
Is it possible to refactor the creation of the ProcessGroupEntity into a separate method? That would encapsulate the initial verification and building of the object prior to it being processed.
##########
File path: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/css/main.css
##########
@@ -286,45 +290,125 @@ span.details-title {
height: 28px;
}
-#template-file-field {
+#select-file-button button {
+ position: absolute;
+ float: right;
+ right: 4px;
+ top: 22px;
+ font-size: 18px;
+ color: #004849;
+ border: none;
+ background-color: transparent;
+}
+
+/*#select-file-button button:hover {*/
Review comment:
Should this commented style be retained, or can it be removed?
##########
File path: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/ProcessGroupResource.java
##########
@@ -4173,6 +4179,227 @@ private void sanitizeRegistryInfo(final VersionedProcessGroup versionedProcessGr
}
}
+ /**
+ * Uploads the specified versioned flow file and adds it to a new process group.
+ *
+ * @param httpServletRequest request
+ * @param in The flow file stream
+ * @return A processGroupEntity
+ * @throws IOException if there is an error during deserialization of the InputStream
+ */
+ @POST
+ @Consumes(MediaType.MULTIPART_FORM_DATA)
+ @Produces(MediaType.APPLICATION_JSON)
+ @Path("{id}/process-groups/upload")
+ @ApiOperation(
+ value = "Uploads a versioned flow file and creates a process group",
+ response = ProcessGroupEntity.class,
+ authorizations = {
+ @Authorization(value = "Write - /process-groups/{uuid}")
+ }
+ )
+ @ApiResponses(
+ value = {
+ @ApiResponse(code = 400, message = "NiFi was unable to complete the request because it was invalid. The request should not be retried without modification."),
+ @ApiResponse(code = 401, message = "Client could not be authenticated."),
+ @ApiResponse(code = 403, message = "Client is not authorized to make this request."),
+ @ApiResponse(code = 404, message = "The specified resource could not be found."),
+ @ApiResponse(code = 409, message = "The request was valid but NiFi was not in the appropriate state to process it. Retrying the same request later may be successful.")
+ }
+ )
+ public Response uploadProcessGroup(
+ @Context final HttpServletRequest httpServletRequest,
+ @ApiParam(
+ value = "The process group id.",
+ required = true
+ )
+ @PathParam("id") final String groupId,
+ @ApiParam(
+ value = "The process group name.",
+ required = true
+ )
+ @FormDataParam("groupName") final String groupName,
+ @ApiParam(
+ value = "The process group X position.",
+ required = true
+ )
+ @FormDataParam("position-x") final Double positionX,
+ @ApiParam(
+ value = "The process group Y position.",
+ required = true
+ )
+ @FormDataParam("position-y") final Double positionY,
+ @ApiParam(
+ value = "The client id.",
+ required = true
+ )
+ @FormDataParam("clientId") final String clientId,
+ @ApiParam(
+ value = "Acknowledges that this node is disconnected to allow for mutable requests to proceed.",
+ required = false
+ )
+ @FormDataParam(DISCONNECTED_NODE_ACKNOWLEDGED) @DefaultValue("false") final Boolean disconnectedNodeAcknowledged,
+ @FormDataParam("file") final InputStream in) throws IOException {
+
+ // ensure the group name is specified
+ if (StringUtils.isBlank(groupName)) {
+ throw new IllegalArgumentException("The process group name is required.");
+ }
+
+ if (StringUtils.isBlank(groupId)) {
+ throw new IllegalArgumentException("The parent process group id must be the same as specified in the URI.");
Review comment:
The check evaluates whether `groupId` is blank, but the error message seems to indicate that some other check is happening, should the error message be changed, or is some other check necessary?
##########
File path: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/ProcessGroupResource.java
##########
@@ -4173,6 +4179,227 @@ private void sanitizeRegistryInfo(final VersionedProcessGroup versionedProcessGr
}
}
+ /**
+ * Uploads the specified versioned flow file and adds it to a new process group.
+ *
+ * @param httpServletRequest request
+ * @param in The flow file stream
+ * @return A processGroupEntity
+ * @throws IOException if there is an error during deserialization of the InputStream
+ */
+ @POST
+ @Consumes(MediaType.MULTIPART_FORM_DATA)
+ @Produces(MediaType.APPLICATION_JSON)
+ @Path("{id}/process-groups/upload")
+ @ApiOperation(
+ value = "Uploads a versioned flow file and creates a process group",
+ response = ProcessGroupEntity.class,
+ authorizations = {
+ @Authorization(value = "Write - /process-groups/{uuid}")
+ }
+ )
+ @ApiResponses(
+ value = {
+ @ApiResponse(code = 400, message = "NiFi was unable to complete the request because it was invalid. The request should not be retried without modification."),
+ @ApiResponse(code = 401, message = "Client could not be authenticated."),
+ @ApiResponse(code = 403, message = "Client is not authorized to make this request."),
+ @ApiResponse(code = 404, message = "The specified resource could not be found."),
+ @ApiResponse(code = 409, message = "The request was valid but NiFi was not in the appropriate state to process it. Retrying the same request later may be successful.")
+ }
+ )
+ public Response uploadProcessGroup(
+ @Context final HttpServletRequest httpServletRequest,
+ @ApiParam(
+ value = "The process group id.",
+ required = true
+ )
+ @PathParam("id") final String groupId,
+ @ApiParam(
+ value = "The process group name.",
+ required = true
+ )
+ @FormDataParam("groupName") final String groupName,
+ @ApiParam(
+ value = "The process group X position.",
+ required = true
+ )
+ @FormDataParam("position-x") final Double positionX,
+ @ApiParam(
+ value = "The process group Y position.",
+ required = true
+ )
+ @FormDataParam("position-y") final Double positionY,
+ @ApiParam(
+ value = "The client id.",
+ required = true
+ )
+ @FormDataParam("clientId") final String clientId,
+ @ApiParam(
+ value = "Acknowledges that this node is disconnected to allow for mutable requests to proceed.",
+ required = false
+ )
+ @FormDataParam(DISCONNECTED_NODE_ACKNOWLEDGED) @DefaultValue("false") final Boolean disconnectedNodeAcknowledged,
+ @FormDataParam("file") final InputStream in) throws IOException {
+
+ // ensure the group name is specified
+ if (StringUtils.isBlank(groupName)) {
+ throw new IllegalArgumentException("The process group name is required.");
+ }
+
+ if (StringUtils.isBlank(groupId)) {
+ throw new IllegalArgumentException("The parent process group id must be the same as specified in the URI.");
+ }
+
+ if (positionX == null) {
+ throw new IllegalArgumentException("The x coordinate of the proposed position must be specified.");
+ }
+
+ if (positionY == null) {
+ throw new IllegalArgumentException("The y coordinate of the proposed position must be specified.");
+ }
+
+ if (StringUtils.isBlank(clientId)) {
+ throw new IllegalArgumentException("The client id must be specified");
+ }
+
+ // create a new process group
+ final ProcessGroupEntity newProcessGroupEntity = new ProcessGroupEntity();
+
+ // get the contents of the InputStream as a String
+ String stringContent = null;
+ if (in != null) {
Review comment:
Should a `null` InputStream throw an Exception? That would allow passing the InputStream directly to `ObjectMapper.readValue()` as opposed to reading it into a String.
##########
File path: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/ProcessGroupResource.java
##########
@@ -4173,6 +4179,227 @@ private void sanitizeRegistryInfo(final VersionedProcessGroup versionedProcessGr
}
}
+ /**
+ * Uploads the specified versioned flow file and adds it to a new process group.
+ *
+ * @param httpServletRequest request
+ * @param in The flow file stream
+ * @return A processGroupEntity
+ * @throws IOException if there is an error during deserialization of the InputStream
+ */
+ @POST
+ @Consumes(MediaType.MULTIPART_FORM_DATA)
+ @Produces(MediaType.APPLICATION_JSON)
+ @Path("{id}/process-groups/upload")
+ @ApiOperation(
+ value = "Uploads a versioned flow file and creates a process group",
+ response = ProcessGroupEntity.class,
+ authorizations = {
+ @Authorization(value = "Write - /process-groups/{uuid}")
+ }
+ )
+ @ApiResponses(
+ value = {
+ @ApiResponse(code = 400, message = "NiFi was unable to complete the request because it was invalid. The request should not be retried without modification."),
+ @ApiResponse(code = 401, message = "Client could not be authenticated."),
+ @ApiResponse(code = 403, message = "Client is not authorized to make this request."),
+ @ApiResponse(code = 404, message = "The specified resource could not be found."),
+ @ApiResponse(code = 409, message = "The request was valid but NiFi was not in the appropriate state to process it. Retrying the same request later may be successful.")
+ }
+ )
+ public Response uploadProcessGroup(
+ @Context final HttpServletRequest httpServletRequest,
+ @ApiParam(
+ value = "The process group id.",
+ required = true
+ )
+ @PathParam("id") final String groupId,
+ @ApiParam(
+ value = "The process group name.",
+ required = true
+ )
+ @FormDataParam("groupName") final String groupName,
+ @ApiParam(
+ value = "The process group X position.",
+ required = true
+ )
+ @FormDataParam("position-x") final Double positionX,
+ @ApiParam(
+ value = "The process group Y position.",
+ required = true
+ )
+ @FormDataParam("position-y") final Double positionY,
+ @ApiParam(
+ value = "The client id.",
+ required = true
+ )
+ @FormDataParam("clientId") final String clientId,
+ @ApiParam(
+ value = "Acknowledges that this node is disconnected to allow for mutable requests to proceed.",
+ required = false
+ )
+ @FormDataParam(DISCONNECTED_NODE_ACKNOWLEDGED) @DefaultValue("false") final Boolean disconnectedNodeAcknowledged,
+ @FormDataParam("file") final InputStream in) throws IOException {
+
+ // ensure the group name is specified
+ if (StringUtils.isBlank(groupName)) {
+ throw new IllegalArgumentException("The process group name is required.");
+ }
+
+ if (StringUtils.isBlank(groupId)) {
+ throw new IllegalArgumentException("The parent process group id must be the same as specified in the URI.");
+ }
+
+ if (positionX == null) {
+ throw new IllegalArgumentException("The x coordinate of the proposed position must be specified.");
+ }
+
+ if (positionY == null) {
+ throw new IllegalArgumentException("The y coordinate of the proposed position must be specified.");
+ }
+
+ if (StringUtils.isBlank(clientId)) {
+ throw new IllegalArgumentException("The client id must be specified");
+ }
+
+ // create a new process group
+ final ProcessGroupEntity newProcessGroupEntity = new ProcessGroupEntity();
+
+ // get the contents of the InputStream as a String
+ String stringContent = null;
+ if (in != null) {
+ stringContent = IOUtils.toString(in, StandardCharsets.UTF_8);
+ }
+
+ // deserialize content to a VersionedFlowSnapshot
+ VersionedFlowSnapshot deserializedSnapshot = null;
+ final ObjectMapper mapper = new ObjectMapper();
+
+ if (stringContent.length() > 0) {
+ try {
+ mapper.setSerializationInclusion(JsonInclude.Include.NON_NULL);
+ mapper.setDefaultPropertyInclusion(JsonInclude.Value.construct(JsonInclude.Include.NON_NULL, JsonInclude.Include.NON_NULL));
+ mapper.setAnnotationIntrospector(new JaxbAnnotationIntrospector(mapper.getTypeFactory()));
+ mapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);
Review comment:
The `ObjectMapper` instance is thread-safe, so it could be configured as a member variable of `ProcessGroupResource` and these configuration methods can be run once instead of on each upload.
##########
File path: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/js/nf/canvas/header/components/nf-ng-group-component.js
##########
@@ -129,17 +133,78 @@
* Initialize the modal.
*/
init: function () {
+ var self = this;
+
+ self.fileForm = $('#file-upload-form').ajaxForm({
+ url: '../nifi-api/process-groups/',
+ dataType: 'json',
+ beforeSubmit: function ($form, options) {
+ // ensure uploading to the current process group
+ options.url += (encodeURIComponent(nfCanvasUtils.getGroupId()) + '/process-groups/upload');
+ }
+ });
+
// configure the new process group dialog
this.getElement().modal({
scrollableContentStyle: 'scrollable',
headerText: 'Add Process Group',
handler: {
close: function () {
+ self.fileToBeUploaded = null;
+ $('#selected-file-name').text('');
+ $('#upload-file-field').val('');
$('#new-process-group-name').val('');
$('#new-process-group-dialog').removeData('pt');
+
+ // reset the form to ensure that the change fire will fire
+ self.fileForm.resetForm();
}
}
});
+
+ $('#upload-file-field-button').on('click', function (e) {
+ $('#upload-file-field').click();
+ });
+
+ $('#upload-file-field').on('change', function (e) {
+ $('#upload-file-field-button').hide();
+
+ self.fileToBeUploaded = e.target;
+ var filename = $(this).val();
+ var filenameExtension;
+ if (!nfCommon.isBlank(filename)) {
+ filenameExtension = filename.replace(/^.*[\\\/]/, '');
+ filename = filename.replace(/^.*[\\\/]/, '').replace(/\..*/, '');
Review comment:
Should these regular expression replacements be moved into a separate function that describes what is happening? Otherwise the purpose is not quite clear.
##########
File path: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/ProcessGroupResource.java
##########
@@ -4173,6 +4179,227 @@ private void sanitizeRegistryInfo(final VersionedProcessGroup versionedProcessGr
}
}
+ /**
+ * Uploads the specified versioned flow file and adds it to a new process group.
+ *
+ * @param httpServletRequest request
+ * @param in The flow file stream
+ * @return A processGroupEntity
+ * @throws IOException if there is an error during deserialization of the InputStream
+ */
+ @POST
+ @Consumes(MediaType.MULTIPART_FORM_DATA)
+ @Produces(MediaType.APPLICATION_JSON)
+ @Path("{id}/process-groups/upload")
+ @ApiOperation(
+ value = "Uploads a versioned flow file and creates a process group",
+ response = ProcessGroupEntity.class,
+ authorizations = {
+ @Authorization(value = "Write - /process-groups/{uuid}")
+ }
+ )
+ @ApiResponses(
+ value = {
+ @ApiResponse(code = 400, message = "NiFi was unable to complete the request because it was invalid. The request should not be retried without modification."),
+ @ApiResponse(code = 401, message = "Client could not be authenticated."),
+ @ApiResponse(code = 403, message = "Client is not authorized to make this request."),
+ @ApiResponse(code = 404, message = "The specified resource could not be found."),
+ @ApiResponse(code = 409, message = "The request was valid but NiFi was not in the appropriate state to process it. Retrying the same request later may be successful.")
+ }
+ )
+ public Response uploadProcessGroup(
+ @Context final HttpServletRequest httpServletRequest,
+ @ApiParam(
+ value = "The process group id.",
+ required = true
+ )
+ @PathParam("id") final String groupId,
+ @ApiParam(
+ value = "The process group name.",
+ required = true
+ )
+ @FormDataParam("groupName") final String groupName,
+ @ApiParam(
+ value = "The process group X position.",
+ required = true
+ )
+ @FormDataParam("position-x") final Double positionX,
+ @ApiParam(
+ value = "The process group Y position.",
+ required = true
+ )
+ @FormDataParam("position-y") final Double positionY,
+ @ApiParam(
+ value = "The client id.",
+ required = true
+ )
+ @FormDataParam("clientId") final String clientId,
+ @ApiParam(
+ value = "Acknowledges that this node is disconnected to allow for mutable requests to proceed.",
+ required = false
+ )
+ @FormDataParam(DISCONNECTED_NODE_ACKNOWLEDGED) @DefaultValue("false") final Boolean disconnectedNodeAcknowledged,
+ @FormDataParam("file") final InputStream in) throws IOException {
+
+ // ensure the group name is specified
+ if (StringUtils.isBlank(groupName)) {
+ throw new IllegalArgumentException("The process group name is required.");
+ }
+
+ if (StringUtils.isBlank(groupId)) {
+ throw new IllegalArgumentException("The parent process group id must be the same as specified in the URI.");
+ }
+
+ if (positionX == null) {
+ throw new IllegalArgumentException("The x coordinate of the proposed position must be specified.");
+ }
+
+ if (positionY == null) {
+ throw new IllegalArgumentException("The y coordinate of the proposed position must be specified.");
+ }
+
+ if (StringUtils.isBlank(clientId)) {
+ throw new IllegalArgumentException("The client id must be specified");
+ }
+
+ // create a new process group
+ final ProcessGroupEntity newProcessGroupEntity = new ProcessGroupEntity();
+
+ // get the contents of the InputStream as a String
+ String stringContent = null;
+ if (in != null) {
+ stringContent = IOUtils.toString(in, StandardCharsets.UTF_8);
+ }
+
+ // deserialize content to a VersionedFlowSnapshot
+ VersionedFlowSnapshot deserializedSnapshot = null;
+ final ObjectMapper mapper = new ObjectMapper();
+
+ if (stringContent.length() > 0) {
+ try {
+ mapper.setSerializationInclusion(JsonInclude.Include.NON_NULL);
+ mapper.setDefaultPropertyInclusion(JsonInclude.Value.construct(JsonInclude.Include.NON_NULL, JsonInclude.Include.NON_NULL));
+ mapper.setAnnotationIntrospector(new JaxbAnnotationIntrospector(mapper.getTypeFactory()));
+ mapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);
+
+ deserializedSnapshot = mapper.readValue(stringContent, VersionedFlowSnapshot.class);
+ } catch (JsonParseException jpe) {
+ logger.warn("The file uploaded is not a valid JSON format: ", jpe);
+ String responseJson = "The specified file is not a valid JSON format.";
Review comment:
This could be changed to a `private static final String` since it is always the same.
##########
File path: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/ProcessGroupResource.java
##########
@@ -4173,6 +4179,227 @@ private void sanitizeRegistryInfo(final VersionedProcessGroup versionedProcessGr
}
}
+ /**
+ * Uploads the specified versioned flow file and adds it to a new process group.
+ *
+ * @param httpServletRequest request
+ * @param in The flow file stream
+ * @return A processGroupEntity
+ * @throws IOException if there is an error during deserialization of the InputStream
+ */
+ @POST
+ @Consumes(MediaType.MULTIPART_FORM_DATA)
+ @Produces(MediaType.APPLICATION_JSON)
+ @Path("{id}/process-groups/upload")
+ @ApiOperation(
+ value = "Uploads a versioned flow file and creates a process group",
+ response = ProcessGroupEntity.class,
+ authorizations = {
+ @Authorization(value = "Write - /process-groups/{uuid}")
+ }
+ )
+ @ApiResponses(
+ value = {
+ @ApiResponse(code = 400, message = "NiFi was unable to complete the request because it was invalid. The request should not be retried without modification."),
+ @ApiResponse(code = 401, message = "Client could not be authenticated."),
+ @ApiResponse(code = 403, message = "Client is not authorized to make this request."),
+ @ApiResponse(code = 404, message = "The specified resource could not be found."),
+ @ApiResponse(code = 409, message = "The request was valid but NiFi was not in the appropriate state to process it. Retrying the same request later may be successful.")
+ }
+ )
+ public Response uploadProcessGroup(
+ @Context final HttpServletRequest httpServletRequest,
+ @ApiParam(
+ value = "The process group id.",
+ required = true
+ )
+ @PathParam("id") final String groupId,
+ @ApiParam(
+ value = "The process group name.",
+ required = true
+ )
+ @FormDataParam("groupName") final String groupName,
+ @ApiParam(
+ value = "The process group X position.",
+ required = true
+ )
+ @FormDataParam("position-x") final Double positionX,
+ @ApiParam(
+ value = "The process group Y position.",
+ required = true
+ )
+ @FormDataParam("position-y") final Double positionY,
+ @ApiParam(
+ value = "The client id.",
+ required = true
+ )
+ @FormDataParam("clientId") final String clientId,
+ @ApiParam(
+ value = "Acknowledges that this node is disconnected to allow for mutable requests to proceed.",
+ required = false
+ )
+ @FormDataParam(DISCONNECTED_NODE_ACKNOWLEDGED) @DefaultValue("false") final Boolean disconnectedNodeAcknowledged,
+ @FormDataParam("file") final InputStream in) throws IOException {
+
+ // ensure the group name is specified
+ if (StringUtils.isBlank(groupName)) {
+ throw new IllegalArgumentException("The process group name is required.");
+ }
+
+ if (StringUtils.isBlank(groupId)) {
+ throw new IllegalArgumentException("The parent process group id must be the same as specified in the URI.");
+ }
+
+ if (positionX == null) {
+ throw new IllegalArgumentException("The x coordinate of the proposed position must be specified.");
+ }
+
+ if (positionY == null) {
+ throw new IllegalArgumentException("The y coordinate of the proposed position must be specified.");
+ }
+
+ if (StringUtils.isBlank(clientId)) {
+ throw new IllegalArgumentException("The client id must be specified");
+ }
+
+ // create a new process group
+ final ProcessGroupEntity newProcessGroupEntity = new ProcessGroupEntity();
+
+ // get the contents of the InputStream as a String
+ String stringContent = null;
+ if (in != null) {
+ stringContent = IOUtils.toString(in, StandardCharsets.UTF_8);
+ }
+
+ // deserialize content to a VersionedFlowSnapshot
+ VersionedFlowSnapshot deserializedSnapshot = null;
+ final ObjectMapper mapper = new ObjectMapper();
+
+ if (stringContent.length() > 0) {
+ try {
+ mapper.setSerializationInclusion(JsonInclude.Include.NON_NULL);
+ mapper.setDefaultPropertyInclusion(JsonInclude.Value.construct(JsonInclude.Include.NON_NULL, JsonInclude.Include.NON_NULL));
+ mapper.setAnnotationIntrospector(new JaxbAnnotationIntrospector(mapper.getTypeFactory()));
+ mapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);
+
+ deserializedSnapshot = mapper.readValue(stringContent, VersionedFlowSnapshot.class);
+ } catch (JsonParseException jpe) {
+ logger.warn("The file uploaded is not a valid JSON format: ", jpe);
+ String responseJson = "The specified file is not a valid JSON format.";
+ return Response.status(Response.Status.OK).entity(responseJson).type("application/json").build();
+ } catch (IOException e) {
+ logger.warn("An error occurred while deserializing the flow version.", e);
Review comment:
This error message references a "flow version" but it should be more general.
```suggestion
logger.warn("Deserialization of uploaded JSON failed", e);
```
##########
File path: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/js/nf/canvas/header/components/nf-ng-group-component.js
##########
@@ -304,7 +461,7 @@
});
}).promise();
}
- }
+ };
Review comment:
Is there a reason for adding this semicolon?
##########
File path: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/ProcessGroupResource.java
##########
@@ -4173,6 +4179,227 @@ private void sanitizeRegistryInfo(final VersionedProcessGroup versionedProcessGr
}
}
+ /**
+ * Uploads the specified versioned flow file and adds it to a new process group.
+ *
+ * @param httpServletRequest request
+ * @param in The flow file stream
+ * @return A processGroupEntity
+ * @throws IOException if there is an error during deserialization of the InputStream
+ */
+ @POST
+ @Consumes(MediaType.MULTIPART_FORM_DATA)
+ @Produces(MediaType.APPLICATION_JSON)
+ @Path("{id}/process-groups/upload")
+ @ApiOperation(
+ value = "Uploads a versioned flow file and creates a process group",
+ response = ProcessGroupEntity.class,
+ authorizations = {
+ @Authorization(value = "Write - /process-groups/{uuid}")
+ }
+ )
+ @ApiResponses(
+ value = {
+ @ApiResponse(code = 400, message = "NiFi was unable to complete the request because it was invalid. The request should not be retried without modification."),
+ @ApiResponse(code = 401, message = "Client could not be authenticated."),
+ @ApiResponse(code = 403, message = "Client is not authorized to make this request."),
+ @ApiResponse(code = 404, message = "The specified resource could not be found."),
+ @ApiResponse(code = 409, message = "The request was valid but NiFi was not in the appropriate state to process it. Retrying the same request later may be successful.")
+ }
+ )
+ public Response uploadProcessGroup(
+ @Context final HttpServletRequest httpServletRequest,
+ @ApiParam(
+ value = "The process group id.",
+ required = true
+ )
+ @PathParam("id") final String groupId,
+ @ApiParam(
+ value = "The process group name.",
+ required = true
+ )
+ @FormDataParam("groupName") final String groupName,
+ @ApiParam(
+ value = "The process group X position.",
+ required = true
+ )
+ @FormDataParam("position-x") final Double positionX,
+ @ApiParam(
+ value = "The process group Y position.",
+ required = true
+ )
+ @FormDataParam("position-y") final Double positionY,
+ @ApiParam(
+ value = "The client id.",
+ required = true
+ )
+ @FormDataParam("clientId") final String clientId,
+ @ApiParam(
+ value = "Acknowledges that this node is disconnected to allow for mutable requests to proceed.",
+ required = false
+ )
+ @FormDataParam(DISCONNECTED_NODE_ACKNOWLEDGED) @DefaultValue("false") final Boolean disconnectedNodeAcknowledged,
+ @FormDataParam("file") final InputStream in) throws IOException {
+
+ // ensure the group name is specified
+ if (StringUtils.isBlank(groupName)) {
+ throw new IllegalArgumentException("The process group name is required.");
+ }
+
+ if (StringUtils.isBlank(groupId)) {
+ throw new IllegalArgumentException("The parent process group id must be the same as specified in the URI.");
+ }
+
+ if (positionX == null) {
+ throw new IllegalArgumentException("The x coordinate of the proposed position must be specified.");
+ }
+
+ if (positionY == null) {
+ throw new IllegalArgumentException("The y coordinate of the proposed position must be specified.");
+ }
+
+ if (StringUtils.isBlank(clientId)) {
+ throw new IllegalArgumentException("The client id must be specified");
+ }
+
+ // create a new process group
+ final ProcessGroupEntity newProcessGroupEntity = new ProcessGroupEntity();
+
+ // get the contents of the InputStream as a String
+ String stringContent = null;
+ if (in != null) {
+ stringContent = IOUtils.toString(in, StandardCharsets.UTF_8);
+ }
+
+ // deserialize content to a VersionedFlowSnapshot
+ VersionedFlowSnapshot deserializedSnapshot = null;
+ final ObjectMapper mapper = new ObjectMapper();
+
+ if (stringContent.length() > 0) {
+ try {
+ mapper.setSerializationInclusion(JsonInclude.Include.NON_NULL);
+ mapper.setDefaultPropertyInclusion(JsonInclude.Value.construct(JsonInclude.Include.NON_NULL, JsonInclude.Include.NON_NULL));
+ mapper.setAnnotationIntrospector(new JaxbAnnotationIntrospector(mapper.getTypeFactory()));
+ mapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);
+
+ deserializedSnapshot = mapper.readValue(stringContent, VersionedFlowSnapshot.class);
+ } catch (JsonParseException jpe) {
+ logger.warn("The file uploaded is not a valid JSON format: ", jpe);
Review comment:
The log message seems to indicate that there should be more information, recommend adjusting the invocation:
```suggestion
logger.warn("Parsing uploaded JSON failed", jpe);
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org