You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by "exceptionfactory (via GitHub)" <gi...@apache.org> on 2026/06/02 13:52:49 UTC

[PR] NIFI-15888 Add size limit to Standard Content Viewer [nifi]

exceptionfactory opened a new pull request, #11303:
URL: https://github.com/apache/nifi/pull/11303

   # Summary
   
   [NIFI-15888](https://issues.apache.org/jira/browse/NIFI-15888) Adds an internal size limit of 10 MB to the Standard Content Viewer, avoiding attempts to load, format, and render large FlowFiles.
   
   The internal limit of 10 MB aligns with a previous comment on Apache Avro formatting, which actually limited to the output size to 2 MB.
   
   The implementation uses the `LimitingInputStream` to avoid reading more than 10 MB from the referenced FlowFile. The limit applies to both formatted and unformatted responses. Unformatted responses with content exceeding the limit return an `HTTP 206` status indicating Partial Content returned. Formatted may throw an exception when the limit does not align with expected structural boundaries. In this case, the server returns an HTTP 500 with message indicating length exceeds maximum. As a best effort viewer, returning an error avoids both server and client memory consumption concerns. The hexadecimal viewer continues to provide a general way to view initial bytes.
   
   Additional unit tests exercise length limiting behavior with formatted and unformatted responses.
   
   # Tracking
   
   Please complete the following tracking steps prior to pull request creation.
   
   ### Issue Tracking
   
   - [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created
   
   ### Pull Request Tracking
   
   - Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-00000`
   - Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-00000`
   - Pull request contains [commits signed](https://docs.github.com/en/authentication/managing-commit-signature-verification/signing-commits) with a registered key indicating `Verified` status
   
   ### Pull Request Formatting
   
   - Pull Request based on current revision of the `main` branch
   - Pull Request refers to a feature branch with one commit containing changes
   
   # Verification
   
   Please indicate the verification steps performed prior to pull request creation.
   
   ### Build
   
   - [X] Build completed using `./mvnw clean install -P contrib-check`
     - [X] JDK 21
     - [X] JDK 25
   
   ### Licensing
   
   - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html)
   - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files
   
   ### Documentation
   
   - [ ] Documentation formatting appears as expected in rendered files
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] NIFI-15888 Add size limit to Standard Content Viewer [nifi]

Posted by "exceptionfactory (via GitHub)" <gi...@apache.org>.
exceptionfactory commented on code in PR #11303:
URL: https://github.com/apache/nifi/pull/11303#discussion_r3342918747


##########
nifi-extension-bundles/nifi-standard-bundle/nifi-standard-content-viewer/src/test/java/org/apache/nifi/web/controller/StandardContentViewerControllerTest.java:
##########
@@ -67,8 +75,39 @@ class StandardContentViewerControllerTest {
     @Mock
     private ContentAccess contentAccess;
 
+    @Captor
+    private ArgumentCaptor<String> errorMessageCaptor;
+
     private final StandardContentViewerController controller = new StandardContentViewerController();
 
+    @Test
+    void testDoGetFormattedContentTypeNotSupported() throws IOException {
+        final InputStream contentStream = new ByteArrayInputStream(XML_DOCUMENT.getBytes(StandardCharsets.UTF_8));
+        final DownloadableContent downloadableContent = new DownloadableContent(FILENAME, APPLICATION_OCTET_STREAM, contentStream);
+        setDownloadableContent(downloadableContent);
+        when(request.getParameter(eq(FORMATTED_PARAMETER))).thenReturn(Boolean.TRUE.toString());
+        when(request.getParameter(eq(CLIENT_ID_PARAMETER))).thenReturn(UUID.randomUUID().toString());
+
+        controller.doGet(request, response);
+
+        verify(response).sendError(eq(HttpServletResponse.SC_NOT_ACCEPTABLE), anyString());
+    }
+
+    @Test
+    void testDoGetNotFormattedPartialContent() throws IOException {
+        final InputStream contentStream = new ByteArrayInputStream(XML_DOCUMENT.getBytes(StandardCharsets.UTF_8));
+        final DownloadableContent downloadableContent = new DownloadableContent(FILENAME, APPLICATION_OCTET_STREAM, contentStream);
+        setDownloadableContent(downloadableContent);
+
+        final MockServletOutputStream mockServletOutputStream = new MockServletOutputStream();
+        when(response.getOutputStream()).thenReturn(mockServletOutputStream);
+
+        final LimitedStandardContentViewerController limitedController = new LimitedStandardContentViewerController();
+        limitedController.doGet(request, response);
+
+        verify(response).setStatus(eq(HttpServletResponse.SC_PARTIAL_CONTENT));

Review Comment:
   Thanks for pointing this out, I reworked the implementation to set the response status earlier.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] NIFI-15888 Add size limit to Standard Content Viewer [nifi]

Posted by "pvillard31 (via GitHub)" <gi...@apache.org>.
pvillard31 commented on code in PR #11303:
URL: https://github.com/apache/nifi/pull/11303#discussion_r3342454716


##########
nifi-extension-bundles/nifi-standard-bundle/nifi-standard-content-viewer/src/test/java/org/apache/nifi/web/controller/StandardContentViewerControllerTest.java:
##########
@@ -67,8 +75,39 @@ class StandardContentViewerControllerTest {
     @Mock
     private ContentAccess contentAccess;
 
+    @Captor
+    private ArgumentCaptor<String> errorMessageCaptor;
+
     private final StandardContentViewerController controller = new StandardContentViewerController();
 
+    @Test
+    void testDoGetFormattedContentTypeNotSupported() throws IOException {
+        final InputStream contentStream = new ByteArrayInputStream(XML_DOCUMENT.getBytes(StandardCharsets.UTF_8));
+        final DownloadableContent downloadableContent = new DownloadableContent(FILENAME, APPLICATION_OCTET_STREAM, contentStream);
+        setDownloadableContent(downloadableContent);
+        when(request.getParameter(eq(FORMATTED_PARAMETER))).thenReturn(Boolean.TRUE.toString());
+        when(request.getParameter(eq(CLIENT_ID_PARAMETER))).thenReturn(UUID.randomUUID().toString());
+
+        controller.doGet(request, response);
+
+        verify(response).sendError(eq(HttpServletResponse.SC_NOT_ACCEPTABLE), anyString());
+    }
+
+    @Test
+    void testDoGetNotFormattedPartialContent() throws IOException {
+        final InputStream contentStream = new ByteArrayInputStream(XML_DOCUMENT.getBytes(StandardCharsets.UTF_8));
+        final DownloadableContent downloadableContent = new DownloadableContent(FILENAME, APPLICATION_OCTET_STREAM, contentStream);
+        setDownloadableContent(downloadableContent);
+
+        final MockServletOutputStream mockServletOutputStream = new MockServletOutputStream();
+        when(response.getOutputStream()).thenReturn(mockServletOutputStream);
+
+        final LimitedStandardContentViewerController limitedController = new LimitedStandardContentViewerController();
+        limitedController.doGet(request, response);
+
+        verify(response).setStatus(eq(HttpServletResponse.SC_PARTIAL_CONTENT));

Review Comment:
   Since MockServletOutputStream never commits the response, does this test really verify that a real container would send 206 after the body is written?



##########
nifi-extension-bundles/nifi-standard-bundle/nifi-standard-content-viewer/src/main/java/org/apache/nifi/web/controller/StandardContentViewerController.java:
##########
@@ -80,38 +92,47 @@ public void doGet(final HttpServletRequest request, final HttpServletResponse re
             return;
         }
 
-        response.setStatus(HttpServletResponse.SC_OK);
-
+        final LimitingInputStream contentStream = getContentStream(downloadableContent);
         final boolean formatted = Boolean.parseBoolean(request.getParameter("formatted"));
-        if (!formatted) {
-            final InputStream contentStream = downloadableContent.getContent();
+        if (formatted) {
+            final ContentType formattedContentType = getFormattedContentType(request, downloadableContent.getType());
+            if (formattedContentType == null) {
+                response.sendError(HttpURLConnection.HTTP_NOT_ACCEPTABLE, "Unknown Content Type");
+            } else {
+                final String dataUri = requestContext.getDataUri();
+                writeContentFormatted(dataUri, formattedContentType, contentStream, response);
+                response.setStatus(HttpServletResponse.SC_OK);
+            }
+        } else {
             contentStream.transferTo(response.getOutputStream());

Review Comment:
   For content larger than the response buffer, the response commits during transferTo, so does the later setStatus(SC_PARTIAL_CONTENT) actually reach the client, or does it silently return 200 with truncated content?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] NIFI-15888 Add size limit to Standard Content Viewer [nifi]

Posted by "pvillard31 (via GitHub)" <gi...@apache.org>.
pvillard31 merged PR #11303:
URL: https://github.com/apache/nifi/pull/11303


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] NIFI-15888 Add size limit to Standard Content Viewer [nifi]

Posted by "exceptionfactory (via GitHub)" <gi...@apache.org>.
exceptionfactory commented on PR #11303:
URL: https://github.com/apache/nifi/pull/11303#issuecomment-4604692714

   Thanks for the review @pvillard31, good catch on the order of setting the response status.
   
   I made some adjustments to get the source content length and pass it to the `DownloadableContent` object, enabling determination of partial content status prior to stream handling.
   
   The `LimitingInputStream` still enforces the limit, even if the content length is something unknown, like `-1`, but the response status determination provides a best-effort approach to indicate whether truncation occurred.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] NIFI-15888 Add size limit to Standard Content Viewer [nifi]

Posted by "exceptionfactory (via GitHub)" <gi...@apache.org>.
exceptionfactory commented on code in PR #11303:
URL: https://github.com/apache/nifi/pull/11303#discussion_r3342915377


##########
nifi-extension-bundles/nifi-standard-bundle/nifi-standard-content-viewer/src/main/java/org/apache/nifi/web/controller/StandardContentViewerController.java:
##########
@@ -80,38 +92,47 @@ public void doGet(final HttpServletRequest request, final HttpServletResponse re
             return;
         }
 
-        response.setStatus(HttpServletResponse.SC_OK);
-
+        final LimitingInputStream contentStream = getContentStream(downloadableContent);
         final boolean formatted = Boolean.parseBoolean(request.getParameter("formatted"));
-        if (!formatted) {
-            final InputStream contentStream = downloadableContent.getContent();
+        if (formatted) {
+            final ContentType formattedContentType = getFormattedContentType(request, downloadableContent.getType());
+            if (formattedContentType == null) {
+                response.sendError(HttpURLConnection.HTTP_NOT_ACCEPTABLE, "Unknown Content Type");
+            } else {
+                final String dataUri = requestContext.getDataUri();
+                writeContentFormatted(dataUri, formattedContentType, contentStream, response);
+                response.setStatus(HttpServletResponse.SC_OK);
+            }
+        } else {
             contentStream.transferTo(response.getOutputStream());

Review Comment:
   Good catch, the test behavior masked this important detail. I adjusted the approach to get the expected Content Length from the source, and used that to get and set the response status prior to handling the stream.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org