You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by "kevdoran (via GitHub)" <gi...@apache.org> on 2026/04/02 12:12:05 UTC

[PR] NIFI-15784 Forward request headers during upload replication [nifi]

kevdoran opened a new pull request, #11094:
URL: https://github.com/apache/nifi/pull/11094

   <!-- Licensed to the Apache Software Foundation (ASF) under one or more -->
   <!-- contributor license agreements.  See the NOTICE file distributed with -->
   <!-- this work for additional information regarding copyright ownership. -->
   <!-- The ASF licenses this file to You under the Apache License, Version 2.0 -->
   <!-- (the "License"); you may not use this file except in compliance with -->
   <!-- the License.  You may obtain a copy of the License at -->
   <!--     http://www.apache.org/licenses/LICENSE-2.0 -->
   <!-- Unless required by applicable law or agreed to in writing, software -->
   <!-- distributed under the License is distributed on an "AS IS" BASIS, -->
   <!-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -->
   <!-- See the License for the specific language governing permissions and -->
   <!-- limitations under the License. -->
   
   # Summary
   
   [NIFI-15784](https://issues.apache.org/jira/browse/NIFI-15784)
   
   UploadRequest replication (used for asset and NAR uploads) did not forward inbound servlet headers to peer nodes. Only three explicit headers (filename, content-type, cluster-id-generation-seed) were included. This caused ConnectorRequestContext to not have the full original request context.
   
   Standard cluster replication (ThreadPoolRequestReplicator) already forwards all inbound headers via ApplicationResource.getHeaders(), with appropriate stripping of credentials, hop-by-hop headers, and replication protocol headers. Upload replication now use the same approach for consistency.
   
   - Extract shared header-preparation logic into ReplicationHeaderUtils so credential stripping, proxied-entity injection, and replication header sanitisation cannot drift between the two replication paths
   - Add UploadRequest.forwardRequestHeaders(Map) so callers can opt in to forwarding inbound servlet headers
   - Rebuild outbound headers in StandardUploadRequestReplicator using a secure pipeline: strip replication headers, strip hop-by-hop, apply explicit builder headers, apply proxy/credential policy, force-set replication flags
   - Wire all current upload call sites (ConnectorResource.createAsset, ParameterContextResource asset upload, ControllerResource.uploadNar) to forward inbound headers
   - Use File.length() instead of InputStream.available() for reliable content-length in upload replication
   
   # Tracking
   
   Please complete the following tracking steps prior to pull request creation.
   
   ### Issue Tracking
   
   - [x] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created
   
   ### Pull Request Tracking
   
   - [x] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-00000`
   - [x] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-00000`
   - [x] Pull request contains [commits signed](https://docs.github.com/en/authentication/managing-commit-signature-verification/signing-commits) with a registered key indicating `Verified` status
   
   ### Pull Request Formatting
   
   - [x] Pull Request based on current revision of the `main` branch
   - [x] Pull Request refers to a feature branch with one commit containing changes
   
   # Verification
   
   Please indicate the verification steps performed prior to pull request creation.
   
   ### Build
   
   - [x] Build completed using `./mvnw clean install -P contrib-check`
     - [x] JDK 21
     - [ ] JDK 25
   
   ### Licensing
   
   - [x] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html)
   - [x] New dependencies are documented in applicable `LICENSE` and `NOTICE` files
   
   ### Documentation
   
   - [x] Documentation formatting appears as expected in rendered files
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] NIFI-15784 Forward request headers during upload replication [nifi]

Posted by "kevdoran (via GitHub)" <gi...@apache.org>.
kevdoran commented on PR #11094:
URL: https://github.com/apache/nifi/pull/11094#issuecomment-4178181502

   Thanks for the review @exceptionfactory. I addressed those test issues, good catch.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] NIFI-15784 Forward request headers during upload replication [nifi]

Posted by "kevdoran (via GitHub)" <gi...@apache.org>.
kevdoran commented on code in PR #11094:
URL: https://github.com/apache/nifi/pull/11094#discussion_r3028583958


##########
nifi-framework-bundle/nifi-framework/nifi-framework-cluster/src/main/java/org/apache/nifi/cluster/coordination/http/replication/ReplicationHeaderUtils.java:
##########
@@ -0,0 +1,148 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.cluster.coordination.http.replication;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.authorization.user.NiFiUser;
+import org.apache.nifi.web.security.ProxiedEntitiesUtils;
+import org.apache.nifi.web.security.http.SecurityCookieName;
+import org.apache.nifi.web.security.http.SecurityHeader;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.util.Map;
+import java.util.Optional;
+import java.util.Set;
+import java.util.stream.Collectors;
+import java.util.stream.Stream;
+
+/**
+ * Shared header-preparation logic for cluster request replication. Both {@link ThreadPoolRequestReplicator}
+ * and {@link StandardUploadRequestReplicator} use these utilities so that credential stripping, proxied-entity
+ * injection, and replication-header sanitisation follow the same policy.
+ */
+public final class ReplicationHeaderUtils {
+
+    private static final Logger logger = LoggerFactory.getLogger(ReplicationHeaderUtils.class);
+
+    private static final String COOKIE_HEADER = "Cookie";
+    private static final String HOST_HEADER = "Host";
+
+    /**
+     * Headers that must not be forwarded because they are hop-by-hop or describe the original
+     * transport framing rather than the replicated request.
+     *
+     * <p>Must stay in sync with
+     * {@link org.apache.nifi.cluster.coordination.http.replication.client.StandardHttpReplicationClient#DISALLOWED_HEADERS StandardHttpReplicationClient.DISALLOWED_HEADERS}.
+     */
+    static final Set<String> HOP_BY_HOP_HEADERS = Set.of(
+            "connection", "content-length", "transfer-encoding", "expect", "host", "upgrade"

Review Comment:
   Good call, `content-encoding` and `te` (transfer encoding) headers from the original http request also need to be stripped when replicating.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] NIFI-15784 Forward request headers during upload replication [nifi]

Posted by "mcgilman (via GitHub)" <gi...@apache.org>.
mcgilman commented on code in PR #11094:
URL: https://github.com/apache/nifi/pull/11094#discussion_r3028412534


##########
nifi-framework-bundle/nifi-framework/nifi-framework-cluster/src/main/java/org/apache/nifi/cluster/coordination/http/replication/ReplicationHeaderUtils.java:
##########
@@ -0,0 +1,148 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.cluster.coordination.http.replication;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.authorization.user.NiFiUser;
+import org.apache.nifi.web.security.ProxiedEntitiesUtils;
+import org.apache.nifi.web.security.http.SecurityCookieName;
+import org.apache.nifi.web.security.http.SecurityHeader;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.util.Map;
+import java.util.Optional;
+import java.util.Set;
+import java.util.stream.Collectors;
+import java.util.stream.Stream;
+
+/**
+ * Shared header-preparation logic for cluster request replication. Both {@link ThreadPoolRequestReplicator}
+ * and {@link StandardUploadRequestReplicator} use these utilities so that credential stripping, proxied-entity
+ * injection, and replication-header sanitisation follow the same policy.
+ */
+public final class ReplicationHeaderUtils {
+
+    private static final Logger logger = LoggerFactory.getLogger(ReplicationHeaderUtils.class);
+
+    private static final String COOKIE_HEADER = "Cookie";
+    private static final String HOST_HEADER = "Host";
+
+    /**
+     * Headers that must not be forwarded because they are hop-by-hop or describe the original
+     * transport framing rather than the replicated request.
+     *
+     * <p>Must stay in sync with
+     * {@link org.apache.nifi.cluster.coordination.http.replication.client.StandardHttpReplicationClient#DISALLOWED_HEADERS StandardHttpReplicationClient.DISALLOWED_HEADERS}.
+     */
+    static final Set<String> HOP_BY_HOP_HEADERS = Set.of(
+            "connection", "content-length", "transfer-encoding", "expect", "host", "upgrade"

Review Comment:
   I believe that `accept-encoding` needs to be added here.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] NIFI-15784 Forward request headers during upload replication [nifi]

Posted by "exceptionfactory (via GitHub)" <gi...@apache.org>.
exceptionfactory merged PR #11094:
URL: https://github.com/apache/nifi/pull/11094


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] NIFI-15784 Forward request headers during upload replication [nifi]

Posted by "pvillard31 (via GitHub)" <gi...@apache.org>.
pvillard31 commented on code in PR #11094:
URL: https://github.com/apache/nifi/pull/11094#discussion_r3028600785


##########
nifi-framework-bundle/nifi-framework/nifi-framework-cluster/src/test/java/org/apache/nifi/cluster/coordination/http/replication/TestReplicationHeaderUtils.java:
##########
@@ -0,0 +1,161 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.cluster.coordination.http.replication;
+
+import org.apache.nifi.authorization.user.StandardNiFiUser;
+import org.apache.nifi.web.security.ProxiedEntitiesUtils;
+import org.junit.jupiter.api.Test;
+
+import java.util.HashMap;
+import java.util.Map;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertFalse;
+import static org.junit.jupiter.api.Assertions.assertNotNull;
+import static org.junit.jupiter.api.Assertions.assertNull;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+class TestReplicationHeaderUtils {
+
+    private static final String TEST_USER_IDENTITY = "alice";
+
+    private static final String AUTHORIZATION_HEADER = "Authorization";
+    private static final String AUTHORIZATION_VALUE = "Bearer secret";
+
+    private static final String CUSTOM_HEADER = "X-Custom-Token";
+    private static final String CUSTOM_HEADER_VALUE = "custom-token-123";
+
+    private static final String COOKIE_HEADER = "Cookie";
+    private static final String HOST_HEADER = "Host";
+    private static final String HOST_VALUE = "original-host:8080";
+
+    private static final String ACCEPT_ENCODING_HEADER = "Accept-Encoding";
+    private static final String CONTENT_ENCODING_HEADER = "Content-Encoding";
+    private static final String CONTENT_LENGTH_HEADER = "Content-Length";
+    private static final String TE_HEADER = "TE";
+    private static final String TRANSFER_ENCODING_HEADER = "Transfer-Encoding";
+    private static final String CONNECTION_HEADER = "Connection";
+
+    private static final String SPOOFED_VALUE = "spoofed";
+    private static final String SHOULD_SURVIVE_VALUE = "should-survive";
+
+    @Test
+    void testApplyUserProxyAndStripCredentialsSetsProxiedEntities() {
+        final var user = new StandardNiFiUser.Builder().identity(TEST_USER_IDENTITY).build();

Review Comment:
   PMD warning



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org