[GitHub] [knox] smolnar82 opened a new pull request, #747: KNOX-2897 - Using Expect:100-continue wherever possible to upload big content

["smolnar82 (via GitHub)" <gi...@apache.org>]

smolnar82 opened a new pull request, #747:
URL: https://github.com/apache/knox/pull/747

   ## What changes were proposed in this pull request?
   
   This commit consists of the following changes:
   - the new-style "Expect:100-continue" approach for PUT/POST requests is disabled by default
   - the new-style "Expect:100-continue" approach is enabled for the ATLAS, ATLAS-API, HIVE, RANGER and RANGER-UI services
   - make sure that this feature for the WEBHDFS service is turned off even in its own service definition file (so that if the default is changed back to true, we'll be safe)
   - HTTP entity creation relies on the 'addExpect100Continue' boolean flag
   - implemented the missing method overload in HdfsHttpClientDispatch
   - KnoxBufferedHttpEntity shall not read content at entity creation time (content reading is postponed until dispatch time when the back-end service actually needs the data)
   
   ## How was this patch tested?
   
   Manual testing on a live cluster with the affected services (ATLAS, HIVE, RANGER)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@knox.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [knox] risdenk commented on a diff in pull request #747: KNOX-2897 - Using Expect:100-continue wherever possible to upload big content

["risdenk (via GitHub)" <gi...@apache.org>]

risdenk commented on code in PR #747:
URL: https://github.com/apache/knox/pull/747#discussion_r1163155678


##########
gateway-spi/src/main/java/org/apache/knox/gateway/dispatch/KnoxBufferedHttpEntity.java:
##########
@@ -0,0 +1,102 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.knox.gateway.dispatch;
+
+import java.io.ByteArrayInputStream;
+import java.io.ByteArrayOutputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+
+import org.apache.http.HttpEntity;
+import org.apache.http.entity.BufferedHttpEntity;
+import org.apache.http.entity.HttpEntityWrapper;
+import org.apache.http.util.Args;
+
+/**
+ * A {@link BufferedHttpEntity} implementation with the following differences:
+ * <ul>
+ * <li>Reading the content of the given entity does not happen at instance creation time (i.e not in the constructor)
+ * <li>Content length calculation is not based on the buffer but a delegate to the wrapped entity
+ * </ul>
+ */
+public class KnoxBufferedHttpEntity extends HttpEntityWrapper {
+
+  private byte[] buffer;

Review Comment:
   There are no bounds checks for copying to this buffer? Can someone just crash Knox by sending a really large content stream that is not repeatable?
   
   I thought that was part of the theory behind why there was a replay buffer size - to avoid copying the entire stream into memory.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@knox.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org