[GitHub] [hadoop] anmolanmol1234 commented on a diff in pull request #4039: HADOOP-18146: ABFS: Added changes for expect hundred continue header

[GitBox <gi...@apache.org>]

anmolanmol1234 commented on code in PR #4039:
URL: https://github.com/apache/hadoop/pull/4039#discussion_r1040936094


##########
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsHttpOperation.java:
##########
@@ -314,18 +314,21 @@ public void sendRequest(byte[] buffer, int offset, int length) throws IOExceptio
     if (this.isTraceEnabled) {
       startTime = System.nanoTime();
     }
-    try (OutputStream outputStream = this.connection.getOutputStream()) {
-      // update bytes sent before they are sent so we may observe
-      // attempted sends as well as successful sends via the
-      // accompanying statusCode
-      this.bytesSent = length;
+    OutputStream outputStream;
+    try {
+      try {
+        outputStream = this.connection.getOutputStream();
+      } catch (IOException e) {
+        // If getOutputStream fails with an exception due to 100-continue

Review Comment:
   1. The first point is valid, I have made the change where getOutputStream throws exception for the cases where 100 continue is not enabled and returns back to the caller when it catches an IOException due to 100 continue being enabled which leads to processResponse getting the correct status code and then eventually the retry logic coming into play. 
   
   2. We need to update the bytes sent for failed as well as passed cases. The current change will not swallow any exceptions.
   The handling for various status code with 100 continue enabled is as follows 
   
   1. Case 1 :- getOutputStream doesn't throw any exception, response is processed and it gives status code of 200, no retry is needed and hence the request succeeds.
   2. Case 2:- getOutputSteam throws exception, we return to the caller and in processResponse in this.connection.getResponseCode() it gives status code of 404 (user error), exponential retry is not needed. We retry without 100 continue enabled.
   3. Case 3:- getOutputSteam throws exception, we return to the caller and in processResponse  it gives status code of 503,
   which shows throttling so we backoff accordingly with exponential retry. Since each append request waits for 100 continue response, the stress on the server gets reduced.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org