You are viewing a plain text version of this content. The canonical link for it is here.

Posted to notifications@apisix.apache.org by "coolboywcm (via GitHub)" <gi...@apache.org> on 2023/08/03 06:51:32 UTC

[GitHub] [apisix] coolboywcm opened a new issue, #9961: bug: http-logger reseponse body decode fail

coolboywcm opened a new issue, #9961:
URL: https://github.com/apache/apisix/issues/9961

   ### Current Behavior
   
   Version: APISIX/3.1.0
   After enabling the http-logger plug-in, if response has been compress by gzip, the response body receives garbled characters
   The configuration is as follows
   
   "http-logger": {
         "_meta": {
           "disable": false
         },
         "include_resp_body": true,
         "uri": "http://xxx.xx.xx.xx:80/apisix/addLog"
       }
   
   See the response body by http-logger:
   "response": {
       "body":"\u001F�\b\u0000\u0000\u0000\u0000\u0000\u0000\u0000�V�N�4T�R242626Q�\u0005\u0000��_�\u0011\u0000\u0000\u0000",
           "headers": {
               "content-encoding": "gzip",
               "content-length": "37",
               "content-type": "application/json;charset=UTF-8",
               "date": "Thu, 03 Aug 2023 06:32:24 GMT",
               "server": "APISIX/3.1.0",
               ........
           },
           "size": 377,
           "status": 200
   }
   
   The upstream server uses gzip to compress the response, and the http-logger plug-in records the response-body, directly converting the gzip content into a string, resulting in garbled characters
   
   ### Expected Behavior
   
   "response": {
       "body":{"key1": "123234"},
       "headers": {
               "content-encoding": "gzip",
               "content-length": "37",
               "content-type": "application/json;charset=UTF-8",
               "date": "Thu, 03 Aug 2023 06:32:24 GMT",
               "server": "APISIX/3.1.0",
               ........
           },
           "size": 377,
           "status": 200
   }
   
   ### Error Logs
   
   _No response_
   
   ### Steps to Reproduce
   
   Open the  plug-in http-logger in route
   Call the route interface
   View the http-logger uri server console and view the log message
   
   ### Environment
   
   APISIX version (3.1.0):
   Operating system (CentOS Linux release 7.6.1810):
   OpenResty / Nginx version (nginx version: openresty/1.21.4.1):
   etcd version, if relevant ("etcd_version":"3.4.0"):
   APISIX Dashboard version, if relevant(3.0.0)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [apisix] coolboywcm commented on issue #9961: bug: http-logger reseponse body decode fail

Posted by "coolboywcm (via GitHub)" <gi...@apache.org>.

coolboywcm commented on issue #9961:
URL: https://github.com/apache/apisix/issues/9961#issuecomment-1665119637

   > 
   "uri": "http://xxx.xx.xx.xx:80/apisix/addLog"
   
   controller method  addLog  is as follows
   
   ```
   import com.alibaba.fastjson.JSONArray;
   import com.alibaba.fastjson.JSONObject;
   
   public void addLog() throws IOException {
           String encoding = this.getRequest().getCharacterEncoding();
           System.out.println("1encoding=========================================>" + encoding);
           if (encoding == null) {
               encoding = StandardCharsets.UTF_8.name(); 
           }
           System.out.println("2encoding=========================================>" + encoding);
           StringBuilder stringBuilder = new StringBuilder();
           BufferedReader reader = new BufferedReader(new InputStreamReader(this.getRequest().getInputStream(), encoding));
           String line;
           while ((line = reader.readLine()) != null) {
               stringBuilder.append(line);
           }
         System.out.println("00000000000000stringBuilder.toString()============================================>" + stringBuilder.toString());
           String jsonString = stringBuilder.toString();
           JSONObject logJson = JSONArray.parseArray(jsonString).getJSONObject(0);
           jsonString = logJson.toJSONString();
           System.out.println("3333333333333jsonString============================================>" + jsonString);
           String resp_body = logJson.getJSONObject("response").getString("body");
           System.out.println("addLog=================================>111resp_body:" + resp_body);
           resp_body = decompress(resp_body.getBytes(encoding);
           System.out.println("addLog=================================>222resp_body:" + resp_body);
   }
   ```
   method  decompress  is as follows
   
   ```
   import java.util.zip.GZIPInputStream;
   import java.util.zip.GZIPOutputStream;
   public static String decompress(byte[] compressedData) throws IOException {
           try (ByteArrayInputStream bis = new ByteArrayInputStream(compressedData);
                ByteArrayOutputStream bos = new ByteArrayOutputStream();
                GZIPInputStream gzipIS = new GZIPInputStream(bis)) {
   
               byte[] buffer = new byte[1024];
               int len;
               while ((len = gzipIS.read(buffer)) > 0) {
                   bos.write(buffer, 0, len);
               }
               return bos.toString("UTF-8");
           }
       }
   ```
   
   console print result:
   
   ```
   1encoding=========================================>UTF-8
   2encoding=========================================>UTF-8
   00000000000000stringBuilder.toString()============================================>[{"response":{"status":200,"headers":{"connection":"close","access-control-allow-origin":"*","server":"APISIX\/3.1.0","content-type":"application\/json;charset=UTF-8","content-encoding":"gzip","date":"Fri, 04 Aug 2023 06:52:28 GMT","access-control-allow-headers":"Content-Type,X-Requested-With,withCredentials","access-control-allow-methods":"GET, POST","content-length":"37"},"body":"\u001f�\b\u0000\u0000\u0000\u0000\u0000\u0000\u0000�V�N�4T�R242626Q�\u0005\u0000��_�\u0011\u0000\u0000\u0000","size":377}}]
   3333333333333jsonString============================================>{"response":{"status":200,"headers":{"connection":"close","access-control-allow-origin":"*","server":"APISIX\/3.1.0","content-type":"application\/json;charset=UTF-8","content-encoding":"gzip","date":"Fri, 04 Aug 2023 06:52:28 GMT","access-control-allow-headers":"Content-Type,X-Requested-With,withCredentials","access-control-allow-methods":"GET, POST","content-length":"37"},"body":"\u001f�\b\u0000\u0000\u0000\u0000\u0000\u0000\u0000�V�N�4T�R242626Q�\u0005\u0000��_�\u0011\u0000\u0000\u0000","size":377}}
   addLog=================================>111resp_body:\u001F�\b\u0000\u0000\u0000\u0000\u0000\u0000\u0000�V�N�4T�R242626Q�\u0005\u0000��_�\u0011\u0000\u0000\u0000
   java.util.zip.ZipException: Not in GZIP format
           at java.util.zip.GZIPInputStream.readHeader(GZIPInputStream.java:165)
           at java.util.zip.GZIPInputStream.<init>(GZIPInputStream.java:79)
           at java.util.zip.GZIPInputStream.<init>(GZIPInputStream.java:91)
           at com.ApiSixController.decompress(ApiSixController.java:332)
   
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [apisix] coolboywcm commented on issue #9961: bug: http-logger reseponse body decode fail

Posted by "coolboywcm (via GitHub)" <gi...@apache.org>.

coolboywcm commented on issue #9961:
URL: https://github.com/apache/apisix/issues/9961#issuecomment-1665139768

   > > > @coolboywcm This is not a bug but a feature request. Currently the logger doesn't decompress the compressed body and logs the body as it is.
   > > 
   > > 
   > > I tried to decompress and convert the response-body of http-logger into a byte stream, but it failed. During the process of converting the gzip content into a string forcibly by http-logger, the original data has been damaged.
   > 
   > For repro purposes, can you show the code how you were decompressing the body?
   
   In the above reply, I provided the code and the result


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org