decompressing gzipped response?

[Arya F <ar...@gmail.com>]

A while ago I asked this question but I sent it to the wrong mailing list.
Hopefully someone here can give me a solution.

I'm using Apache HTTPClient 4.5.3 to make some HTTP requests, I am getting
a gzipped response back. I'm looking for a way to decpde/convert the
gzipped content. I have tried many things I found online but non of them
worked. I still get gibberish when I print the response. Below is the
relevant code. I do understand that if I remove the
request.addHeader("Accept-Encoding", "gzip"); I will get plain readable
results back, but I need a way to convert/decode the gzipped content. What
are my options for achieving this? please don't suggest that I can just
remove request.addHeader("Accept-Encoding", "gzip"); I am looking for a way
to decode the gzipped content

static public CloseableHttpClient CreateHttpClient() {
    // return
    // HttpClients.custom().disableAutomaticRetries().setHttpProcessor(HttpProcessorBuilder.create().build())
    // .build();

    return HttpClientBuilder.create().disableAutomaticRetries()
            .setHttpProcessor(HttpProcessorBuilder.create().build()).build();
}

static public RequestConfig GetConfig() {
    return RequestConfig.custom().setSocketTimeout(READTIMEOUT).setConnectTimeout(CONNECTTIMEOUT)
            .setConnectionRequestTimeout(REQUESTTIMEOUT).build();
}

static public String updates() {
    String result = "";
    String url = "https://example.com";
    CloseableHttpClient httpClient = CreateHttpClient();
    CloseableHttpResponse response = null;
    URL urlObj;
    RequestConfig config = GetConfig();

    try {
        urlObj = new URL(url);

        HttpPost request = new HttpPost(url);
        request.setConfig(config);

        StringEntity params = new StringEntity("example");

        request.addHeader("Accept-Language", "en");
        request.addHeader("Content-Type", "application/json; charset=UTF-8");
        request.addHeader("Content-Length",
String.valueOf(params.getContentLength()));
        request.addHeader("Host", urlObj.getHost());
        request.addHeader("Connection", "Keep-Alive");
        request.addHeader("Accept-Encoding", "gzip");

        request.setEntity(params);
        response = httpClient.execute(request);

        int responseCode = response.getStatusLine().getStatusCode();

        System.out.println("updates response code: " + responseCode);


        // BufferedReader rd = new BufferedReader(new
        // InputStreamReader(response.getEntity().getContent(), "UTF-8"));

        result = EntityUtils.toString(response.getEntity());
        // String line = "";
        // while ((line = rd.readLine()) != null) {
        // result.append(line);
        // }

    } catch (Exception e) {
        e.printStackTrace();
    } finally {
        try {
            if (response != null)
                response.close();
            httpClient.close();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }

    return result;
}

RE: decompressing gzipped response?

[Yossi Tamari <yo...@yossi.at>]

This is supposed to work out of the box.
My suggestion would be to remove the setHttpProcessor call, remove all the addHeader calls (but pass the content type to the StringEntity creation), as the necessary headers are added automatically, and see what happens. If it still doesn't work I would suggest using Wireshark to analyse the request (which will be easier if you use HTTP instead of HTTPS).

> -----Original Message-----
> From: Arya F [mailto:arya6000@gmail.com]
> Sent: 16 November 2017 12:16
> To: HttpClient User Discussion <ht...@hc.apache.org>
> Subject: decompressing gzipped response?
> 
> A while ago I asked this question but I sent it to the wrong mailing list.
> Hopefully someone here can give me a solution.
> 
> I'm using Apache HTTPClient 4.5.3 to make some HTTP requests, I am getting a
> gzipped response back. I'm looking for a way to decpde/convert the gzipped
> content. I have tried many things I found online but non of them worked. I still
> get gibberish when I print the response. Below is the relevant code. I do
> understand that if I remove the request.addHeader("Accept-Encoding", "gzip"); I
> will get plain readable results back, but I need a way to convert/decode the
> gzipped content. What are my options for achieving this? please don't suggest
> that I can just remove request.addHeader("Accept-Encoding", "gzip"); I am
> looking for a way to decode the gzipped content
> 
> static public CloseableHttpClient CreateHttpClient() {
>     // return
>     //
> HttpClients.custom().disableAutomaticRetries().setHttpProcessor(HttpProcessor
> Builder.create().build())
>     // .build();
> 
>     return HttpClientBuilder.create().disableAutomaticRetries()
>             .setHttpProcessor(HttpProcessorBuilder.create().build()).build();
> }
> 
> static public RequestConfig GetConfig() {
>     return
> RequestConfig.custom().setSocketTimeout(READTIMEOUT).setConnectTimeout(
> CONNECTTIMEOUT)
>             .setConnectionRequestTimeout(REQUESTTIMEOUT).build();
> }
> 
> static public String updates() {
>     String result = "";
>     String url = "https://example.com";
>     CloseableHttpClient httpClient = CreateHttpClient();
>     CloseableHttpResponse response = null;
>     URL urlObj;
>     RequestConfig config = GetConfig();
> 
>     try {
>         urlObj = new URL(url);
> 
>         HttpPost request = new HttpPost(url);
>         request.setConfig(config);
> 
>         StringEntity params = new StringEntity("example");
> 
>         request.addHeader("Accept-Language", "en");
>         request.addHeader("Content-Type", "application/json; charset=UTF-8");
>         request.addHeader("Content-Length",
> String.valueOf(params.getContentLength()));
>         request.addHeader("Host", urlObj.getHost());
>         request.addHeader("Connection", "Keep-Alive");
>         request.addHeader("Accept-Encoding", "gzip");
> 
>         request.setEntity(params);
>         response = httpClient.execute(request);
> 
>         int responseCode = response.getStatusLine().getStatusCode();
> 
>         System.out.println("updates response code: " + responseCode);
> 
> 
>         // BufferedReader rd = new BufferedReader(new
>         // InputStreamReader(response.getEntity().getContent(), "UTF-8"));
> 
>         result = EntityUtils.toString(response.getEntity());
>         // String line = "";
>         // while ((line = rd.readLine()) != null) {
>         // result.append(line);
>         // }
> 
>     } catch (Exception e) {
>         e.printStackTrace();
>     } finally {
>         try {
>             if (response != null)
>                 response.close();
>             httpClient.close();
>         } catch (IOException e) {
>             e.printStackTrace();
>         }
>     }
> 
>     return result;
> }


---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org