Re: Disabling compression

[Kevin Risden <kr...@apache.org>]

I was following up on this today and saw that the Hue issue linked was
closed.

I was able to figure out that this was happening due to consumption of http
> post input stream before doPost in Knox. So, Knox itself was sending empty
> body.


Is there an action item here? Knox uses a replayBuffer to ensure that post
messages can typically be replayed. There is also the possibility of using
something like "Expect: 100-Continue" to avoid the replay buffer.


Kevin Risden


On Mon, Dec 30, 2019 at 6:09 PM Aj J <hi...@gmail.com> wrote:

> Thanks for looking into this Kevin.
>
> For API details and logs, please refer this-
> https://github.com/cloudera/hue/issues/1019#issuecomment-569820528
>
> I added couple of log statements in the hue notebook APIs and can see the
> payload for POST missing.
>
>
> On Mon, Dec 30, 2019 at 1:53 PM Kevin Risden <kr...@apache.org> wrote:
>
>> Hmmmm this sounds suspicious like something else is going on. Knox and
>> Hue have been used pretty heavily recently and I don't think there have
>> been issues w/ any specific types of requests.
>>
>> From the Hue logs I can see that request body for almost all POST
>>> requests are missing.
>>>
>>
>> For most POST requests, you typically won't see any of the data in the
>> server logs unless you are like debug level. The idea is that the body is
>> large and doesn't need to be logged by default.
>>
>> Is this for a specific API call? Is this for certain sizes of POST
>> requests?
>>
>> What is the issue that was hit to cause you to look into the logs
>> originally? Maybe there is something else happening before you got to this
>> line of questioning.
>>
>> It would be good to try to narrow this down a bit further. Hue should be
>> capable of handling gzipped requests.
>>
>> Kevin Risden
>>
>>
>> On Fri, Dec 27, 2019 at 7:54 PM Aj J <hi...@gmail.com> wrote:
>>
>>> I'm facing an issue similar to one mentioned here-
>>> https://www.mail-archive.com/user@knox.apache.org/msg00542.html
>>>
>>> I have a kerberized environment with Knox receiving all the requests,
>>> doing authN and forwarding it to Hue.
>>>
>>> From the Hue logs I can see that request body for almost all POST
>>> requests are missing.
>>> I see that in the discussion on the mailing thread above, the reporter
>>> resolved this issue by putting nginx to do decompress requests.
>>>
>>> I was wondering if knox has made any changes to resolve this.
>>>
>>> Any workarounds/ suggestions are welcome.
>>> I would like to avoid putting another nginx server all together.
>>>
>>> I'm planning on investigating further on the dispatch side to see what
>>> can be done.
>>>
>>> Thanks,
>>> Ajay
>>>
>>>

Re: Disabling compression

[Aj J <hi...@gmail.com>]

This was happening due to a custom change in knox from our side.

Thanks for the help Kevin.

On Tue, Jan 7, 2020 at 9:50 AM Kevin Risden <kr...@apache.org> wrote:

> I was following up on this today and saw that the Hue issue linked was
> closed.
>
> I was able to figure out that this was happening due to consumption of
>> http post input stream before doPost in Knox. So, Knox itself was sending
>> empty body.
>
>
> Is there an action item here? Knox uses a replayBuffer to ensure that post
> messages can typically be replayed. There is also the possibility of using
> something like "Expect: 100-Continue" to avoid the replay buffer.
>
>
> Kevin Risden
>
>
> On Mon, Dec 30, 2019 at 6:09 PM Aj J <hi...@gmail.com> wrote:
>
>> Thanks for looking into this Kevin.
>>
>> For API details and logs, please refer this-
>> https://github.com/cloudera/hue/issues/1019#issuecomment-569820528
>>
>> I added couple of log statements in the hue notebook APIs and can see the
>> payload for POST missing.
>>
>>
>> On Mon, Dec 30, 2019 at 1:53 PM Kevin Risden <kr...@apache.org> wrote:
>>
>>> Hmmmm this sounds suspicious like something else is going on. Knox and
>>> Hue have been used pretty heavily recently and I don't think there have
>>> been issues w/ any specific types of requests.
>>>
>>> From the Hue logs I can see that request body for almost all POST
>>>> requests are missing.
>>>>
>>>
>>> For most POST requests, you typically won't see any of the data in the
>>> server logs unless you are like debug level. The idea is that the body is
>>> large and doesn't need to be logged by default.
>>>
>>> Is this for a specific API call? Is this for certain sizes of POST
>>> requests?
>>>
>>> What is the issue that was hit to cause you to look into the logs
>>> originally? Maybe there is something else happening before you got to this
>>> line of questioning.
>>>
>>> It would be good to try to narrow this down a bit further. Hue should be
>>> capable of handling gzipped requests.
>>>
>>> Kevin Risden
>>>
>>>
>>> On Fri, Dec 27, 2019 at 7:54 PM Aj J <hi...@gmail.com> wrote:
>>>
>>>> I'm facing an issue similar to one mentioned here-
>>>> https://www.mail-archive.com/user@knox.apache.org/msg00542.html
>>>>
>>>> I have a kerberized environment with Knox receiving all the requests,
>>>> doing authN and forwarding it to Hue.
>>>>
>>>> From the Hue logs I can see that request body for almost all POST
>>>> requests are missing.
>>>> I see that in the discussion on the mailing thread above, the reporter
>>>> resolved this issue by putting nginx to do decompress requests.
>>>>
>>>> I was wondering if knox has made any changes to resolve this.
>>>>
>>>> Any workarounds/ suggestions are welcome.
>>>> I would like to avoid putting another nginx server all together.
>>>>
>>>> I'm planning on investigating further on the dispatch side to see what
>>>> can be done.
>>>>
>>>> Thanks,
>>>> Ajay
>>>>
>>>>