You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by "Christian V." <ch...@zeropiu.it> on 2006/11/07 11:24:05 UTC
De-Chunking
Hi ,
i 'm running a third-party web service authentication module that hangs
when the request coming from the client is splitted out in different
chunks. I don't have access to the module and to the client neither, so
I'm thinking to write an input filter that collects all the chunks and
pass'em to the downstream filter or handler .
Is that possible?
Best regards,
Chris.
Re: De-Chunking
Posted by "Christian V." <ch...@zeropiu.it>.
Christian V. wrote:
> Nick Kew wrote:
>> On Tue, 07 Nov 2006 11:24:05 +0100 "Christian V."
>> <ch...@zeropiu.it> wrote:
>>
>>> Hi ,
>>>
>>> i 'm running a third-party web service authentication module that
>>> hangs when the request coming from the client is splitted out in
>>> different chunks. I don't have access to the module and to the
>>> client neither, so I'm thinking to write an input filter that
>>> collects all the chunks and pass'em to the downstream filter or
>>> handler . Is that possible?
>>
>> It's possible, yes.
>>
>> Whether it'll fix the problem for you is not certain. I'd suggest
>> starting with a quick hack (or a dechunking proxy in front of your
>> server) to test it first, if you really can't get the source.
>>
>
>
> Maybe the Proxy will fix it but it will not be the solution, so i
> think i'm gonna write the module-filter. But i need to know how
> Apache handle multi chunk request, as im not able to find this
> information. Is request coming entirely to my filter in the form of
> bucket&brigades then passed to down-streams module or brigades are
> passed down as soon as they come? (I hope i explained it well)
>
> Tnx a lot, Chris.
>
>
>
Let me explain well as im not 100% sure the problem is the 3rd party
module, and other people may had met the same issue:
[ CLIENT ] --> [ APACHE R.PROXY (SSL) + 3rd MODULE ] --> [WEB SERVICE]
The Web Service receives requests from both Java and .net clients.
Our problem is the following.
The .Net clients (we have one in c# and one in VB, both programmed with
visual studio 2005 ) will split the client's XML request in multiple
1024 byte packets. This happens only over HTTPS, and causes problems
with the 3rd party module
To debug this we have programmed an apache module on the reverse proxy
that dumps the stream of data as it receives it from the clients, and
our .Net client, over HTTPS, splits it up in multiple chunks, as seen here:
[Tue Oct 31 15:09:56 2006] [notice] (IN) bucketdumper: mode READBYTES;
blocking; 8192 bytes
[Tue Oct 31 15:09:57 2006] [notice] (IN) - (AFTER bucket_read)
-\tbucketdumper:\tbytes: 1024 - lenght read: 1024 - data: <?xml
version="1.0" encoding="utf-8"?><soap:Envelope
xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/"
xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/"
xmlns:tns="http://www.acme.com.com/wsdl/HelloMoto.wsdl"
xmlns:types="http://www.acme.com.com/wsdl/HelloMoto.wsdl/encodedTypes"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:xsd="http://www.w3.org/2001/XMLSchema"><soap:Body
soap:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"><q1:sayHello
xmlns:q1="urn:examples:HelloMoto"><TAG1
xsi:type="xsd:string">TES</TAG1><TAG2
xsi:type="xsd:string">TES</TAG2><TAG3
xsi:type="xsd:string">TES</TAG3><TAG4
xsi:type="xsd:string">TES</TAG4><TAG5
xsi:type="xsd:string">TES</TAG5><TAG6
xsi:type="xsd:string">TES</TAG6><TAG7
xsi:type="xsd:string">TES</TAG7><TAG8
xsi:type="xsd:string">TES</TAG8><TAG9
xsi:type="xsd:string">TES</TAG9><TAG10
xsi:type="xsd:string">TES</TAG10><TAG11
xsi:type="xsd:string">TES</TAG11><TAG12
xsi:type="xsd:string">TEST</TAG12></q1:sayHello></soap:Body></soap:Envelope
-
[Tue Oct 31 15:09:57 2006] [notice] (IN) Complete Bucket :
[Tue Oct 31 15:09:57 2006] [notice] (IN) bucketdumper: mode READBYTES;
blocking; 8192 bytes
[Tue Oct 31 15:09:58 2006] [notice] (IN) - (AFTER bucket_read)
-\tbucketdumper:\tbytes: 1 - lenght read: 1 - data: > -
[Tue Oct 31 15:09:58 2006] [notice] (IN) - (AFTER bucket_read)
-\tbucketdumper:\tbytes: 0 - lenght read: 0 - data: > -
Note how the XML is 1025 bytes long, and gets send in one 1024 byte
packet first, followed by a second 1 byte packet (that contains just the
final ">").
This does not happen over HTTP, where the entire XML arrives in one 1025
byte long data chunk.
Also, our Java clients do not split up the XML when posting in HTTPS,
independently of how long it is. Here is a request made by one of our
java clients:
[Tue Oct 31 15:12:57 2006] [notice] (IN) bucketdumper: mode READBYTES;
blocking; 8192 bytes
[Tue Oct 31 15:12:57 2006] [notice] (IN) - (AFTER bucket_read)
-\tbucketdumper:\tbytes: 4333 - lenght read: 4333 - data: <?xml
version="1.0" encoding="UTF-8"?><soapenv:Envelope
xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"
xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><soapenv:Body><beregnLivorno
xmlns="http://prognosiRiservata.acme.com"><startinput><ns1:channel
xmlns:ns1="http://">LIV</ns1:channel><ns2:clientId
xmlns:ns2="http://">TEST</ns2:clientId><ns3:clientPassword
xsi:nil="true" xmlns:ns3="http://"/><ns4:test
xmlns:ns4="http://">false</ns4:test><ns5:useCache
----------------------------------------
----------------------------------------
mlns:ns36="http://data.prognosiRiservata.acme.com">true</ns36:returnerBeskrivelser><ns37:returnerThreadSide
xmlns:ns37="http://Livorno.data.prognosiRiservata.acme.com">true</ns37:returnerThreadSide><ns38:returnerResultaterAvtale
xmlns:ns38="http://Livorno.data.prognosiRiservata.acme.com">true</ns38:returnerResultaterAvtale></prognosiRiservatainput></beregnLivorno></soapenv:Body></soapenv:Envelope>
-
[Tue Oct 31 15:12:57 2006] [notice] (IN) Complete Bucket :
[Tue Oct 31 15:12:57 2006] [notice] (IN) bucketdumper: mode READBYTES;
blocking; 8192 bytes
[Tue Oct 31 15:12:57 2006] [notice] (IN) - (AFTER bucket_read)
-\tbucketdumper:\tbytes: 0 - lenght read: 0 - data: -
Notice how the XML (4333 bytes long) is sent in one entire chunk.
---
I realized that my initial idea of having an input filter is not
applicable as long as the 3rd party module lives in the
authentication/authorization phase, accessing data before any input filter.
Ummm...
The DeChunking-Proxy hypothesis seems to be, at present, the only valid
solution! :-(
Chris.
Re: De-Chunking
Posted by "Christian V." <ch...@zeropiu.it>.
Nick Kew wrote:
> On Tue, 07 Nov 2006 11:24:05 +0100
> "Christian V." <ch...@zeropiu.it> wrote:
>
>> Hi ,
>>
>> i 'm running a third-party web service authentication module that
>> hangs when the request coming from the client is splitted out in
>> different chunks. I don't have access to the module and to the client
>> neither, so I'm thinking to write an input filter that collects all
>> the chunks and pass'em to the downstream filter or handler .
>> Is that possible?
>
> It's possible, yes.
>
> Whether it'll fix the problem for you is not certain. I'd
> suggest starting with a quick hack (or a dechunking proxy
> in front of your server) to test it first, if you really
> can't get the source.
>
Maybe the Proxy will fix it but it will not be the solution, so i think
i'm gonna write the module-filter.
But i need to know how Apache handle multi chunk request, as im not able
to find this information.
Is request coming entirely to my filter in the form of bucket&brigades
then passed to down-streams module or brigades are passed down as soon
as they come? (I hope i explained it well)
Tnx a lot,
Chris.
Re: De-Chunking
Posted by Nick Kew <ni...@webthing.com>.
On Tue, 07 Nov 2006 11:24:05 +0100
"Christian V." <ch...@zeropiu.it> wrote:
> Hi ,
>
> i 'm running a third-party web service authentication module that
> hangs when the request coming from the client is splitted out in
> different chunks. I don't have access to the module and to the client
> neither, so I'm thinking to write an input filter that collects all
> the chunks and pass'em to the downstream filter or handler .
> Is that possible?
It's possible, yes.
Whether it'll fix the problem for you is not certain. I'd
suggest starting with a quick hack (or a dechunking proxy
in front of your server) to test it first, if you really
can't get the source.
--
Nick Kew
Application Development with Apache - the Apache Modules Book
http://www.apachetutor.org/