You are viewing a plain text version of this content. The canonical link for it is here.
Posted to c-dev@axis.apache.org by "Michael Dufel (JIRA)" <ax...@ws.apache.org> on 2006/02/21 21:36:27 UTC
[jira] Created: (AXISCPP-930) Chunked encoding is not implemented
correctly
Chunked encoding is not implemented correctly
---------------------------------------------
Key: AXISCPP-930
URL: http://issues.apache.org/jira/browse/AXISCPP-930
Project: Axis-C++
Type: Bug
Components: Transport (axis2), Transport (axis3)
Versions: 1.5 Final
Environment: Solaris 8
Reporter: Michael Dufel
<Rant>The chunked transfer encoding is broken and results in a 'peekNextChar' error similar to the one described in AXISCPP-555. I was forced to re-write HTTPTransport::getBytes() from almost the ground up because of the unreadableness of the code. Nested do/while loops??? Come on ... Yes I know, I am a code snob. </Rant>
I believe the problem in the original code was reflected in the case that the buffer from the channel contains data in the following form:
<some continued data>CRLF
<chunk length 1>CRLF
<some data 1><CRLF>
<chunk length 2>CRLF
<some data 2>CRLF
.
.
.
<chunk length n>CRLF
<some data n>
everything after <chunk length 2> was being dropped by the transport library. The transport library sends a 'TRANSPORT FINISHED' response and the xml parser is only given a partial message to deal with. This causes the 'peekNextChar' error.
Yes, I know that solaris 8 is not supported in Axis 1.5, but this is a protocol issue, not a platform related issue. Again, since I had to re-write the getBytes method, I can't offer a code snippet which will 'magically' fix this problem.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Closed: (AXISCPP-930) Chunked encoding is not implemented
correctly
Posted by "nadir amra (JIRA)" <ax...@ws.apache.org>.
[ https://issues.apache.org/jira/browse/AXISCPP-930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
nadir amra closed AXISCPP-930.
------------------------------
Resolution: Fixed
Fix Version/s: current (nightly)
I believe this has finally be resolved.
Problems:
==============================================
=== 1 === Chunked processing
The transport did not keep track of how many bytes of the chunk had been consumed, and thus if not all of the chunk was consumed the next time through it would assume that the remaining partial chunk was part of the chunk size, resulting in the transport to be in a weird state, to say the least.
=== 2 === Chunk processing
The transport did not remove CRLF's from the input buffers correctly or completely, again resulting in possible weird states.
=== 3 === Chunk processing
Chunk size processing returned 0 when passed an invalid chunk size. This caused me headaches to no end. Again, causing transport to not be in a valid state.
=== 4 === Non-chunked process, no Content-Length
Non-chunked processing where there is no content-length may occasionally result in AxisGenException with no message detailing what is wrong. This was a result on the reliance of an uninitialized m_iContentLength class variable. This was an intermittent error.
==============================================
To fix the problems above, I created a new state, eSOAPMessageHasContentLength, and seperated the logic for state processing so that getBytes() now calls:
getBytes_MessageIsChunked()
getBytes_MessageIsNotChunked()
getBytes_MessageHasContentLength()
I also create resetInputStateMachine() so that it will properly initialize the state variables from whereever it is called.
In addition, the transport did not recover from connections that had been closed by the Channel when reopen was false. I added code so that a Channel's reopenRequired() is invoked to determine if an open should be done. More information on this will be in AXISCPP-481.
I also ensure that when evaluating a chunk-size string, it is valid - i.e. all hexadecimal characters. Otherwise, an exception is thrown. Before, it would simple return some value. This caused me headaches to no end. Once I added this, it revealed that the mockserver did not handle CRLF correctly, so I had to fix that and now it does so properly as long as chunk sizes as set to ### (except the last, which must be 0).
On the performance side, there is a lot less substring'ing. I make heavy use of the string erase() function, and there is now no reason to seperate chunked messages when copying data to parser buffer. In addition, the channel use to read from socket in own buffer and then copy it to transport buffer, that is no longer the case. I have also added some minor infrastructure that in the future will let us read the data into the parser buffer directly. However, I still have some work to do on this.
I also added support for new SoapTransport.h method isThereResponseData() that will be used for one-way requests. More on this will be in AXISCPP-925.
Finally, and probably more important of all, I added lots of comments so that at least it will be easy next time to understand what is going on and the intricacies of the logic. Prior to this there was a lot of "hidden" assumptions, to put it kindly, that caused me much pain when moving code around. No comments is a bad thing, people! :-)
So have at it and let me know if there are any problems.
> Chunked encoding is not implemented correctly
> ---------------------------------------------
>
> Key: AXISCPP-930
> URL: https://issues.apache.org/jira/browse/AXISCPP-930
> Project: Axis-C++
> Issue Type: Bug
> Components: Transport (axis3)
> Affects Versions: 1.5 Final
> Environment: Solaris 8
> Reporter: Michael Dufel
> Assigned To: nadir amra
> Fix For: current (nightly)
>
>
> <Rant>The chunked transfer encoding is broken and results in a 'peekNextChar' error similar to the one described in AXISCPP-555. I was forced to re-write HTTPTransport::getBytes() from almost the ground up because of the unreadableness of the code. Nested do/while loops??? Come on ... Yes I know, I am a code snob. </Rant>
> I believe the problem in the original code was reflected in the case that the buffer from the channel contains data in the following form:
> <some continued data>CRLF
> <chunk length 1>CRLF
> <some data 1><CRLF>
> <chunk length 2>CRLF
> <some data 2>CRLF
> .
> .
> .
> <chunk length n>CRLF
> <some data n>
> everything after <chunk length 2> was being dropped by the transport library. The transport library sends a 'TRANSPORT FINISHED' response and the xml parser is only given a partial message to deal with. This causes the 'peekNextChar' error.
> Yes, I know that solaris 8 is not supported in Axis 1.5, but this is a protocol issue, not a platform related issue. Again, since I had to re-write the getBytes method, I can't offer a code snippet which will 'magically' fix this problem.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: axis-c-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: axis-c-dev-help@ws.apache.org
[jira] Updated: (AXISCPP-930) Chunked encoding is not implemented
correctly
Posted by "nadir amra (JIRA)" <ax...@ws.apache.org>.
[ http://issues.apache.org/jira/browse/AXISCPP-930?page=all ]
nadir amra updated AXISCPP-930:
-------------------------------
Component/s: (was: Transport (axis2))
> Chunked encoding is not implemented correctly
> ---------------------------------------------
>
> Key: AXISCPP-930
> URL: http://issues.apache.org/jira/browse/AXISCPP-930
> Project: Axis-C++
> Issue Type: Bug
> Components: Transport (axis3)
> Affects Versions: 1.5 Final
> Environment: Solaris 8
> Reporter: Michael Dufel
>
> <Rant>The chunked transfer encoding is broken and results in a 'peekNextChar' error similar to the one described in AXISCPP-555. I was forced to re-write HTTPTransport::getBytes() from almost the ground up because of the unreadableness of the code. Nested do/while loops??? Come on ... Yes I know, I am a code snob. </Rant>
> I believe the problem in the original code was reflected in the case that the buffer from the channel contains data in the following form:
> <some continued data>CRLF
> <chunk length 1>CRLF
> <some data 1><CRLF>
> <chunk length 2>CRLF
> <some data 2>CRLF
> .
> .
> .
> <chunk length n>CRLF
> <some data n>
> everything after <chunk length 2> was being dropped by the transport library. The transport library sends a 'TRANSPORT FINISHED' response and the xml parser is only given a partial message to deal with. This causes the 'peekNextChar' error.
> Yes, I know that solaris 8 is not supported in Axis 1.5, but this is a protocol issue, not a platform related issue. Again, since I had to re-write the getBytes method, I can't offer a code snippet which will 'magically' fix this problem.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: axis-c-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: axis-c-dev-help@ws.apache.org
[jira] Commented: (AXISCPP-930) Chunked encoding is not implemented
correctly
Posted by "Fred Preston (JIRA)" <ax...@ws.apache.org>.
[ http://issues.apache.org/jira/browse/AXISCPP-930?page=comments#action_12367489 ]
Fred Preston commented on AXISCPP-930:
--------------------------------------
Hi Michael,
Just looking at this JIRA. The getBytes method is basically a switch statement that has three distinct states. These are defined as follows:-
eWaitingForHTTPHeader :- Waiting for a 'valid' HTTP header. A header is valid when a 'HTTP' string is found that is subsequently followed by a CRLFCRLF sequence. The HTTP header is then read and if the message number is incorrect an exception is thrown. The header also contains information on whether the accompanying message is or is not chunked.
eSOAPMessageIsChunked :- When a message is chunked, it has been broken into a number of blocks. Each block is preceeded by a hex number. The number is the chunk length. A zero length chunk represents the end of the message. NB: because AXIS uses a 'pull' parser, the last chunk size should never be read as the parser will return once it has encountered '</envelope>'. If it does read the last '0', this could indicate that there is a problem in the message.
eSOAPMessageIsNotChunked :- The message is read until the number of message bytes read equals the HTTP header content length value.
State 'eWaitingForHTTPHeader' calls readHTTPHeader() which waits for a valid HTTP header. The buffer m_strReceived is emptied and then m_pActiveChannel is called to copy any Rx'ed data into a temporary buffer (m_pszRxBuffer). This buffer is then appended to the m_strReceived buffer. This process continues until a HTTP header is detected or timeout. If we assume that a HTTP header is deteted before the timeout, then whatever data has been captured will be held in m_strReceived. The method will then calls the method processHTTPHeader, which does just that. Depending on the header information it will change the state (m_GetBytesState) to be either chunked (eSOAPMessageIsChunked) or non-chunked (eSOAPMessageIsNotChunked). Or, if after further processing the HTTP header is not recognised as valid, it will change the state back to eWaitingForHTTPHeader and throw an exception. If all is still well, then the method calls checkHTTPStatusCode which checks the HTTP status code. After that the HTTP header is removed from the buffer (m_strReceived) leaving it pointing to the first character after the HTTP header (m_iBytesLeft contains the length of this remaining string). If the message is valid the method will drop out of the bottom of the 'eWaitingForHTTPHeader' state and into the 'eSOAPMessageIsChunked' state.
State 'eSOAPMessageIsChunked' first checks that the state is infact 'eSOAPMessageIsChunked' and if it is, it begins to process the data in the buffer (m_strReceived). If the buffer is empty (m_iBytesLeft = 0), the first method to be called is getNextDataPacket which will wait for more data to be Rx'ed (if data is recieved, it will be appended to the buffer (m_strReceived) and the length (m_iBytesLeft) will be updated) or timeout and throw an exception. The method will then call getChunkSize to read the hex digits that should be the first characters of the buffer and puts the decimal equivalent into m_iContentLength. The method will then enter a loop, that can only be exited when either enough data is recieved or there is a timeout (causing an exception to be thrown). If the length of the buffer is greater than the expected chunk length, then save the data past the chunk size in a temporary buffer (nextChunk). Then, set the buffer (m_strReceived) to the size of the chunk (m_iContentLength).
State 'eSOAPMessageIsNotChunked' uses m_iContentLength to countdown the remaining size of data that is expected. While m_iContentLength is greater than 0, it will call getNextDataPacket to get any Rx'ed data and append that to the buffer (m_strReceived).
At then end of the switch statement is a section of 'common' code that call copyDataToParserBuffer that copies m_iBytesLeft of the buffer (m_strReceived) into the XMLParser buffer (pcBuffer) and then removes the first m_iBytesLeft from the buffer (m_strReceived). Finally, then temporary buffer (nextChunk) is appended back onto the buffer (m_strReceived) and the new buffer size is found.
I think what you are looking at is old code, because what you are describing no longer happens. I cannot see a situation where what you are describing can happen, but I can only test this if you can send me your WSDL and response message.
Regards,
Fred Preston.
> Chunked encoding is not implemented correctly
> ---------------------------------------------
>
> Key: AXISCPP-930
> URL: http://issues.apache.org/jira/browse/AXISCPP-930
> Project: Axis-C++
> Type: Bug
> Components: Transport (axis3), Transport (axis2)
> Versions: 1.5 Final
> Environment: Solaris 8
> Reporter: Michael Dufel
>
> <Rant>The chunked transfer encoding is broken and results in a 'peekNextChar' error similar to the one described in AXISCPP-555. I was forced to re-write HTTPTransport::getBytes() from almost the ground up because of the unreadableness of the code. Nested do/while loops??? Come on ... Yes I know, I am a code snob. </Rant>
> I believe the problem in the original code was reflected in the case that the buffer from the channel contains data in the following form:
> <some continued data>CRLF
> <chunk length 1>CRLF
> <some data 1><CRLF>
> <chunk length 2>CRLF
> <some data 2>CRLF
> .
> .
> .
> <chunk length n>CRLF
> <some data n>
> everything after <chunk length 2> was being dropped by the transport library. The transport library sends a 'TRANSPORT FINISHED' response and the xml parser is only given a partial message to deal with. This causes the 'peekNextChar' error.
> Yes, I know that solaris 8 is not supported in Axis 1.5, but this is a protocol issue, not a platform related issue. Again, since I had to re-write the getBytes method, I can't offer a code snippet which will 'magically' fix this problem.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Assigned: (AXISCPP-930) Chunked encoding is not implemented
correctly
Posted by "nadir amra (JIRA)" <ax...@ws.apache.org>.
[ https://issues.apache.org/jira/browse/AXISCPP-930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
nadir amra reassigned AXISCPP-930:
----------------------------------
Assignee: nadir amra
> Chunked encoding is not implemented correctly
> ---------------------------------------------
>
> Key: AXISCPP-930
> URL: https://issues.apache.org/jira/browse/AXISCPP-930
> Project: Axis-C++
> Issue Type: Bug
> Components: Transport (axis3)
> Affects Versions: 1.5 Final
> Environment: Solaris 8
> Reporter: Michael Dufel
> Assigned To: nadir amra
>
> <Rant>The chunked transfer encoding is broken and results in a 'peekNextChar' error similar to the one described in AXISCPP-555. I was forced to re-write HTTPTransport::getBytes() from almost the ground up because of the unreadableness of the code. Nested do/while loops??? Come on ... Yes I know, I am a code snob. </Rant>
> I believe the problem in the original code was reflected in the case that the buffer from the channel contains data in the following form:
> <some continued data>CRLF
> <chunk length 1>CRLF
> <some data 1><CRLF>
> <chunk length 2>CRLF
> <some data 2>CRLF
> .
> .
> .
> <chunk length n>CRLF
> <some data n>
> everything after <chunk length 2> was being dropped by the transport library. The transport library sends a 'TRANSPORT FINISHED' response and the xml parser is only given a partial message to deal with. This causes the 'peekNextChar' error.
> Yes, I know that solaris 8 is not supported in Axis 1.5, but this is a protocol issue, not a platform related issue. Again, since I had to re-write the getBytes method, I can't offer a code snippet which will 'magically' fix this problem.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: axis-c-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: axis-c-dev-help@ws.apache.org
[jira] Commented: (AXISCPP-930) Chunked encoding is not implemented
correctly
Posted by "Michael Dufel (JIRA)" <ax...@ws.apache.org>.
[ http://issues.apache.org/jira/browse/AXISCPP-930?page=comments#action_12367371 ]
Michael Dufel commented on AXISCPP-930:
---------------------------------------
Chunking fails not after chunk 1, but after chunk 2.
I can't imagine that this would be something you could easily test
for. I did not discover the problem when the client was directly
communicating with the server; it was only after the client/server
communication started going through an intermediate proxy server.
I'll try to think of a way I can demonstrate the issue in a way that
can be tested for in a regression suite.
Mike
> Chunked encoding is not implemented correctly
> ---------------------------------------------
>
> Key: AXISCPP-930
> URL: http://issues.apache.org/jira/browse/AXISCPP-930
> Project: Axis-C++
> Type: Bug
> Components: Transport (axis2), Transport (axis3)
> Versions: 1.5 Final
> Environment: Solaris 8
> Reporter: Michael Dufel
>
> <Rant>The chunked transfer encoding is broken and results in a 'peekNextChar' error similar to the one described in AXISCPP-555. I was forced to re-write HTTPTransport::getBytes() from almost the ground up because of the unreadableness of the code. Nested do/while loops??? Come on ... Yes I know, I am a code snob. </Rant>
> I believe the problem in the original code was reflected in the case that the buffer from the channel contains data in the following form:
> <some continued data>CRLF
> <chunk length 1>CRLF
> <some data 1><CRLF>
> <chunk length 2>CRLF
> <some data 2>CRLF
> .
> .
> .
> <chunk length n>CRLF
> <some data n>
> everything after <chunk length 2> was being dropped by the transport library. The transport library sends a 'TRANSPORT FINISHED' response and the xml parser is only given a partial message to deal with. This causes the 'peekNextChar' error.
> Yes, I know that solaris 8 is not supported in Axis 1.5, but this is a protocol issue, not a platform related issue. Again, since I had to re-write the getBytes method, I can't offer a code snippet which will 'magically' fix this problem.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Commented: (AXISCPP-930) Chunked encoding is not implemented
correctly
Posted by "Fred Preston (JIRA)" <ax...@ws.apache.org>.
[ http://issues.apache.org/jira/browse/AXISCPP-930?page=comments#action_12373297 ]
Fred Preston commented on AXISCPP-930:
--------------------------------------
Hi,
Is this still a problem? If so, can I have specific examples with WSDLs and TCP monitor output so I can attempt to track this down.
Regards,
Fred Preston.
> Chunked encoding is not implemented correctly
> ---------------------------------------------
>
> Key: AXISCPP-930
> URL: http://issues.apache.org/jira/browse/AXISCPP-930
> Project: Axis-C++
> Type: Bug
> Components: Transport (axis2), Transport (axis3)
> Versions: 1.5 Final
> Environment: Solaris 8
> Reporter: Michael Dufel
>
> <Rant>The chunked transfer encoding is broken and results in a 'peekNextChar' error similar to the one described in AXISCPP-555. I was forced to re-write HTTPTransport::getBytes() from almost the ground up because of the unreadableness of the code. Nested do/while loops??? Come on ... Yes I know, I am a code snob. </Rant>
> I believe the problem in the original code was reflected in the case that the buffer from the channel contains data in the following form:
> <some continued data>CRLF
> <chunk length 1>CRLF
> <some data 1><CRLF>
> <chunk length 2>CRLF
> <some data 2>CRLF
> .
> .
> .
> <chunk length n>CRLF
> <some data n>
> everything after <chunk length 2> was being dropped by the transport library. The transport library sends a 'TRANSPORT FINISHED' response and the xml parser is only given a partial message to deal with. This causes the 'peekNextChar' error.
> Yes, I know that solaris 8 is not supported in Axis 1.5, but this is a protocol issue, not a platform related issue. Again, since I had to re-write the getBytes method, I can't offer a code snippet which will 'magically' fix this problem.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Commented: (AXISCPP-930) Chunked encoding is not implemented
correctly
Posted by "Michael Dufel (JIRA)" <ax...@ws.apache.org>.
[ http://issues.apache.org/jira/browse/AXISCPP-930?page=comments#action_12367376 ]
Michael Dufel commented on AXISCPP-930:
---------------------------------------
*******************Code Block: HTTPTransport::getBytes************
if (0 <= m_iBytesLeft)
{
int iIterationCountdown;
try
{
m_pszRxBuffer [0] = '\0';
*m_pActiveChannel >> m_pszRxBuffer;
m_strReceived = m_pszRxBuffer;
************************************************************************
You see here that any prior contents of m_strReceived are erased in favor of new data from the channel. A simple examination of the code that handles the case of continued chunk data does not loop to consume all the data in m_strReceived. Thus there is leftover data that gets dropped the next time the getBytes function is called. The only way this code works is if the content of m_strReceived is of the form:
<continued data>CRLF
<content length>CRLF
<content data <= content length>
You might do something like m_strReceived += m_pszRxBuffer, but that is not a complete fix - if it was that simple I would have done it instead of restructuring the code.
> Chunked encoding is not implemented correctly
> ---------------------------------------------
>
> Key: AXISCPP-930
> URL: http://issues.apache.org/jira/browse/AXISCPP-930
> Project: Axis-C++
> Type: Bug
> Components: Transport (axis2), Transport (axis3)
> Versions: 1.5 Final
> Environment: Solaris 8
> Reporter: Michael Dufel
>
> <Rant>The chunked transfer encoding is broken and results in a 'peekNextChar' error similar to the one described in AXISCPP-555. I was forced to re-write HTTPTransport::getBytes() from almost the ground up because of the unreadableness of the code. Nested do/while loops??? Come on ... Yes I know, I am a code snob. </Rant>
> I believe the problem in the original code was reflected in the case that the buffer from the channel contains data in the following form:
> <some continued data>CRLF
> <chunk length 1>CRLF
> <some data 1><CRLF>
> <chunk length 2>CRLF
> <some data 2>CRLF
> .
> .
> .
> <chunk length n>CRLF
> <some data n>
> everything after <chunk length 2> was being dropped by the transport library. The transport library sends a 'TRANSPORT FINISHED' response and the xml parser is only given a partial message to deal with. This causes the 'peekNextChar' error.
> Yes, I know that solaris 8 is not supported in Axis 1.5, but this is a protocol issue, not a platform related issue. Again, since I had to re-write the getBytes method, I can't offer a code snippet which will 'magically' fix this problem.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
Re: [jira] Commented: (AXISCPP-930) Chunked encoding is not implemented
correctly
Posted by Fred Preston <PR...@uk.ibm.com>.
Hi Mike,
I think I understand your problem, but you need to upgrade to the
latest transport code (1.5 is now so out-of-date that the errors you are
seeing may well have been fixed) as what you are describing can no longer
happen.
Regards,
Fred Preston.
"Michael Dufel (JIRA)" <ax...@ws.apache.org>
23/02/2006 17:07
Please respond to "Apache AXIS C Developers List"
To: axis-c-dev@ws.apache.org
cc:
Subject: [jira] Commented: (AXISCPP-930) Chunked encoding
is not implemented correctly
[
http://issues.apache.org/jira/browse/AXISCPP-930?page=comments#action_12367536
]
Michael Dufel commented on AXISCPP-930:
---------------------------------------
Fred,
Thanks for the 'in english' description of the getBytes method. It's too
bad I didn't have it handy when I first started debugging the function.
There is a branch of particular importance:
if( m_strReceived.length () > 0) {
/*Expects <chunk-length>CRLF<chunk-data>.... */
//Call this Branch A
}else if( m_bChunked){
/*Expects <partial data>CRLF<chunk-length>....*/
//call this Branch B
}
Consider that the contents of m_strReceived are:
5678
0008
12345678
0009
123456789
000A
1234
And now consider that m_iContentLength = 4 signifying that there are 4
more bytes to read before reading the next chunk.
1. strTemp is set to 5678
2. m_strRecieved is substring'd to be:
12345678
0009
123456789
000A
1234
3. strTemp and m_strReceived are spliced together yielding:
567812345678
0009
123456789
000A
1234
4. branch B ends.
5. The following string gets sent to the xml parser:
567812345678
0009
123456789
000A
1234
This is obviously incorrect because now the XML parser is reading in parts
of the http protocol. Furthermore, the next time that getBytes is called
you will have a big problem because the m_iContentLength variable will be
set to 0 and the axtoi method will not be passed something that is NOT the
chunk-length field.
The WSDL won't help you and neither would a response message if I sent one
to you. This issue was first discovered while attempting to contact a web
service running on a Weblogic application server. It also appeared when I
instrumented the code to print out the contents of the XML that was
passing over the wire. I theorize that the time taken to print out the XML
contents allowed the channel to buffer more data from the server resulting
in the case I have just described. This is funamentally a timing issue
more than anything else.
The joys of protocol programming.... :)
> Chunked encoding is not implemented correctly
> ---------------------------------------------
>
> Key: AXISCPP-930
> URL: http://issues.apache.org/jira/browse/AXISCPP-930
> Project: Axis-C++
> Type: Bug
> Components: Transport (axis2), Transport (axis3)
> Versions: 1.5 Final
> Environment: Solaris 8
> Reporter: Michael Dufel
>
> <Rant>The chunked transfer encoding is broken and results in a
'peekNextChar' error similar to the one described in AXISCPP-555. I was
forced to re-write HTTPTransport::getBytes() from almost the ground up
because of the unreadableness of the code. Nested do/while loops??? Come
on ... Yes I know, I am a code snob. </Rant>
> I believe the problem in the original code was reflected in the case
that the buffer from the channel contains data in the following form:
> <some continued data>CRLF
> <chunk length 1>CRLF
> <some data 1><CRLF>
> <chunk length 2>CRLF
> <some data 2>CRLF
> .
> .
> .
> <chunk length n>CRLF
> <some data n>
> everything after <chunk length 2> was being dropped by the transport
library. The transport library sends a 'TRANSPORT FINISHED' response and
the xml parser is only given a partial message to deal with. This causes
the 'peekNextChar' error.
> Yes, I know that solaris 8 is not supported in Axis 1.5, but this is a
protocol issue, not a platform related issue. Again, since I had to
re-write the getBytes method, I can't offer a code snippet which will
'magically' fix this problem.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Commented: (AXISCPP-930) Chunked encoding is not implemented
correctly
Posted by "Michael Dufel (JIRA)" <ax...@ws.apache.org>.
[ http://issues.apache.org/jira/browse/AXISCPP-930?page=comments#action_12367536 ]
Michael Dufel commented on AXISCPP-930:
---------------------------------------
Fred,
Thanks for the 'in english' description of the getBytes method. It's too bad I didn't have it handy when I first started debugging the function.
There is a branch of particular importance:
if( m_strReceived.length () > 0) {
/*Expects <chunk-length>CRLF<chunk-data>.... */
//Call this Branch A
}else if( m_bChunked){
/*Expects <partial data>CRLF<chunk-length>....*/
//call this Branch B
}
Consider that the contents of m_strReceived are:
5678
0008
12345678
0009
123456789
000A
1234
And now consider that m_iContentLength = 4 signifying that there are 4 more bytes to read before reading the next chunk.
1. strTemp is set to 5678
2. m_strRecieved is substring'd to be:
12345678
0009
123456789
000A
1234
3. strTemp and m_strReceived are spliced together yielding:
567812345678
0009
123456789
000A
1234
4. branch B ends.
5. The following string gets sent to the xml parser:
567812345678
0009
123456789
000A
1234
This is obviously incorrect because now the XML parser is reading in parts of the http protocol. Furthermore, the next time that getBytes is called you will have a big problem because the m_iContentLength variable will be set to 0 and the axtoi method will not be passed something that is NOT the chunk-length field.
The WSDL won't help you and neither would a response message if I sent one to you. This issue was first discovered while attempting to contact a web service running on a Weblogic application server. It also appeared when I instrumented the code to print out the contents of the XML that was passing over the wire. I theorize that the time taken to print out the XML contents allowed the channel to buffer more data from the server resulting in the case I have just described. This is funamentally a timing issue more than anything else.
The joys of protocol programming.... :)
> Chunked encoding is not implemented correctly
> ---------------------------------------------
>
> Key: AXISCPP-930
> URL: http://issues.apache.org/jira/browse/AXISCPP-930
> Project: Axis-C++
> Type: Bug
> Components: Transport (axis2), Transport (axis3)
> Versions: 1.5 Final
> Environment: Solaris 8
> Reporter: Michael Dufel
>
> <Rant>The chunked transfer encoding is broken and results in a 'peekNextChar' error similar to the one described in AXISCPP-555. I was forced to re-write HTTPTransport::getBytes() from almost the ground up because of the unreadableness of the code. Nested do/while loops??? Come on ... Yes I know, I am a code snob. </Rant>
> I believe the problem in the original code was reflected in the case that the buffer from the channel contains data in the following form:
> <some continued data>CRLF
> <chunk length 1>CRLF
> <some data 1><CRLF>
> <chunk length 2>CRLF
> <some data 2>CRLF
> .
> .
> .
> <chunk length n>CRLF
> <some data n>
> everything after <chunk length 2> was being dropped by the transport library. The transport library sends a 'TRANSPORT FINISHED' response and the xml parser is only given a partial message to deal with. This causes the 'peekNextChar' error.
> Yes, I know that solaris 8 is not supported in Axis 1.5, but this is a protocol issue, not a platform related issue. Again, since I had to re-write the getBytes method, I can't offer a code snippet which will 'magically' fix this problem.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Commented: (AXISCPP-930) Chunked encoding is not implemented
correctly
Posted by "John Hawkins (JIRA)" <ax...@ws.apache.org>.
[ http://issues.apache.org/jira/browse/AXISCPP-930?page=comments#action_12367317 ]
John Hawkins commented on AXISCPP-930:
--------------------------------------
I'm confused by your statement here that chunking fails after chnk one. We have many many tests that have large numbers of chunks they all work now and did in 1.5?
Perhaps if you could send us the response msg we can see more clearly what the issue is.
> Chunked encoding is not implemented correctly
> ---------------------------------------------
>
> Key: AXISCPP-930
> URL: http://issues.apache.org/jira/browse/AXISCPP-930
> Project: Axis-C++
> Type: Bug
> Components: Transport (axis2), Transport (axis3)
> Versions: 1.5 Final
> Environment: Solaris 8
> Reporter: Michael Dufel
>
> <Rant>The chunked transfer encoding is broken and results in a 'peekNextChar' error similar to the one described in AXISCPP-555. I was forced to re-write HTTPTransport::getBytes() from almost the ground up because of the unreadableness of the code. Nested do/while loops??? Come on ... Yes I know, I am a code snob. </Rant>
> I believe the problem in the original code was reflected in the case that the buffer from the channel contains data in the following form:
> <some continued data>CRLF
> <chunk length 1>CRLF
> <some data 1><CRLF>
> <chunk length 2>CRLF
> <some data 2>CRLF
> .
> .
> .
> <chunk length n>CRLF
> <some data n>
> everything after <chunk length 2> was being dropped by the transport library. The transport library sends a 'TRANSPORT FINISHED' response and the xml parser is only given a partial message to deal with. This causes the 'peekNextChar' error.
> Yes, I know that solaris 8 is not supported in Axis 1.5, but this is a protocol issue, not a platform related issue. Again, since I had to re-write the getBytes method, I can't offer a code snippet which will 'magically' fix this problem.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira