[users@httpd] CGI based WebDAV client and Transfer-Encoding: Chunked issue

[Paul <pv...@gmail.com>]

Hello,

I am working on patching a program that was written by a different
developer who is no longer with company.  The program is a CGI program
that acts as a WebDAV server.  Recently we found a problem when a mac
user running Mac OS 10.5 (Leopard) tries to paste a file into the
WebDAV folder on their computer.  Instead of putting the file on the
server, it was creating a 0 kb file.  After searching I managed to
find out this was caused by Apple's switch to using
"Transfer-Encoding: Chunked" rather than defining a Content-Length.

The problem I am having is that I do not know how to find out the
length of the input file that is coming from the client.  According to
the Transfer-Encoding specification, the data is supposed to be in the
form of:
"23
This is the data in the first chunk
1A
and this is the second one
0"

but when I do something like:
" cin.read((char *) myBuffer, 30);"
(note, 30 is just an example here), the result in the buffer starts
with the text being sent, and does not include any chunk sizes.  Am I
reading in the data wrong? Is the Mac sending the wrong information?
Is Apache filtering out the chunk data?

Thanks for any help you can give me,
Paul

---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
   "   from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org

Re: CGI based WebDAV client and Transfer-Encoding: Chunked issue

[Paul <pv...@gmail.com>]

Thanks for your responses,

I am now having troubles with getting the entire file that is being
sent to the server and knowing when the entire file has arrived.  What
I have noticed is that for larger files (ex greater than 90,000
characters) when I use "cin.seek(0, ios::end);" to find the end of the
file, I only get a size of around 66,000 characters.  If I put the
cin.seek in a loop, I will get the entire file, but I don't know when
to stop waiting for more parts to the file since there can be a delay
between the different chunks arriving at the server.

@Andre I have not looked into mod_dav because I would imagine that
would require pretty much a complete re-write of the program. As for
sending a header asking the client not to use chunked
transfer-encoding, that is an interesting idea that I will look into,
but if I can modify the program to work with chunked data that would
be preferable.

Again, thanks for the help,
- Paul

On Mon, Dec 8, 2008 at 4:11 PM, Paul <pv...@gmail.com> wrote:
> Hello,
>
> I am working on patching a program that was written by a different
> developer who is no longer with company.  The program is a CGI program
> that acts as a WebDAV server.  Recently we found a problem when a mac
> user running Mac OS 10.5 (Leopard) tries to paste a file into the
> WebDAV folder on their computer.  Instead of putting the file on the
> server, it was creating a 0 kb file.  After searching I managed to
> find out this was caused by Apple's switch to using
> "Transfer-Encoding: Chunked" rather than defining a Content-Length.
>
> The problem I am having is that I do not know how to find out the
> length of the input file that is coming from the client.  According to
> the Transfer-Encoding specification, the data is supposed to be in the
> form of:
> "23
> This is the data in the first chunk
> 1A
> and this is the second one
> 0"
>
> but when I do something like:
> " cin.read((char *) myBuffer, 30);"
> (note, 30 is just an example here), the result in the buffer starts
> with the text being sent, and does not include any chunk sizes.  Am I
> reading in the data wrong? Is the Mac sending the wrong information?
> Is Apache filtering out the chunk data?
>
> Thanks for any help you can give me,
> Paul
>

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org

Re: [users@httpd] CGI based WebDAV client and Transfer-Encoding: Chunked issue

["William A. Rowe, Jr." <wr...@rowe-clan.net>]

Paul wrote:
> Hello,
> 
> I am working on patching a program that was written by a different
> developer who is no longer with company.  The program is a CGI program
> that acts as a WebDAV server.  Recently we found a problem when a mac
> user running Mac OS 10.5 (Leopard) tries to paste a file into the
> WebDAV folder on their computer.  Instead of putting the file on the
> server, it was creating a 0 kb file.  After searching I managed to
> find out this was caused by Apple's switch to using
> "Transfer-Encoding: Chunked" rather than defining a Content-Length.
> 
> The problem I am having is that I do not know how to find out the
> length of the input file that is coming from the client.  According to
> the Transfer-Encoding specification, the data is supposed to be in the
> form of:
> "23
> This is the data in the first chunk
> 1A
> and this is the second one
> 0"
> 
> but when I do something like:
> " cin.read((char *) myBuffer, 30);"
> (note, 30 is just an example here), the result in the buffer starts
> with the text being sent, and does not include any chunk sizes.  Am I
> reading in the data wrong? Is the Mac sending the wrong information?
> Is Apache filtering out the chunk data?

Yes.  Your CGI should be blistfully unaware of the change to chunked
encoding, simply read to the EOF (end of file) status of cin.

The same is true of your CGI responses; httpd will chunk them when the
client is capable, and again your program has nothing to do with this;
it's part of the HTTP/1.1 protocol.

Bill

---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
   "   from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org

Re: [users@httpd] CGI based WebDAV client and Transfer-Encoding: Chunked issue

[André Warnier <aw...@ice-sa.com>]

Paul wrote:
[...]
Hi.
I do not know the answer to your original question, but here are a 
couple of alternative suggestions :

1) It may be possible, in the initial HTTP headers sent by the script to 
the client, to let the client know that the "chunked" encoding is not 
acceptable.  The client may then decide on another way to send the data, 
which might be easier to handle by the script.
Details are left as a further exercise..
(Accept-Encoding header etc..)

2) as an alternative, have you considered using the real mod_dav module, 
but with a twist : implement a filter before, and a filter after, that 
will get the file as soon as it has been uploaded to the server, and 
move it to some location of your choice.
This in a way similar to what mod_cache does.
The reason for using mod_dav ("Dav on" etc..), is that it probably 
already handles the dialog with the client very well, is maintained, 
tested, etc..
Again, details left for a further exercise.

But, in the case of 2, I can add that I have done something similar to 
that, using perl and mod_perl, and it works quite well.


---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
   "   from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org