You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by S Ahmed <sa...@gmail.com> on 2012/01/04 16:39:50 UTC
clarification on how data is streamed and handled by tomcat
Say I have a simple servlet or spring mvc application running on tomcat.
Tomcat is running as a webserver on port 80.
A client makes a http POST request to my page www.example.com/submit
If the client adds a very large file with the POST, does this mean that
when my servlet code starts to execute, tomcat already has the entire
request content in memory? (including the very large POST parameter)
Re: clarification on how data is streamed and handled by tomcat
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Ahmed,
On 1/4/12 1:47 PM, S Ahmed wrote:
> The point is that I want to know the effects of others sending me
> large values in a http post (not an image upload, but a form
> post).
>
> I'm assuming once it is sent by the client as a http post, and my
> servlet responds to the request tomcat has already streamed that
> data and whether I do:
>
> String p1 = request.getParameter("big_payload")
>
> or not, it has already been loaded into memory.
If you load "big_payload" into "String p1" then yes, it will be loaded
into memory, no matter what API you use.
If, instead, you use stream-oriented APIs to fetch your data, then you
won't run the risk of loading too much data at once.
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAk8E360ACgkQ9CaO5/Lv0PBmRgCgiHSZhXVzViaTQpj2EeyUWPri
a0EAoITzXP6Hb0du416K7Nh4dUbFdM0d
=Iz1x
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: clarification on how data is streamed and handled by tomcat
Posted by Daniel Mikusa <dm...@vmware.com>.
On Wed, 2012-01-04 at 10:47 -0800, S Ahmed wrote:
> Tomcat 6
> Spring MVC, where my controllers method has both httpservletrequest and
> httpservletresponse as parameters.
>
> The point is that I want to know the effects of others sending me large
> values in a http post (not an image upload, but a form post).
>
> I'm assuming once it is sent by the client as a http post, and my servlet
> responds to the request tomcat has already streamed that data and whether I
> do:
>
> String p1 = request.getParameter("big_payload")
>
> or not, it has already been loaded into memory.
>
> Am I correct?
>
> Is there a maximize size setting in tomcat?
Perhaps "maxPostSize"?
https://tomcat.apache.org/tomcat-6.0-doc/config/http.html
Dan
>
> On Wed, Jan 4, 2012 at 1:03 PM, Konstantin Kolinko
> <kn...@gmail.com>wrote:
>
> > 2012/1/4 S Ahmed <sa...@gmail.com>:
> > > Say I have a simple servlet or spring mvc application running on tomcat.
> > >
> > > Tomcat is running as a webserver on port 80.
> > >
> > > A client makes a http POST request to my page www.example.com/submit
> > >
> > > If the client adds a very large file with the POST, does this mean that
> > > when my servlet code starts to execute, tomcat already has the entire
> > > request content in memory? (including the very large POST parameter)
> >
> > 1. Tomcat version =?
> > 2. What API are you using to read the file?
> >
> > In general request processing starts after the headers in HTTP request
> > have been read. At that point you can call getInputStream() and read
> > the body of the POST request while it is being received from the
> > client.
> >
> > But if you call certain APIs (e.g. getParameter()) the call will hang
> > until entire POST request body is received from the client and
> > processed. It is said that those methods "consume" the body of a POST
> > request.
> >
> > 3. In most implementations large file uploads are not stored in
> > memory, but are written to a temporary file on your hard drive. (So
> > the request body is processed, but you cannot say that it is "in
> > memory").
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> > For additional commands, e-mail: users-help@tomcat.apache.org
> >
> >
Re: clarification on how data is streamed and handled by tomcat
Posted by André Warnier <aw...@ice-sa.com>.
S Ahmed wrote:
> " But if you call certain APIs (e.g. getParameter()) the call will hang
> until entire POST request body is received from the client and
> processed. "
>
> So this means if you don't reference request.getParameter then it won't be
> streamed from the client?
>
> The reason I am asking is I will perform api usage limites, and it would be
> great if I don't have to bring in the entire request header + body if I can
> get away with it.
Re-read Konstantin's response carefully.
The client sends the request, and it will not stop sending until it has sent the whole
request, unless either of these conditions become true :
- the whole series of buffers between the client and Tomcat get full, and TCP tells the
client to wait. That can be a lot of data in buffers at the client side, in intermediate
network nodes, in the TCP stack of the Tomcat host etc...
- or Tomcat or the application in Tomcat closes the connection, at which point the client
would get an error if it tries to send more data over that connection.
(And the client may then retry the same request again, see below).
At the Tomcat side :
- Tomcat will anyway read the headers of the request, because it needs to do that for
example to know how to handle the request (to which host it should go, to which
application etc..).
- THEN Tomcat hands over the request to your application.
If your application at that point does nothing more, then at some point the buffers will
get full, and the client will have to wait if it still has something to send (see above).
If your application asks only for requests /headers/ from Tomat, then since they are
already in memory, the body of the request is not being read.
If your application starts to read the body of the request itself, then it is "consuming"
the body, the buffers will free up, and the client can keep sending again.
If your application asks Tomcat to parse the request body (for example, it would do that
if it asks Tomcat for one or more request parameters), then Tomcat will read (aka
"consume") the request body (it needs to do that, to parse the parameters from it); in the
meantime, your application will wait ("block") until that is done; because Tomcat is
consuming the request body, the network buffers will be emptied, and the client will be
able to send more data (if it has more data to send).
And then when Tomcat has read the request body entirely, it will parse the body into
parameters, and the getParameter(s) call will return, and your application will run again.
By that time of course the client will have sent everything.
So, if your application can, on the base /only/ of the request headers, determine that it
should reject the client request, then it could return an error code and close the
connection before the body is read; and then you could, maybe, avoid that the client sends
the whole body of data.
Maybe. Because by that time, it is possible that the whole data has been already sent
anyway, and is already held in network buffers somewhere in-between.
What I mean by all this, is that it not so that the client sends the request headers, and
then waits for some signal from your application or from Tomcat before it starts sending
the body. The client will keep sending data until it has none left to send, or until the
connection is closed in front of its nose, or until the buffers fill up.
That's the standard way in which it works.
There are ways to change that, but not with a standard browser as a client. You need to
control the application on the client side also, to do that kind of thing.
Have a look here : http://www.w3.org/Protocols/rfc2616/rfc2616-sec8.html
Section 8.2.3
(and the whole of the 8.2 section)
>
> On Wed, Jan 4, 2012 at 1:58 PM, André Warnier <aw...@ice-sa.com> wrote:
>
>> S Ahmed wrote:
>>
>>> Tomcat 6
>>> Spring MVC, where my controllers method has both httpservletrequest and
>>> httpservletresponse as parameters.
>>>
>>> The point is that I want to know the effects of others sending me large
>>> values in a http post (not an image upload, but a form post).
>>>
>>> I'm assuming once it is sent by the client as a http post, and my servlet
>>> responds to the request tomcat has already streamed that data and whether
>>> I
>>> do:
>>>
>>> String p1 = request.getParameter("big_**payload")
>>>
>>> or not, it has already been loaded into memory.
>>>
>>> Am I correct?
>>>
>> To me, it seems that Konstantin already answered those questions precisely
>> and in detail below.
>>
>>
>>
>>> Is there a maximize size setting in tomcat?
>>>
>>> On Wed, Jan 4, 2012 at 1:03 PM, Konstantin Kolinko
>>> <kn...@gmail.com>wrote:
>>>
>>> 2012/1/4 S Ahmed <sa...@gmail.com>:
>>>>> Say I have a simple servlet or spring mvc application running on tomcat.
>>>>>
>>>>> Tomcat is running as a webserver on port 80.
>>>>>
>>>>> A client makes a http POST request to my page www.example.com/submit
>>>>>
>>>>> If the client adds a very large file with the POST, does this mean that
>>>>> when my servlet code starts to execute, tomcat already has the entire
>>>>> request content in memory? (including the very large POST parameter)
>>>>>
>>>> 1. Tomcat version =?
>>>> 2. What API are you using to read the file?
>>>>
>>>> In general request processing starts after the headers in HTTP request
>>>> have been read. At that point you can call getInputStream() and read
>>>> the body of the POST request while it is being received from the
>>>> client.
>>>>
>>>> But if you call certain APIs (e.g. getParameter()) the call will hang
>>>> until entire POST request body is received from the client and
>>>> processed. It is said that those methods "consume" the body of a POST
>>>> request.
>>>>
>>>> 3. In most implementations large file uploads are not stored in
>>>> memory, but are written to a temporary file on your hard drive. (So
>>>> the request body is processed, but you cannot say that it is "in
>>>> memory").
>>>>
>>>> ------------------------------**------------------------------**
>>>> ---------
>>>> To unsubscribe, e-mail: users-unsubscribe@tomcat.**apache.org<us...@tomcat.apache.org>
>>>> For additional commands, e-mail: users-help@tomcat.apache.org
>>>>
>>>>
>>>>
>> ------------------------------**------------------------------**---------
>> To unsubscribe, e-mail: users-unsubscribe@tomcat.**apache.org<us...@tomcat.apache.org>
>> For additional commands, e-mail: users-help@tomcat.apache.org
>>
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: clarification on how data is streamed and handled by tomcat
Posted by S Ahmed <sa...@gmail.com>.
" But if you call certain APIs (e.g. getParameter()) the call will hang
until entire POST request body is received from the client and
processed. "
So this means if you don't reference request.getParameter then it won't be
streamed from the client?
The reason I am asking is I will perform api usage limites, and it would be
great if I don't have to bring in the entire request header + body if I can
get away with it.
On Wed, Jan 4, 2012 at 1:58 PM, André Warnier <aw...@ice-sa.com> wrote:
> S Ahmed wrote:
>
>> Tomcat 6
>> Spring MVC, where my controllers method has both httpservletrequest and
>> httpservletresponse as parameters.
>>
>> The point is that I want to know the effects of others sending me large
>> values in a http post (not an image upload, but a form post).
>>
>> I'm assuming once it is sent by the client as a http post, and my servlet
>> responds to the request tomcat has already streamed that data and whether
>> I
>> do:
>>
>> String p1 = request.getParameter("big_**payload")
>>
>> or not, it has already been loaded into memory.
>>
>> Am I correct?
>>
>
> To me, it seems that Konstantin already answered those questions precisely
> and in detail below.
>
>
>
>> Is there a maximize size setting in tomcat?
>>
>> On Wed, Jan 4, 2012 at 1:03 PM, Konstantin Kolinko
>> <kn...@gmail.com>wrote:
>>
>> 2012/1/4 S Ahmed <sa...@gmail.com>:
>>>
>>>> Say I have a simple servlet or spring mvc application running on tomcat.
>>>>
>>>> Tomcat is running as a webserver on port 80.
>>>>
>>>> A client makes a http POST request to my page www.example.com/submit
>>>>
>>>> If the client adds a very large file with the POST, does this mean that
>>>> when my servlet code starts to execute, tomcat already has the entire
>>>> request content in memory? (including the very large POST parameter)
>>>>
>>> 1. Tomcat version =?
>>> 2. What API are you using to read the file?
>>>
>>> In general request processing starts after the headers in HTTP request
>>> have been read. At that point you can call getInputStream() and read
>>> the body of the POST request while it is being received from the
>>> client.
>>>
>>> But if you call certain APIs (e.g. getParameter()) the call will hang
>>> until entire POST request body is received from the client and
>>> processed. It is said that those methods "consume" the body of a POST
>>> request.
>>>
>>> 3. In most implementations large file uploads are not stored in
>>> memory, but are written to a temporary file on your hard drive. (So
>>> the request body is processed, but you cannot say that it is "in
>>> memory").
>>>
>>> ------------------------------**------------------------------**
>>> ---------
>>> To unsubscribe, e-mail: users-unsubscribe@tomcat.**apache.org<us...@tomcat.apache.org>
>>> For additional commands, e-mail: users-help@tomcat.apache.org
>>>
>>>
>>>
>>
>
> ------------------------------**------------------------------**---------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.**apache.org<us...@tomcat.apache.org>
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
Re: clarification on how data is streamed and handled by tomcat
Posted by André Warnier <aw...@ice-sa.com>.
S Ahmed wrote:
> Tomcat 6
> Spring MVC, where my controllers method has both httpservletrequest and
> httpservletresponse as parameters.
>
> The point is that I want to know the effects of others sending me large
> values in a http post (not an image upload, but a form post).
>
> I'm assuming once it is sent by the client as a http post, and my servlet
> responds to the request tomcat has already streamed that data and whether I
> do:
>
> String p1 = request.getParameter("big_payload")
>
> or not, it has already been loaded into memory.
>
> Am I correct?
To me, it seems that Konstantin already answered those questions precisely and in detail
below.
>
> Is there a maximize size setting in tomcat?
>
> On Wed, Jan 4, 2012 at 1:03 PM, Konstantin Kolinko
> <kn...@gmail.com>wrote:
>
>> 2012/1/4 S Ahmed <sa...@gmail.com>:
>>> Say I have a simple servlet or spring mvc application running on tomcat.
>>>
>>> Tomcat is running as a webserver on port 80.
>>>
>>> A client makes a http POST request to my page www.example.com/submit
>>>
>>> If the client adds a very large file with the POST, does this mean that
>>> when my servlet code starts to execute, tomcat already has the entire
>>> request content in memory? (including the very large POST parameter)
>> 1. Tomcat version =?
>> 2. What API are you using to read the file?
>>
>> In general request processing starts after the headers in HTTP request
>> have been read. At that point you can call getInputStream() and read
>> the body of the POST request while it is being received from the
>> client.
>>
>> But if you call certain APIs (e.g. getParameter()) the call will hang
>> until entire POST request body is received from the client and
>> processed. It is said that those methods "consume" the body of a POST
>> request.
>>
>> 3. In most implementations large file uploads are not stored in
>> memory, but are written to a temporary file on your hard drive. (So
>> the request body is processed, but you cannot say that it is "in
>> memory").
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>> For additional commands, e-mail: users-help@tomcat.apache.org
>>
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: clarification on how data is streamed and handled by tomcat
Posted by S Ahmed <sa...@gmail.com>.
Tomcat 6
Spring MVC, where my controllers method has both httpservletrequest and
httpservletresponse as parameters.
The point is that I want to know the effects of others sending me large
values in a http post (not an image upload, but a form post).
I'm assuming once it is sent by the client as a http post, and my servlet
responds to the request tomcat has already streamed that data and whether I
do:
String p1 = request.getParameter("big_payload")
or not, it has already been loaded into memory.
Am I correct?
Is there a maximize size setting in tomcat?
On Wed, Jan 4, 2012 at 1:03 PM, Konstantin Kolinko
<kn...@gmail.com>wrote:
> 2012/1/4 S Ahmed <sa...@gmail.com>:
> > Say I have a simple servlet or spring mvc application running on tomcat.
> >
> > Tomcat is running as a webserver on port 80.
> >
> > A client makes a http POST request to my page www.example.com/submit
> >
> > If the client adds a very large file with the POST, does this mean that
> > when my servlet code starts to execute, tomcat already has the entire
> > request content in memory? (including the very large POST parameter)
>
> 1. Tomcat version =?
> 2. What API are you using to read the file?
>
> In general request processing starts after the headers in HTTP request
> have been read. At that point you can call getInputStream() and read
> the body of the POST request while it is being received from the
> client.
>
> But if you call certain APIs (e.g. getParameter()) the call will hang
> until entire POST request body is received from the client and
> processed. It is said that those methods "consume" the body of a POST
> request.
>
> 3. In most implementations large file uploads are not stored in
> memory, but are written to a temporary file on your hard drive. (So
> the request body is processed, but you cannot say that it is "in
> memory").
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
Re: clarification on how data is streamed and handled by tomcat
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Konstantin,
On 1/4/12 1:03 PM, Konstantin Kolinko wrote:
> But if you call certain APIs (e.g. getParameter()) the call will
> hang until entire POST request body is received from the client
> and processed. It is said that those methods "consume" the body of
> a POST request.
I should point out that this only happens if the request method is
POST and the content-type is application/x-www-form-urlencoded or if
the servlet is marked with @MultipartConfig and the content-type is
multipart/form-data, or if you have used the parseBodyMethods setting
with a value of anything other than "POST" and have an appropriate
content-type.
If the content-type is something else, or the request is not POST,
then Tomcat will only read the query string when getParameter() is called.
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAk8E30YACgkQ9CaO5/Lv0PDelACfbjyl6ajogZa6uCUVZ4jyiilC
6aMAoKUIxRLVw2GoGd2xAqbw2fDSkJky
=6VPd
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: clarification on how data is streamed and handled by tomcat
Posted by Konstantin Kolinko <kn...@gmail.com>.
2012/1/4 S Ahmed <sa...@gmail.com>:
> Say I have a simple servlet or spring mvc application running on tomcat.
>
> Tomcat is running as a webserver on port 80.
>
> A client makes a http POST request to my page www.example.com/submit
>
> If the client adds a very large file with the POST, does this mean that
> when my servlet code starts to execute, tomcat already has the entire
> request content in memory? (including the very large POST parameter)
1. Tomcat version =?
2. What API are you using to read the file?
In general request processing starts after the headers in HTTP request
have been read. At that point you can call getInputStream() and read
the body of the POST request while it is being received from the
client.
But if you call certain APIs (e.g. getParameter()) the call will hang
until entire POST request body is received from the client and
processed. It is said that those methods "consume" the body of a POST
request.
3. In most implementations large file uploads are not stored in
memory, but are written to a temporary file on your hard drive. (So
the request body is processed, but you cannot say that it is "in
memory").
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org