You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tomcat.apache.org by Dave Rathnow <Da...@umsltd.com> on 2007/10/22 22:00:22 UTC
Measuring bytes sent and received from and to Tomcat
We have an application that collects data from, and sends data to,
remote embedded devices. Traditionally we have used TCP and UDP to send
and receive data over satellite. The latest release of our product will
be using other communication medium with our devices making HTTP request
to our application that is running under JBoss/Tomcat.
The way we bill our clients is by charging them a usage fee based on the
number of bytes being sent over the air/wire. Because of this, we need
to have a accurate count of the number of bytes sent and received from
each site, which is uniquely identified by it's IP address. Using
either UDP and TCP this is simple as we are in control of the end
socket.
Is there a way we can do the same thing with Tomcat? It's simple for us
to measure the number of byte in the payload of the HTTP
request/response, however that isn't enough. We need to know the total
number of bytes being sent and received for each HTTP request.
Can someone suggest a way I could get an accurate count of these bytes?
Thanks,
Dave.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
RE: Measuring bytes sent and received from and to Tomcat
Posted by Dave Rathnow <Da...@umsltd.com>.
For our application, it's not that complicated. All we need to know is
the number of bytes sent and received over the wire. We aren't using
SSL, redirects, etc.
-----Original Message-----
From: Johnny Kewl [mailto:john@kewlstuff.co.za]
Sent: October 22, 2007 11:26 PM
To: Tomcat Developers List
Subject: Re: Measuring bytes sent and received from and to Tomcat
------------------------------------------------------------------------
---
HARBOR: http://coolharbor.100free.com/index.htm
Now Tomcat is also a cool application server
------------------------------------------------------------------------
---
----- Original Message -----
From: "Dave Rathnow" <Da...@umsltd.com>
To: "Tomcat Developers List" <de...@tomcat.apache.org>
Sent: Monday, October 22, 2007 10:00 PM
Subject: Measuring bytes sent and received from and to Tomcat
=====================================================
Hi there, interesting question, more I think about it, more complicated
it gets ;)
Dont think its easy from TC, its too sophisticated, compression, SSL,
redirects, dispatches, clustering... think its hard to get a true
network measurement.
I would plunder something like TCPMon
https://tcpmon.dev.java.net/source/browse/tcpmon/
Its a NB plugin so can play with it first....
Its really just a (bind - client) ie port 8080 to 8081 type idea - so
its easy to install, and easy to setup across multiple sites, clusters
etc etc.
Steal this (relay or tunnel) code and just mod it... I think you will be
able to modify it for client IP's cookies, special headers... anything
and then call it from a browser and get client billing breakdowns....
maybe...
==============================================
We have an application that collects data from, and sends data to,
remote embedded devices. Traditionally we have used TCP and UDP to send
and receive data over satellite. The latest release of our product will
be using other communication medium with our devices making HTTP request
to our application that is running under JBoss/Tomcat.
The way we bill our clients is by charging them a usage fee based on the
number of bytes being sent over the air/wire. Because of this, we need
to have a accurate count of the number of bytes sent and received from
each site, which is uniquely identified by it's IP address. Using
either UDP and TCP this is simple as we are in control of the end
socket.
Is there a way we can do the same thing with Tomcat? It's simple for us
to measure the number of byte in the payload of the HTTP
request/response, however that isn't enough. We need to know the total
number of bytes being sent and received for each HTTP request.
Can someone suggest a way I could get an accurate count of these bytes?
Thanks,
Dave.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org For additional
commands, e-mail: dev-help@tomcat.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org For additional
commands, e-mail: dev-help@tomcat.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: Measuring bytes sent and received from and to Tomcat
Posted by Johnny Kewl <jo...@kewlstuff.co.za>.
---------------------------------------------------------------------------
HARBOR: http://coolharbor.100free.com/index.htm
Now Tomcat is also a cool application server
---------------------------------------------------------------------------
----- Original Message -----
From: "Dave Rathnow" <Da...@umsltd.com>
To: "Tomcat Developers List" <de...@tomcat.apache.org>
Sent: Monday, October 22, 2007 10:00 PM
Subject: Measuring bytes sent and received from and to Tomcat
=====================================================
Hi there, interesting question, more I think about it, more complicated it
gets ;)
Dont think its easy from TC, its too sophisticated, compression, SSL,
redirects, dispatches, clustering... think its hard to get a true network
measurement.
I would plunder something like TCPMon
https://tcpmon.dev.java.net/source/browse/tcpmon/
Its a NB plugin so can play with it first....
Its really just a (bind - client) ie port 8080 to 8081 type idea - so its
easy to install, and easy to setup across multiple sites, clusters etc etc.
Steal this (relay or tunnel) code and just mod it... I think you will be
able to modify it for client IP's cookies, special headers... anything
and then call it from a browser and get client billing breakdowns....
maybe...
==============================================
We have an application that collects data from, and sends data to,
remote embedded devices. Traditionally we have used TCP and UDP to send
and receive data over satellite. The latest release of our product will
be using other communication medium with our devices making HTTP request
to our application that is running under JBoss/Tomcat.
The way we bill our clients is by charging them a usage fee based on the
number of bytes being sent over the air/wire. Because of this, we need
to have a accurate count of the number of bytes sent and received from
each site, which is uniquely identified by it's IP address. Using
either UDP and TCP this is simple as we are in control of the end
socket.
Is there a way we can do the same thing with Tomcat? It's simple for us
to measure the number of byte in the payload of the HTTP
request/response, however that isn't enough. We need to know the total
number of bytes being sent and received for each HTTP request.
Can someone suggest a way I could get an accurate count of these bytes?
Thanks,
Dave.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: Measuring bytes sent and received from and to Tomcat
Posted by Filip Hanik - Dev Lists <de...@hanik.com>.
it's very easy, take a look at
http://svn.apache.org/viewvc/tomcat/tc6.0.x/trunk/java/org/apache/coyote/http11/InternalOutputBuffer.java?view=markup
then look for (case sensitive search)
outputStream.write(
and that is where we write bytes to the socket, just modify this class
for your own purposes
Filip
Dave Rathnow wrote:
> I took a look at lamdaprobe but it only counts the payload and not the
> HTTP request.
>
> Back to my original question.....can anyone help?
>
>
> -----Original Message-----
> From: Henri Gomez [mailto:henri.gomez@gmail.com]
> Sent: October 26, 2007 05:31 AM
> To: Tomcat Developers List
> Subject: Re: Measuring bytes sent and received from and to Tomcat
>
> Well it should works since the Lamba Probe, got these numbers for HTTP
> and AJP.
>
> ie :
>
> http://www.lambdaprobe.org/d/screenshots/full/charts.png
>
> Regards
>
> 2007/10/25, Dave Rathnow <Da...@umsltd.com>:
>
>> Hello Again,
>>
>> I was wondering if someone could give me some help with this. I think
>>
>
>
>> using a connector is probably the way to go to solve this problem;
>> however, I'm not sure where to start. How do I create my own
>> connector and the plumb it into Tomcat so it will be used. I will be
>> using a separate port other than 8080 for the devices that will be
>> sending and receiveing data so, if possible, I would like to leave the
>>
>
>
>> default connector on port 8080.
>>
>> Again, I don't want to reimplement the code that parses the HTTP. All
>>
>
>
>> I need to do is count the number of bytes arriving and being sent so
>> if I can reuse code from an existing connector, that would be great.
>>
>> Thanks,
>> Dave.
>>
>> -----Original Message-----
>> From: costin@gmail.com [mailto:costin@gmail.com] On Behalf Of Costin
>> Manolache
>> Sent: October 22, 2007 04:50 PM
>> To: Dave Rathnow
>> Cc: Tomcat Developers List
>> Subject: Re: Measuring bytes sent and received from and to Tomcat
>>
>> Well, if you want absolute byte - connector seems the only place,
>> there are space and tabs beeing skipped when parsing headers, etc.
>>
>> If you are ok with an estimate - the AccessLogValve is ok, add all the
>>
>
>
>> header lengths + method + http/1.1. You'll miss bytes for encodings,
>> spaces.
>>
>> Re. where to add - each connector is different on how it reads/parse
>> the message, you probably want to do it close to the 'read()' call,
>> save it somewhere associated with the request ( a note or attribute )
>> and read it in a valve or filter.
>>
>> Costin
>>
>>
>> On 10/22/07, Dave Rathnow <Da...@umsltd.com> wrote:
>>
>>> I looked at connectors but wasn't sure if this was what I wanted.
>>> To avoid anther wild goose chase I decided to ask. Can you point me
>>>
>
>
>>> in the direction of some documentation where I might be able to get
>>>
>> started?
>>
>>> Dave.
>>>
>>> -----Original Message-----
>>> From: Costin Manolache [mailto:costin@gmail.com]
>>> Sent: October 22, 2007 04:28 PM
>>> To: Tomcat Developers List
>>> Subject: Re: Measuring bytes sent and received from and to Tomcat
>>>
>>> 'bytes' should be counted at a lower level, in connector. I'm not
>>> sure
>>>
>>> this is something generic enough - but you can make some changes to
>>> your tomcat, where read() is done from socket.
>>>
>>> I guess it would be nice to have a JMX graph with bytes/sec in/out.
>>>
>>> Costin
>>> 'bytes'
>>>
>>> On 10/22/07, Dave Rathnow <Da...@umsltd.com> wrote:
>>>
>>>> We looked at using a valve but we weren't sure if it would work.
>>>> Correct me if I'm wrong, but it appears as though valves are
>>>> chained
>>>>
>>>> together in a calling sequence and that some valves could change
>>>> the
>>>>
>>>> content of the request or response. This means we may not get an
>>>> accurate measure of the number of total number bytes that make up
>>>> the request.
>>>>
>>>> Also, the AccessLogValve has a pattern code to get the number of
>>>> bytes
>>>>
>>>> sent, excluding the HTTP headers, but does not have a pattern code
>>>>
>
>
>>>> to get the number of bytes sent, including the HTTP headers, which
>>>>
>
>
>>>> is what we really need.
>>>>
>>>> Have I missed something?
>>>>
>>>> Dave.
>>>>
>>>>
>>>> -----Original Message-----
>>>> From: yoavshapira@gmail.com [mailto:yoavshapira@gmail.com] On
>>>> Behalf
>>>>
>>>> Of Yoav Shapira
>>>> Sent: October 22, 2007 02:36 PM
>>>> To: Tomcat Developers List
>>>> Subject: Re: Measuring bytes sent and received from and to Tomcat
>>>>
>>>> Hey,
>>>>
>>>> On 10/22/07, Dave Rathnow <Da...@umsltd.com> wrote:
>>>>
>>>>> Is there a way we can do the same thing with Tomcat? It's
>>>>> simple for us to measure the number of byte in the payload of
>>>>> the HTTP request/response, however that isn't enough. We need
>>>>> to know the total number of bytes being sent and received for
>>>>> each HTTP
>>>>>
>> request.
>>
>>>>> Can someone suggest a way I could get an accurate count of these
>>>>>
>>>> bytes?
>>>>
>>>> You can probably start with the AccessLogValve that ships with
>>>>
>> Tomcat:
>>
>>>> http://tomcat.apache.org/tomcat-6.0-doc/config/valve.html
>>>>
>>>> Out of the box it will get you the complete bytes in the response.
>>>> See the above docs on how to configure that. If you want to log
>>>> the
>>>>
>>>> complete bytes on the request, I think you'll have to extend the
>>>> Valve, but it should be pretty easy to do.
>>>>
>>>> Yoav
>>>>
>>>> ------------------------------------------------------------------
>>>> --
>>>> - To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org For
>>>> additional commands, e-mail: dev-help@tomcat.apache.org
>>>>
>>>>
>>>> ------------------------------------------------------------------
>>>> --
>>>> - To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org For
>>>> additional commands, e-mail: dev-help@tomcat.apache.org
>>>>
>>>>
>>>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org For
>> additional commands, e-mail: dev-help@tomcat.apache.org
>>
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org For additional
> commands, e-mail: dev-help@tomcat.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: dev-help@tomcat.apache.org
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: Measuring bytes sent and received from and to Tomcat
Posted by Johnny Kewl <jo...@kewlstuff.co.za>.
---------------------------------------------------------------------------
HARBOR: http://coolharbor.100free.com/index.htm
Now Tomcat is also a cool pojo application server
---------------------------------------------------------------------------
David, have you looked at this...
https://tcpmon.dev.java.net/source/browse/tcpmon/
Its going to be easy... because it almost does what you want already.
It does mean that in TC you will change the connector port to say 8081
But the clients will still see 8080
ie [Client whether that be Browser or Apache] --- 8080[Billing
System]8081 ----- [TC]
So you dont mess with TC's internal stable code (what are you going to do
with the data if you do that?).... keep it outside.
Ig start writing reports to disk from TC's buffer socket.... I think you
going to kill TC's performance, so then you going to want to sneak the data
into a filter or create a special header.... oh boy!
Now... if you build your tunnel (Billing System) in a servlet....
>From your admin port say (8085) you from a browser, simple start/stop
Billing System.
As soon as that happens clients can see TC, and if the billing system is
off.... clients are off.
Then you also have a way to get data.... you can ask the Billing System...
how many bytes for Client IP xyz (its a servlet), etc.
When you business get huge with load sharing and remote offices.... the
billing system is just a servlet, so its easy.
I would keep it outside of TC.... but put it in a servlet.
If you CVS that swoftware down I think you going to find the relay socket
logic is probably 30 lines of code... it listens and forwards, cant be too
bad...
And then... if you want (unlikely) you embed your special servlet in TC, and
add some xml configuration...
My 1/2 cent worth ;)
----- Original Message -----
From: "Dave Rathnow" <Da...@umsltd.com>
To: "Tomcat Developers List" <de...@tomcat.apache.org>
Sent: Friday, October 26, 2007 11:58 PM
Subject: RE: Measuring bytes sent and received from and to Tomcat
I took a look at lamdaprobe but it only counts the payload and not the
HTTP request.
Back to my original question.....can anyone help?
-----Original Message-----
From: Henri Gomez [mailto:henri.gomez@gmail.com]
Sent: October 26, 2007 05:31 AM
To: Tomcat Developers List
Subject: Re: Measuring bytes sent and received from and to Tomcat
Well it should works since the Lamba Probe, got these numbers for HTTP
and AJP.
ie :
http://www.lambdaprobe.org/d/screenshots/full/charts.png
Regards
2007/10/25, Dave Rathnow <Da...@umsltd.com>:
>
> Hello Again,
>
> I was wondering if someone could give me some help with this. I think
> using a connector is probably the way to go to solve this problem;
> however, I'm not sure where to start. How do I create my own
> connector and the plumb it into Tomcat so it will be used. I will be
> using a separate port other than 8080 for the devices that will be
> sending and receiveing data so, if possible, I would like to leave the
> default connector on port 8080.
>
> Again, I don't want to reimplement the code that parses the HTTP. All
> I need to do is count the number of bytes arriving and being sent so
> if I can reuse code from an existing connector, that would be great.
>
> Thanks,
> Dave.
>
> -----Original Message-----
> From: costin@gmail.com [mailto:costin@gmail.com] On Behalf Of Costin
> Manolache
> Sent: October 22, 2007 04:50 PM
> To: Dave Rathnow
> Cc: Tomcat Developers List
> Subject: Re: Measuring bytes sent and received from and to Tomcat
>
> Well, if you want absolute byte - connector seems the only place,
> there are space and tabs beeing skipped when parsing headers, etc.
>
> If you are ok with an estimate - the AccessLogValve is ok, add all the
> header lengths + method + http/1.1. You'll miss bytes for encodings,
> spaces.
>
> Re. where to add - each connector is different on how it reads/parse
> the message, you probably want to do it close to the 'read()' call,
> save it somewhere associated with the request ( a note or attribute )
> and read it in a valve or filter.
>
> Costin
>
>
> On 10/22/07, Dave Rathnow <Da...@umsltd.com> wrote:
> >
> >
> > I looked at connectors but wasn't sure if this was what I wanted.
> > To avoid anther wild goose chase I decided to ask. Can you point me
> > in the direction of some documentation where I might be able to get
> started?
> >
> > Dave.
> >
> > -----Original Message-----
> > From: Costin Manolache [mailto:costin@gmail.com]
> > Sent: October 22, 2007 04:28 PM
> > To: Tomcat Developers List
> > Subject: Re: Measuring bytes sent and received from and to Tomcat
> >
> > 'bytes' should be counted at a lower level, in connector. I'm not
> > sure
>
> > this is something generic enough - but you can make some changes to
> > your tomcat, where read() is done from socket.
> >
> > I guess it would be nice to have a JMX graph with bytes/sec in/out.
> >
> > Costin
> > 'bytes'
> >
> > On 10/22/07, Dave Rathnow <Da...@umsltd.com> wrote:
> > >
> > >
> > > We looked at using a valve but we weren't sure if it would work.
> > > Correct me if I'm wrong, but it appears as though valves are
> > > chained
>
> > > together in a calling sequence and that some valves could change
> > > the
>
> > > content of the request or response. This means we may not get an
> > > accurate measure of the number of total number bytes that make up
> > > the request.
> > >
> > > Also, the AccessLogValve has a pattern code to get the number of
> > > bytes
> >
> > > sent, excluding the HTTP headers, but does not have a pattern code
> > > to get the number of bytes sent, including the HTTP headers, which
> > > is what we really need.
> > >
> > > Have I missed something?
> > >
> > > Dave.
> > >
> > >
> > > -----Original Message-----
> > > From: yoavshapira@gmail.com [mailto:yoavshapira@gmail.com] On
> > > Behalf
>
> > > Of Yoav Shapira
> > > Sent: October 22, 2007 02:36 PM
> > > To: Tomcat Developers List
> > > Subject: Re: Measuring bytes sent and received from and to Tomcat
> > >
> > > Hey,
> > >
> > > On 10/22/07, Dave Rathnow <Da...@umsltd.com> wrote:
> > > > Is there a way we can do the same thing with Tomcat? It's
> > > > simple for us to measure the number of byte in the payload of
> > > > the HTTP request/response, however that isn't enough. We need
> > > > to know the total number of bytes being sent and received for
> > > > each HTTP
> request.
> > > >
> > > > Can someone suggest a way I could get an accurate count of these
> > > bytes?
> > >
> > > You can probably start with the AccessLogValve that ships with
> Tomcat:
> > > http://tomcat.apache.org/tomcat-6.0-doc/config/valve.html
> > >
> > > Out of the box it will get you the complete bytes in the response.
> > > See the above docs on how to configure that. If you want to log
> > > the
>
> > > complete bytes on the request, I think you'll have to extend the
> > > Valve, but it should be pretty easy to do.
> > >
> > > Yoav
> > >
> > > ------------------------------------------------------------------
> > > --
> > > - To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org For
> > > additional commands, e-mail: dev-help@tomcat.apache.org
> > >
> > >
> > > ------------------------------------------------------------------
> > > --
> > > - To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org For
> > > additional commands, e-mail: dev-help@tomcat.apache.org
> > >
> > >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org For
> additional commands, e-mail: dev-help@tomcat.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org For additional
commands, e-mail: dev-help@tomcat.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
RE: Measuring bytes sent and received from and to Tomcat
Posted by Dave Rathnow <Da...@umsltd.com>.
I took a look at lamdaprobe but it only counts the payload and not the
HTTP request.
Back to my original question.....can anyone help?
-----Original Message-----
From: Henri Gomez [mailto:henri.gomez@gmail.com]
Sent: October 26, 2007 05:31 AM
To: Tomcat Developers List
Subject: Re: Measuring bytes sent and received from and to Tomcat
Well it should works since the Lamba Probe, got these numbers for HTTP
and AJP.
ie :
http://www.lambdaprobe.org/d/screenshots/full/charts.png
Regards
2007/10/25, Dave Rathnow <Da...@umsltd.com>:
>
> Hello Again,
>
> I was wondering if someone could give me some help with this. I think
> using a connector is probably the way to go to solve this problem;
> however, I'm not sure where to start. How do I create my own
> connector and the plumb it into Tomcat so it will be used. I will be
> using a separate port other than 8080 for the devices that will be
> sending and receiveing data so, if possible, I would like to leave the
> default connector on port 8080.
>
> Again, I don't want to reimplement the code that parses the HTTP. All
> I need to do is count the number of bytes arriving and being sent so
> if I can reuse code from an existing connector, that would be great.
>
> Thanks,
> Dave.
>
> -----Original Message-----
> From: costin@gmail.com [mailto:costin@gmail.com] On Behalf Of Costin
> Manolache
> Sent: October 22, 2007 04:50 PM
> To: Dave Rathnow
> Cc: Tomcat Developers List
> Subject: Re: Measuring bytes sent and received from and to Tomcat
>
> Well, if you want absolute byte - connector seems the only place,
> there are space and tabs beeing skipped when parsing headers, etc.
>
> If you are ok with an estimate - the AccessLogValve is ok, add all the
> header lengths + method + http/1.1. You'll miss bytes for encodings,
> spaces.
>
> Re. where to add - each connector is different on how it reads/parse
> the message, you probably want to do it close to the 'read()' call,
> save it somewhere associated with the request ( a note or attribute )
> and read it in a valve or filter.
>
> Costin
>
>
> On 10/22/07, Dave Rathnow <Da...@umsltd.com> wrote:
> >
> >
> > I looked at connectors but wasn't sure if this was what I wanted.
> > To avoid anther wild goose chase I decided to ask. Can you point me
> > in the direction of some documentation where I might be able to get
> started?
> >
> > Dave.
> >
> > -----Original Message-----
> > From: Costin Manolache [mailto:costin@gmail.com]
> > Sent: October 22, 2007 04:28 PM
> > To: Tomcat Developers List
> > Subject: Re: Measuring bytes sent and received from and to Tomcat
> >
> > 'bytes' should be counted at a lower level, in connector. I'm not
> > sure
>
> > this is something generic enough - but you can make some changes to
> > your tomcat, where read() is done from socket.
> >
> > I guess it would be nice to have a JMX graph with bytes/sec in/out.
> >
> > Costin
> > 'bytes'
> >
> > On 10/22/07, Dave Rathnow <Da...@umsltd.com> wrote:
> > >
> > >
> > > We looked at using a valve but we weren't sure if it would work.
> > > Correct me if I'm wrong, but it appears as though valves are
> > > chained
>
> > > together in a calling sequence and that some valves could change
> > > the
>
> > > content of the request or response. This means we may not get an
> > > accurate measure of the number of total number bytes that make up
> > > the request.
> > >
> > > Also, the AccessLogValve has a pattern code to get the number of
> > > bytes
> >
> > > sent, excluding the HTTP headers, but does not have a pattern code
> > > to get the number of bytes sent, including the HTTP headers, which
> > > is what we really need.
> > >
> > > Have I missed something?
> > >
> > > Dave.
> > >
> > >
> > > -----Original Message-----
> > > From: yoavshapira@gmail.com [mailto:yoavshapira@gmail.com] On
> > > Behalf
>
> > > Of Yoav Shapira
> > > Sent: October 22, 2007 02:36 PM
> > > To: Tomcat Developers List
> > > Subject: Re: Measuring bytes sent and received from and to Tomcat
> > >
> > > Hey,
> > >
> > > On 10/22/07, Dave Rathnow <Da...@umsltd.com> wrote:
> > > > Is there a way we can do the same thing with Tomcat? It's
> > > > simple for us to measure the number of byte in the payload of
> > > > the HTTP request/response, however that isn't enough. We need
> > > > to know the total number of bytes being sent and received for
> > > > each HTTP
> request.
> > > >
> > > > Can someone suggest a way I could get an accurate count of these
> > > bytes?
> > >
> > > You can probably start with the AccessLogValve that ships with
> Tomcat:
> > > http://tomcat.apache.org/tomcat-6.0-doc/config/valve.html
> > >
> > > Out of the box it will get you the complete bytes in the response.
> > > See the above docs on how to configure that. If you want to log
> > > the
>
> > > complete bytes on the request, I think you'll have to extend the
> > > Valve, but it should be pretty easy to do.
> > >
> > > Yoav
> > >
> > > ------------------------------------------------------------------
> > > --
> > > - To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org For
> > > additional commands, e-mail: dev-help@tomcat.apache.org
> > >
> > >
> > > ------------------------------------------------------------------
> > > --
> > > - To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org For
> > > additional commands, e-mail: dev-help@tomcat.apache.org
> > >
> > >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org For
> additional commands, e-mail: dev-help@tomcat.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org For additional
commands, e-mail: dev-help@tomcat.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: Measuring bytes sent and received from and to Tomcat
Posted by Henri Gomez <he...@gmail.com>.
Well it should works since the Lamba Probe, got these numbers for HTTP and AJP.
ie :
http://www.lambdaprobe.org/d/screenshots/full/charts.png
Regards
2007/10/25, Dave Rathnow <Da...@umsltd.com>:
>
> Hello Again,
>
> I was wondering if someone could give me some help with this. I think
> using a connector is probably the way to go to solve this problem;
> however, I'm not sure where to start. How do I create my own connector
> and the plumb it into Tomcat so it will be used. I will be using a
> separate port other than 8080 for the devices that will be sending and
> receiveing data so, if possible, I would like to leave the default
> connector on port 8080.
>
> Again, I don't want to reimplement the code that parses the HTTP. All I
> need to do is count the number of bytes arriving and being sent so if I
> can reuse code from an existing connector, that would be great.
>
> Thanks,
> Dave.
>
> -----Original Message-----
> From: costin@gmail.com [mailto:costin@gmail.com] On Behalf Of Costin
> Manolache
> Sent: October 22, 2007 04:50 PM
> To: Dave Rathnow
> Cc: Tomcat Developers List
> Subject: Re: Measuring bytes sent and received from and to Tomcat
>
> Well, if you want absolute byte - connector seems the only place, there
> are space and tabs beeing skipped when parsing headers, etc.
>
> If you are ok with an estimate - the AccessLogValve is ok, add all the
> header lengths + method + http/1.1. You'll miss bytes for encodings,
> spaces.
>
> Re. where to add - each connector is different on how it reads/parse the
> message, you probably want to do it close to the 'read()' call, save it
> somewhere associated with the request ( a note or attribute ) and read
> it in a valve or filter.
>
> Costin
>
>
> On 10/22/07, Dave Rathnow <Da...@umsltd.com> wrote:
> >
> >
> > I looked at connectors but wasn't sure if this was what I wanted. To
> > avoid anther wild goose chase I decided to ask. Can you point me in
> > the direction of some documentation where I might be able to get
> started?
> >
> > Dave.
> >
> > -----Original Message-----
> > From: Costin Manolache [mailto:costin@gmail.com]
> > Sent: October 22, 2007 04:28 PM
> > To: Tomcat Developers List
> > Subject: Re: Measuring bytes sent and received from and to Tomcat
> >
> > 'bytes' should be counted at a lower level, in connector. I'm not sure
>
> > this is something generic enough - but you can make some changes to
> > your tomcat, where read() is done from socket.
> >
> > I guess it would be nice to have a JMX graph with bytes/sec in/out.
> >
> > Costin
> > 'bytes'
> >
> > On 10/22/07, Dave Rathnow <Da...@umsltd.com> wrote:
> > >
> > >
> > > We looked at using a valve but we weren't sure if it would work.
> > > Correct me if I'm wrong, but it appears as though valves are chained
>
> > > together in a calling sequence and that some valves could change the
>
> > > content of the request or response. This means we may not get an
> > > accurate measure of the number of total number bytes that make up
> > > the request.
> > >
> > > Also, the AccessLogValve has a pattern code to get the number of
> > > bytes
> >
> > > sent, excluding the HTTP headers, but does not have a pattern code
> > > to get the number of bytes sent, including the HTTP headers, which
> > > is what we really need.
> > >
> > > Have I missed something?
> > >
> > > Dave.
> > >
> > >
> > > -----Original Message-----
> > > From: yoavshapira@gmail.com [mailto:yoavshapira@gmail.com] On Behalf
>
> > > Of Yoav Shapira
> > > Sent: October 22, 2007 02:36 PM
> > > To: Tomcat Developers List
> > > Subject: Re: Measuring bytes sent and received from and to Tomcat
> > >
> > > Hey,
> > >
> > > On 10/22/07, Dave Rathnow <Da...@umsltd.com> wrote:
> > > > Is there a way we can do the same thing with Tomcat? It's simple
> > > > for us to measure the number of byte in the payload of the HTTP
> > > > request/response, however that isn't enough. We need to know the
> > > > total number of bytes being sent and received for each HTTP
> request.
> > > >
> > > > Can someone suggest a way I could get an accurate count of these
> > > bytes?
> > >
> > > You can probably start with the AccessLogValve that ships with
> Tomcat:
> > > http://tomcat.apache.org/tomcat-6.0-doc/config/valve.html
> > >
> > > Out of the box it will get you the complete bytes in the response.
> > > See the above docs on how to configure that. If you want to log the
>
> > > complete bytes on the request, I think you'll have to extend the
> > > Valve, but it should be pretty easy to do.
> > >
> > > Yoav
> > >
> > > --------------------------------------------------------------------
> > > - To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org For
> > > additional commands, e-mail: dev-help@tomcat.apache.org
> > >
> > >
> > > --------------------------------------------------------------------
> > > - To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org For
> > > additional commands, e-mail: dev-help@tomcat.apache.org
> > >
> > >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: dev-help@tomcat.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
RE: Measuring bytes sent and received from and to Tomcat
Posted by Dave Rathnow <Da...@umsltd.com>.
Hello Again,
I was wondering if someone could give me some help with this. I think
using a connector is probably the way to go to solve this problem;
however, I'm not sure where to start. How do I create my own connector
and the plumb it into Tomcat so it will be used. I will be using a
separate port other than 8080 for the devices that will be sending and
receiveing data so, if possible, I would like to leave the default
connector on port 8080.
Again, I don't want to reimplement the code that parses the HTTP. All I
need to do is count the number of bytes arriving and being sent so if I
can reuse code from an existing connector, that would be great.
Thanks,
Dave.
-----Original Message-----
From: costin@gmail.com [mailto:costin@gmail.com] On Behalf Of Costin
Manolache
Sent: October 22, 2007 04:50 PM
To: Dave Rathnow
Cc: Tomcat Developers List
Subject: Re: Measuring bytes sent and received from and to Tomcat
Well, if you want absolute byte - connector seems the only place, there
are space and tabs beeing skipped when parsing headers, etc.
If you are ok with an estimate - the AccessLogValve is ok, add all the
header lengths + method + http/1.1. You'll miss bytes for encodings,
spaces.
Re. where to add - each connector is different on how it reads/parse the
message, you probably want to do it close to the 'read()' call, save it
somewhere associated with the request ( a note or attribute ) and read
it in a valve or filter.
Costin
On 10/22/07, Dave Rathnow <Da...@umsltd.com> wrote:
>
>
> I looked at connectors but wasn't sure if this was what I wanted. To
> avoid anther wild goose chase I decided to ask. Can you point me in
> the direction of some documentation where I might be able to get
started?
>
> Dave.
>
> -----Original Message-----
> From: Costin Manolache [mailto:costin@gmail.com]
> Sent: October 22, 2007 04:28 PM
> To: Tomcat Developers List
> Subject: Re: Measuring bytes sent and received from and to Tomcat
>
> 'bytes' should be counted at a lower level, in connector. I'm not sure
> this is something generic enough - but you can make some changes to
> your tomcat, where read() is done from socket.
>
> I guess it would be nice to have a JMX graph with bytes/sec in/out.
>
> Costin
> 'bytes'
>
> On 10/22/07, Dave Rathnow <Da...@umsltd.com> wrote:
> >
> >
> > We looked at using a valve but we weren't sure if it would work.
> > Correct me if I'm wrong, but it appears as though valves are chained
> > together in a calling sequence and that some valves could change the
> > content of the request or response. This means we may not get an
> > accurate measure of the number of total number bytes that make up
> > the request.
> >
> > Also, the AccessLogValve has a pattern code to get the number of
> > bytes
>
> > sent, excluding the HTTP headers, but does not have a pattern code
> > to get the number of bytes sent, including the HTTP headers, which
> > is what we really need.
> >
> > Have I missed something?
> >
> > Dave.
> >
> >
> > -----Original Message-----
> > From: yoavshapira@gmail.com [mailto:yoavshapira@gmail.com] On Behalf
> > Of Yoav Shapira
> > Sent: October 22, 2007 02:36 PM
> > To: Tomcat Developers List
> > Subject: Re: Measuring bytes sent and received from and to Tomcat
> >
> > Hey,
> >
> > On 10/22/07, Dave Rathnow <Da...@umsltd.com> wrote:
> > > Is there a way we can do the same thing with Tomcat? It's simple
> > > for us to measure the number of byte in the payload of the HTTP
> > > request/response, however that isn't enough. We need to know the
> > > total number of bytes being sent and received for each HTTP
request.
> > >
> > > Can someone suggest a way I could get an accurate count of these
> > bytes?
> >
> > You can probably start with the AccessLogValve that ships with
Tomcat:
> > http://tomcat.apache.org/tomcat-6.0-doc/config/valve.html
> >
> > Out of the box it will get you the complete bytes in the response.
> > See the above docs on how to configure that. If you want to log the
> > complete bytes on the request, I think you'll have to extend the
> > Valve, but it should be pretty easy to do.
> >
> > Yoav
> >
> > --------------------------------------------------------------------
> > - To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org For
> > additional commands, e-mail: dev-help@tomcat.apache.org
> >
> >
> > --------------------------------------------------------------------
> > - To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org For
> > additional commands, e-mail: dev-help@tomcat.apache.org
> >
> >
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: Measuring bytes sent and received from and to Tomcat
Posted by Costin Manolache <co...@apache.org>.
Well, if you want absolute byte - connector seems the only place, there are
space and tabs beeing skipped when parsing headers, etc.
If you are ok with an estimate - the AccessLogValve is ok, add all the
header lengths + method + http/1.1. You'll miss bytes for encodings, spaces.
Re. where to add - each connector is different on how it reads/parse the
message, you probably want to do it close to the 'read()' call, save it
somewhere associated with the request ( a note or attribute ) and read it in
a valve or filter.
Costin
On 10/22/07, Dave Rathnow <Da...@umsltd.com> wrote:
>
>
> I looked at connectors but wasn't sure if this was what I wanted. To
> avoid anther wild goose chase I decided to ask. Can you point me in the
> direction of some documentation where I might be able to get started?
>
> Dave.
>
> -----Original Message-----
> From: Costin Manolache [mailto:costin@gmail.com]
> Sent: October 22, 2007 04:28 PM
> To: Tomcat Developers List
> Subject: Re: Measuring bytes sent and received from and to Tomcat
>
> 'bytes' should be counted at a lower level, in connector. I'm not sure
> this is something generic enough - but you can make some changes to your
> tomcat, where read() is done from socket.
>
> I guess it would be nice to have a JMX graph with bytes/sec in/out.
>
> Costin
> 'bytes'
>
> On 10/22/07, Dave Rathnow <Da...@umsltd.com> wrote:
> >
> >
> > We looked at using a valve but we weren't sure if it would work.
> > Correct me if I'm wrong, but it appears as though valves are chained
> > together in a calling sequence and that some valves could change the
> > content of the request or response. This means we may not get an
> > accurate measure of the number of total number bytes that make up the
> > request.
> >
> > Also, the AccessLogValve has a pattern code to get the number of bytes
>
> > sent, excluding the HTTP headers, but does not have a pattern code to
> > get the number of bytes sent, including the HTTP headers, which is
> > what we really need.
> >
> > Have I missed something?
> >
> > Dave.
> >
> >
> > -----Original Message-----
> > From: yoavshapira@gmail.com [mailto:yoavshapira@gmail.com] On Behalf
> > Of Yoav Shapira
> > Sent: October 22, 2007 02:36 PM
> > To: Tomcat Developers List
> > Subject: Re: Measuring bytes sent and received from and to Tomcat
> >
> > Hey,
> >
> > On 10/22/07, Dave Rathnow <Da...@umsltd.com> wrote:
> > > Is there a way we can do the same thing with Tomcat? It's simple
> > > for us to measure the number of byte in the payload of the HTTP
> > > request/response, however that isn't enough. We need to know the
> > > total number of bytes being sent and received for each HTTP request.
> > >
> > > Can someone suggest a way I could get an accurate count of these
> > bytes?
> >
> > You can probably start with the AccessLogValve that ships with Tomcat:
> > http://tomcat.apache.org/tomcat-6.0-doc/config/valve.html
> >
> > Out of the box it will get you the complete bytes in the response.
> > See the above docs on how to configure that. If you want to log the
> > complete bytes on the request, I think you'll have to extend the
> > Valve, but it should be pretty easy to do.
> >
> > Yoav
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org For
> > additional commands, e-mail: dev-help@tomcat.apache.org
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org For
> > additional commands, e-mail: dev-help@tomcat.apache.org
> >
> >
>
RE: Measuring bytes sent and received from and to Tomcat
Posted by Dave Rathnow <Da...@umsltd.com>.
I looked at connectors but wasn't sure if this was what I wanted. To
avoid anther wild goose chase I decided to ask. Can you point me in the
direction of some documentation where I might be able to get started?
Dave.
-----Original Message-----
From: Costin Manolache [mailto:costin@gmail.com]
Sent: October 22, 2007 04:28 PM
To: Tomcat Developers List
Subject: Re: Measuring bytes sent and received from and to Tomcat
'bytes' should be counted at a lower level, in connector. I'm not sure
this is something generic enough - but you can make some changes to your
tomcat, where read() is done from socket.
I guess it would be nice to have a JMX graph with bytes/sec in/out.
Costin
'bytes'
On 10/22/07, Dave Rathnow <Da...@umsltd.com> wrote:
>
>
> We looked at using a valve but we weren't sure if it would work.
> Correct me if I'm wrong, but it appears as though valves are chained
> together in a calling sequence and that some valves could change the
> content of the request or response. This means we may not get an
> accurate measure of the number of total number bytes that make up the
> request.
>
> Also, the AccessLogValve has a pattern code to get the number of bytes
> sent, excluding the HTTP headers, but does not have a pattern code to
> get the number of bytes sent, including the HTTP headers, which is
> what we really need.
>
> Have I missed something?
>
> Dave.
>
>
> -----Original Message-----
> From: yoavshapira@gmail.com [mailto:yoavshapira@gmail.com] On Behalf
> Of Yoav Shapira
> Sent: October 22, 2007 02:36 PM
> To: Tomcat Developers List
> Subject: Re: Measuring bytes sent and received from and to Tomcat
>
> Hey,
>
> On 10/22/07, Dave Rathnow <Da...@umsltd.com> wrote:
> > Is there a way we can do the same thing with Tomcat? It's simple
> > for us to measure the number of byte in the payload of the HTTP
> > request/response, however that isn't enough. We need to know the
> > total number of bytes being sent and received for each HTTP request.
> >
> > Can someone suggest a way I could get an accurate count of these
> bytes?
>
> You can probably start with the AccessLogValve that ships with Tomcat:
> http://tomcat.apache.org/tomcat-6.0-doc/config/valve.html
>
> Out of the box it will get you the complete bytes in the response.
> See the above docs on how to configure that. If you want to log the
> complete bytes on the request, I think you'll have to extend the
> Valve, but it should be pretty easy to do.
>
> Yoav
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org For
> additional commands, e-mail: dev-help@tomcat.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org For
> additional commands, e-mail: dev-help@tomcat.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: Measuring bytes sent and received from and to Tomcat
Posted by Costin Manolache <co...@gmail.com>.
'bytes' should be counted at a lower level, in connector. I'm not sure this
is something generic enough - but you can make some changes to your tomcat,
where read() is done from socket.
I guess it would be nice to have a JMX graph with bytes/sec in/out.
Costin
'bytes'
On 10/22/07, Dave Rathnow <Da...@umsltd.com> wrote:
>
>
> We looked at using a valve but we weren't sure if it would work.
> Correct me if I'm wrong, but it appears as though valves are chained
> together in a calling sequence and that some valves could change the
> content of the request or response. This means we may not get an
> accurate measure of the number of total number bytes that make up the
> request.
>
> Also, the AccessLogValve has a pattern code to get the number of bytes
> sent, excluding the HTTP headers, but does not have a pattern code to
> get the number of bytes sent, including the HTTP headers, which is what
> we really need.
>
> Have I missed something?
>
> Dave.
>
>
> -----Original Message-----
> From: yoavshapira@gmail.com [mailto:yoavshapira@gmail.com] On Behalf Of
> Yoav Shapira
> Sent: October 22, 2007 02:36 PM
> To: Tomcat Developers List
> Subject: Re: Measuring bytes sent and received from and to Tomcat
>
> Hey,
>
> On 10/22/07, Dave Rathnow <Da...@umsltd.com> wrote:
> > Is there a way we can do the same thing with Tomcat? It's simple for
> > us to measure the number of byte in the payload of the HTTP
> > request/response, however that isn't enough. We need to know the
> > total number of bytes being sent and received for each HTTP request.
> >
> > Can someone suggest a way I could get an accurate count of these
> bytes?
>
> You can probably start with the AccessLogValve that ships with Tomcat:
> http://tomcat.apache.org/tomcat-6.0-doc/config/valve.html
>
> Out of the box it will get you the complete bytes in the response.
> See the above docs on how to configure that. If you want to log the
> complete bytes on the request, I think you'll have to extend the Valve,
> but it should be pretty easy to do.
>
> Yoav
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org For additional
> commands, e-mail: dev-help@tomcat.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: dev-help@tomcat.apache.org
>
>
RE: Measuring bytes sent and received from and to Tomcat
Posted by Dave Rathnow <Da...@umsltd.com>.
We looked at using a valve but we weren't sure if it would work.
Correct me if I'm wrong, but it appears as though valves are chained
together in a calling sequence and that some valves could change the
content of the request or response. This means we may not get an
accurate measure of the number of total number bytes that make up the
request.
Also, the AccessLogValve has a pattern code to get the number of bytes
sent, excluding the HTTP headers, but does not have a pattern code to
get the number of bytes sent, including the HTTP headers, which is what
we really need.
Have I missed something?
Dave.
-----Original Message-----
From: yoavshapira@gmail.com [mailto:yoavshapira@gmail.com] On Behalf Of
Yoav Shapira
Sent: October 22, 2007 02:36 PM
To: Tomcat Developers List
Subject: Re: Measuring bytes sent and received from and to Tomcat
Hey,
On 10/22/07, Dave Rathnow <Da...@umsltd.com> wrote:
> Is there a way we can do the same thing with Tomcat? It's simple for
> us to measure the number of byte in the payload of the HTTP
> request/response, however that isn't enough. We need to know the
> total number of bytes being sent and received for each HTTP request.
>
> Can someone suggest a way I could get an accurate count of these
bytes?
You can probably start with the AccessLogValve that ships with Tomcat:
http://tomcat.apache.org/tomcat-6.0-doc/config/valve.html
Out of the box it will get you the complete bytes in the response.
See the above docs on how to configure that. If you want to log the
complete bytes on the request, I think you'll have to extend the Valve,
but it should be pretty easy to do.
Yoav
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org For additional
commands, e-mail: dev-help@tomcat.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: Measuring bytes sent and received from and to Tomcat
Posted by Yoav Shapira <yo...@apache.org>.
Hey,
On 10/22/07, Dave Rathnow <Da...@umsltd.com> wrote:
> Is there a way we can do the same thing with Tomcat? It's simple for us
> to measure the number of byte in the payload of the HTTP
> request/response, however that isn't enough. We need to know the total
> number of bytes being sent and received for each HTTP request.
>
> Can someone suggest a way I could get an accurate count of these bytes?
You can probably start with the AccessLogValve that ships with Tomcat:
http://tomcat.apache.org/tomcat-6.0-doc/config/valve.html
Out of the box it will get you the complete bytes in the response.
See the above docs on how to configure that. If you want to log the
complete bytes on the request, I think you'll have to extend the
Valve, but it should be pretty easy to do.
Yoav
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org