You are viewing a plain text version of this content. The canonical link for it is here.
Posted to server-user@james.apache.org by Craig Raw <cr...@quirk.co.za> on 2004/05/13 12:51:22 UTC
Recounting the bytes of SMTP
Sorry to harp on about this, but I am still as lost as ever in understanding the large difference
between SMTP bytes measured on the wire and the actual size of emails that enter the root processor.
While I understand that the SMTP conversation must be added to the email size for a true comparison,
I am regularly seeing differences of over 5000% between the two. To my eyes, something is seriously
wrong.
I have been taking a close look at the smtpserver log under debug, particularly watching for the
multiple RCPT TO commands that Danny mentioned in an earlier email. I can't see any evidence of this
happening.
Over a period of 1 hour during the weekend, 51Mb of incoming smtp wire traffic was measured (by two
different applications, to be sure) while my mailet in the root processor only saw about 800kb. (The
mailet logs information on all incoming Mail objects unconditionally.) Over this period, 39 RCPT TO
commands were received, and 35 emails were emails were successfully spooled into the root processor.
One email was "Successfully spooled" according to the smtpserver log, but never made it into the
root processor. There was, as far as I can see, no particularly unusual SMTP traffic during this
time. There are blocks of consecutive "Calling reset()" lines, but looking at the source suggests
this is just the watchdog timer resetting. In terms of the general load on the mail server, the
800kb is the 'reasonable' figure, and the 51Mb is not.
Any ideas?
Thanks,
Craig
---------------------------------------------------------------------
To unsubscribe, e-mail: server-user-unsubscribe@james.apache.org
For additional commands, e-mail: server-user-help@james.apache.org
Re: Recounting the bytes of SMTP
Posted by Craig Raw <cr...@quirk.co.za>.
Serge Knystautas wrote:
>
> http://www.zaval.org/products/proxy/index.html
>
> That's a GPL little Java proxy that I use... actually in the process of
> refactoring since I think the authors have given up on it. It creates 2
> log files (1 input, 1 output) per connection, though I think there are
> other ways to log if you wanted.
>
> Maybe run this for a day and then you could have raw socket traffic to
> figure out what that extra traffic is. I cannot imagine what's causing
> 50:1 smtp traffic to message sizes.
>
Thanks Serge, I'll defintely give this a go. The connections separation in logging makes it look
much better than wading through tcpdump files with Ethereal.
---------------------------------------------------------------------
To unsubscribe, e-mail: server-user-unsubscribe@james.apache.org
For additional commands, e-mail: server-user-help@james.apache.org
Re: Recounting the bytes of SMTP
Posted by Serge Knystautas <se...@lokitech.com>.
Craig Raw wrote:
>> You may have mentioned this earlier, but how is your mailet
>> calculating the size of the message?
>
> Here's the block of code inside the mailet:
>
> ByteArrayOutputStream bais = new ByteArrayOutputStream();
> try
> {
> mail.getMessage().writeTo( bais );
> }
> catch( IOException e )
> {
> log( "Error calculating message size", e );
> }
> int msgSize = bais.size();
Yeah, that would work. Well, depending on how much you want to track
this down, I would suggest tcp proxy server to record all raw network
traffic. Basically you'd configure the proxy to listen at port 25, run
James on something like port 6025, and have the proxy just literally log
and transmit all communication to port 6025.
http://www.zaval.org/products/proxy/index.html
That's a GPL little Java proxy that I use... actually in the process of
refactoring since I think the authors have given up on it. It creates 2
log files (1 input, 1 output) per connection, though I think there are
other ways to log if you wanted.
Maybe run this for a day and then you could have raw socket traffic to
figure out what that extra traffic is. I cannot imagine what's causing
50:1 smtp traffic to message sizes.
--
Serge Knystautas
President
Lokitech >> software . strategy . design >> http://www.lokitech.com
p. 301.656.5501
e. sergek@lokitech.com
---------------------------------------------------------------------
To unsubscribe, e-mail: server-user-unsubscribe@james.apache.org
For additional commands, e-mail: server-user-help@james.apache.org
Re: Recounting the bytes of SMTP
Posted by Craig Raw <cr...@quirk.co.za>.
Serge Knystautas wrote:
> Craig Raw wrote:
>
>> Sorry to harp on about this, but I am still as lost as ever in
>> understanding the large difference between SMTP bytes measured on the
>> wire and the actual size of emails that enter the root processor.
>> While I understand that the SMTP conversation must be added to the
>> email size for a true comparison, I am regularly seeing differences of
>> over 5000% between the two. To my eyes, something is seriously wrong.
>
>
> You may have mentioned this earlier, but how is your mailet calculating
> the size of the message?
>
Here's the block of code inside the mailet:
ByteArrayOutputStream bais = new ByteArrayOutputStream();
try
{
mail.getMessage().writeTo( bais );
}
catch( IOException e )
{
log( "Error calculating message size", e );
}
int msgSize = bais.size();
---------------------------------------------------------------------
To unsubscribe, e-mail: server-user-unsubscribe@james.apache.org
For additional commands, e-mail: server-user-help@james.apache.org
Re: Recounting the bytes of SMTP
Posted by Serge Knystautas <se...@lokitech.com>.
Craig Raw wrote:
> Sorry to harp on about this, but I am still as lost as ever in
> understanding the large difference between SMTP bytes measured on the
> wire and the actual size of emails that enter the root processor. While
> I understand that the SMTP conversation must be added to the email size
> for a true comparison, I am regularly seeing differences of over 5000%
> between the two. To my eyes, something is seriously wrong.
You may have mentioned this earlier, but how is your mailet calculating
the size of the message?
--
Serge Knystautas
President
Lokitech >> software . strategy . design >> http://www.lokitech.com
p. 301.656.5501
e. sergek@lokitech.com
---------------------------------------------------------------------
To unsubscribe, e-mail: server-user-unsubscribe@james.apache.org
For additional commands, e-mail: server-user-help@james.apache.org