Blocking defect for JAMES v2.3

["Noel J. Bergman" <no...@devtech.com>]

Vincenzo has found what appears to be a blocker.  He posted e-mail to
Stefano and myself, the gist of which is:

"A mailet [like ClamAVScan or SetMimeHeader] sets a header and issues a
saveChanges(). Before and after such mailet the message is ok. But as soon
it goes to *another processor* the message get truncated and corrupted."

Needless to say, until this is resolved, I'm not going to post a new beta.

	--- Noel


---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org

RE: Testing for production [was: Re: Blocking defect for JAMES v2.3]

["Noel J. Bergman" <no...@devtech.com>]

Bernd Fondermann wrote:

> > Bernd, we should have some round-trip testing that would detect
> > this sort of problem.  :-)

> What Postage today is already helpful at is building a "clean room"
> environment. All mail is kept within. This is good for debugging or
> staging to production.

I know. :-)

> It's not satisfying to see debugging going on on production systems.

No kidding!  :-)

> What Postage does not support is provide means to inspect its received
> emails. I take this as a todo. We would probably want to check headers,
> sizes etc.

Exactly.  That was the point of my aside to you.  And see my reply to Norman
for another thought (re: scripting).

	--- Noel


---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org

Testing for production [was: Re: Blocking defect for JAMES v2.3]

[Bernd Fondermann <bf...@brainlounge.de>]

Noel J. Bergman wrote:
> OK, here is a scenario to reproduce.  In the root processor, add:
> 
>   LogMessage
>   ToRepository (pre-header)
>   SetMimeHeader
>   LogMessage
>   ToRepository (post-header)
>   ToProcessor (test)
> 
> Needless to say, all of the ToRepository mailets should have passThrough
> set.  And in the "test" processor, add:
> 
>   LogMessage
>   ToRepository (post-processor)
> 
> You could also instrument AvalonMailRepository.store to add
> mc.getMessage().writeTo(System.out) for debugging.  You will see that the
> message is fine until we write to the spool, at which point it is corrupt.
> 
> Still looking, but will resume in the morning after checking e-mail.
> 
> Bernd, we should have some round-trip testing that would detect this sort of
> problem.  :-)

What Postage today is already helpful at is building a "clean room" 
environment. All mail is kept within. This is good for debugging or 
staging to production.

It's not satisfying to see debugging going on on production systems.

What Postage does not support is provide means to inspect its received 
emails. I take this as a todo. We would probably want to check headers, 
sizes etc.
Until we have that I'd recommend to attach the debugger of your choice 
to James (or Postage) to see what's happening.

   Bernd

---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org

Re: Testing for production [was: Re: Blocking defect for JAMES v2.3]

[Bernd Fondermann <bf...@brainlounge.de>]

Norman Maurer wrote:
> Am Sonntag, den 09.07.2006, 12:25 +0200 schrieb Bernd Fondermann:
> 
>>Noel J. Bergman wrote:
>>>Bernd, we should have some round-trip testing that would detect this sort of
>>>problem.  :-)
>>
>>What Postage today is already helpful at is building a "clean room" 
>>environment. All mail is kept within. This is good for debugging or 
>>staging to production.
>>
>>It's not satisfying to see debugging going on on production systems.
>>
>>What Postage does not support is provide means to inspect its received 
>>emails. I take this as a todo. We would probably want to check headers, 
>>sizes etc.
>>Until we have that I'd recommend to attach the debugger of your choice 
>>to James (or Postage) to see what's happening.
>>
>>   Bernd
> 
> 
> IMHO such a "debugging" should be better done with JUNIT tests...
> Postage is great for "performance tests" etc.. 

Agreed, a unit test would be the best and most lightweight tool to 
reproduce and resolve this issue and make sure it does not appear again.

A unit test only focusses one specific class (or small group of classes).

If you want to do end-to-end tests for actually testing your 
fully-fledged production server you can no longer rely on your unit tests.

 From my experience, the best way to work with a new release would be:
1. Clone production system into test/staging system
2. Change it as slightly as possible into a 'clean-room'.
3. Deploy new James into test system
4. Generate production-like load
5. Run tests
6. no bug: deploy to production
7. Detect and narrow bug
8. Write unit test reproducing the bug
9. Resolve, test, build. goto 3.

   Bernd

---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org

Re: Testing for production [was: Re: Blocking defect for JAMESv2.3]

[Bernd Fondermann <bf...@brainlounge.de>]

Noel J. Bergman wrote:
> 
> Perhaps adding BSF (or JSR-223) support to Postage would be a good thing.
> Postage providing core capabilities, and allowing scripting for tests.

OK, good idea. I'd like to have standard tests, like checking headers, 
built into Postage directly and provide scripting for more complicated 
evaluations.

   Bernd

---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org

RE: Testing for production [was: Re: Blocking defect for JAMESv2.3]

["Noel J. Bergman" <no...@devtech.com>]

Norman Maurer wrote:

> IMHO such a "debugging" should be better done with JUNIT tests...
> Postage is great for "performance tests" etc..

JUNIT testing is not the same as an external test harness.  We should be
able to drive a test from SMTP connection through to POP3 or IMAP.  So
Postage could send a known message and test the results that it receives,
including testing for any expected changes (e.g., use of any
SetMimeHeaders).

Perhaps adding BSF (or JSR-223) support to Postage would be a good thing.
Postage providing core capabilities, and allowing scripting for tests.

	--- Noel


---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org

Re: Testing for production [was: Re: Blocking defect for JAMES v2.3]

[Norman Maurer <nm...@byteaction.de>]

Am Sonntag, den 09.07.2006, 12:25 +0200 schrieb Bernd Fondermann:
> Noel J. Bergman wrote:
> > OK, here is a scenario to reproduce.  In the root processor, add:
> > 
> >   LogMessage
> >   ToRepository (pre-header)
> >   SetMimeHeader
> >   LogMessage
> >   ToRepository (post-header)
> >   ToProcessor (test)
> > 
> > Needless to say, all of the ToRepository mailets should have passThrough
> > set.  And in the "test" processor, add:
> > 
> >   LogMessage
> >   ToRepository (post-processor)
> > 
> > You could also instrument AvalonMailRepository.store to add
> > mc.getMessage().writeTo(System.out) for debugging.  You will see that the
> > message is fine until we write to the spool, at which point it is corrupt.
> > 
> > Still looking, but will resume in the morning after checking e-mail.
> > 
> > Bernd, we should have some round-trip testing that would detect this sort of
> > problem.  :-)
> 
> What Postage today is already helpful at is building a "clean room" 
> environment. All mail is kept within. This is good for debugging or 
> staging to production.
> 
> It's not satisfying to see debugging going on on production systems.
> 
> What Postage does not support is provide means to inspect its received 
> emails. I take this as a todo. We would probably want to check headers, 
> sizes etc.
> Until we have that I'd recommend to attach the debugger of your choice 
> to James (or Postage) to see what's happening.
> 
>    Bernd

IMHO such a "debugging" should be better done with JUNIT tests...
Postage is great for "performance tests" etc.. 

bye
Norman

RE: Blocking defect for JAMES v2.3

["Noel J. Bergman" <no...@devtech.com>]

Working on it ... looks like we corrupt the message body in
AvalonMailRepository when we call sr.put(key).  Side-effect of not loading
the message into memory, I suspect.

	--- Noel


---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org

RE: Blocking defect for JAMES v2.3

["Noel J. Bergman" <no...@devtech.com>]

OK, here is a scenario to reproduce.  In the root processor, add:

  LogMessage
  ToRepository (pre-header)
  SetMimeHeader
  LogMessage
  ToRepository (post-header)
  ToProcessor (test)

Needless to say, all of the ToRepository mailets should have passThrough
set.  And in the "test" processor, add:

  LogMessage
  ToRepository (post-processor)

You could also instrument AvalonMailRepository.store to add
mc.getMessage().writeTo(System.out) for debugging.  You will see that the
message is fine until we write to the spool, at which point it is corrupt.

Still looking, but will resume in the morning after checking e-mail.

Bernd, we should have some round-trip testing that would detect this sort of
problem.  :-)

	--- Noel


---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org