Tests (was RE: cvs commit: jakarta-james/src/java/org/apache/james/testing BaseTest.java Main.java POP3Test.java TestMethod.java runtest.bat runtest.sh testconf.xml)

[Danny Angus <da...@apache.org>]

> I was thinking a short sanity test
> suite, a longer regression test suite and a set of tests that mimick
> specific problems like memory leak or crash would be good.

I'm aiming to develop EndToEnd into a pre-release functionality testsuite, running a series of transactions to confirm that the basic functionality of a build is working, executing this test as part of the "build everything" task will ensure that releases are only made when James is working.

> 
> In the past there was talk about creating templates and running those
> templates as a record and replay mechanism. I was wondering what everyone
> thought of that.

I'd like to see a mechanism that'd allow us to create emails as plain text files, or complete SMTP conversations, so that we could quickly create test cases to check handling of specific protocol related issues, for instance some of the less common mail address syntax like quoted local parts, and mail types like read-receipts.


--
To unsubscribe, e-mail:   <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>

Re: Tests (was RE: cvs commit: jakarta-james/src/java/org/apache/james/testing BaseTest.java Main.java POP3Test.java TestMethod.java runtest.bat runtest.sh testconf.xml)

[Harmeet Bedi <ha...@kodemuse.com>]

From: "Serge Knystautas" <se...@lokitech.com>
> I'm sure with some
> analysis of real server logs we could build standard smtp, pop3, imap,
> nntp, etc... traffic scripts.

Emacs has a protocol debug flag that should help collect complete protocol
traces. I was output from there to feed templates

----- Original Message -----
From: "Danny Angus" <da...@apache.org>
I'd like to see a mechanism that'd allow us to create emails as plain text
files, or complete SMTP conversations, so that we could quickly create test
cases to check handling of specific protocol related issues, for instance
some of the less common mail address syntax like quoted local parts, and
mail types like read-receipts.



You can do something like this now by going to tests/src/conf directory and
doing
$ java org.apache.james.testing.ProtocolSimulator localhost 25
smtp.protocolsession


Harmeet


--
To unsubscribe, e-mail:   <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>

Re: Tests (was RE: cvs commit: jakarta-james/src/java/org/apache/james/testing BaseTest.java Main.java POP3Test.java TestMethod.java runtest.bat runtest.sh testconf.xml)

[Harmeet Bedi <ha...@kodemuse.com>]

Added a ProtocolSimulator. Here is a description

 * Simulates protocol sessions. Connects to a protocol server. Test
 * reads template and sends data or reads data from server and
 * validates against given template file.
 *
 * Template rules are:
 *   - All lines starting with '#' are regarded as comments.
 *   - All template lines start with 'C: ' or 'S: '
 *   - Each template line is treated as a seprate unit.
 *   - 'C: ' indicates client side communication that is communication
 *     from this test to remote server.
 *   - 'S: ' indicates server response.
 *   - Client side communication is obtained from template file and is
 *     sent as is to the server.
 *   - Expected Server side responses are read from template. Expected
 *     Server side response is matched against actual server response
 *     using perl regular expressions.
 *
 * Example POP3 prototol script to test for authentication:
 *   # note \ before '+'. '+' needs to be escaped
 *   S: \+OK.*
 *   C: USER test
 *   S: \+OK.*
 *   C: PASS invalidpwd
 *   S: -ERR.*
 *   C: USER test
 *   S: \+OK.*
 *   C: PASS test
 *   S: \+OK.*


Usage is
java org.apache.tools.testing.ProtocolSimulator <host> <port> <template>

Please take a look at it. I think it can be used to test protocol
interactions effectively. Thought a bit about general template engines, but
this seemed to provide max gain, with reasonable effort.

Comments....
Harmeet


--
To unsubscribe, e-mail:   <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>