You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cocoon.apache.org by Upayavira <uv...@upaya.co.uk> on 2003/11/01 18:21:47 UTC
[RT] Unit testing and CocoonUnit
Dear All,
After reading of Steve K's efforts on this list, and reading "Extreme
Programming Java Cookbook", I would like to propose a Cocoon pipeline
unit testing framework, based arount HTTPUnit, XMLUnit and the CocoonBean.
HTTPUnit
-------
HTTPUnit provides a user agent for engaging in 'conversations' with a
web server, and allows you to test the responses at any stage within
this conversation. E.g. is there a form called 'cart' on the HTML page;
is an error message 'Email address required' displayed within the page.
HTTPUnit, to my mind, provides some valuable functionality, but given
the nature of the web, it is primarily aimed at functional testing
rather than unit testing. It tests whether the end result of a process
is correct, but not what happened to get there.
XMLUnit
-------
XMLUnit extends JUnit to give XML aware assertions, which can compare
such things as "<node/>" and "<node></node>" and correctly identify
these as being the same.
CocoonUnit
----------
CocoonUnit will be based upon, and if possible extend, HTTPUnit. A 'web
conversation' will take place in the same way as it would with HTTPUnit.
The only difference is that you would be able to get at the XML from
each stage of a pipeline with response.getPipelineXML(pipelinePos); .
Then, using XMLUnit, you can test to ensure that the XML at any relevant
stage in the pipeline is as you would want.
Example
-------
public class MyXMLTestCase extends XMLTestCase {
public MyXMLTestCase(String name) {
super(name);
}
public void testForEquality() throws Exception {
CocoonConversation cocoonConversation = new CocoonConversation();
CocoonRequest request = new
GetMethodCocoonRequest("/samples/hello-world/hello.html");
CocoonResponse response = webConversation.getResponse(request);
String generatorXML = response.getPipelineXML(0);
String transformedXML = response.getPipelineXML(1);
assertXpathExists("//para[contains(.,'Cocoon')]", generatorXML);
assertXpathNotExists("//para[contains(.,'Active Server Pages')]",
generatorXML);
// further tests on the transformedXML
}
}
This class would then be executed as any normal JUnit test case, e.g.
direct from your IDE, or as a part of your build process via an Ant
build script.
Implementation
--------------
CocoonUnit would use the CocoonBean to pass requests to Cocoon. The bean
would prepare an Environment (maybe even an HTTPEnvironment), and would
place an empty ArrayList into the object model with a key of UNIT_TEST_XML.
Then, in AbstractProcessingPipeline, there are two process() methods,
each of which call connectPipeline(). Immediately before these calls to
connectPipeline, a test would be done for the UNIT_TEST_XML object in
the object model. If it is present, the transformer ArrayLists (all four
of them), would be modified. At the beginning, and then between each
transformer, a DOMGatheringTransformer will be inserted. This
DOMGatheringTransformer simply builds a DOM object and adds it to the
UNIT_TEST_XML ArrayList stored in the object model. and then forwards
its SAX events unmodified to the next stage in the pipeline.
Once a resource has been generated, the CocoonBean can extract this
UNIT_TEST_XML ArrayList and pass it back to the CocoonResponse, so that
its XML DOMs can be available via response.getPipelineXML();
This so far seems relatively straightforward. The most complicated part
to implement, as I see it, will be extending HTTPUnit to interface with
Cocoon instead of its own methods for getting web resources. I have only
had a cursory look at HTTPUnit, but it does seem to have a reasonably
active mailing list that I hope could help me. Should this part prove to
be too complex, I would write my own (limited) version of HTTPUnit. It
seems that you can either use HTTP to get pages, or use its own
'simulated servlet container' to run servlets without a real container.
I could potentially extend or amend either of these to use the CocoonBean.
The CocoonBean may need some extending to enable it to create one Cocoon
instance at the beginning of the testing process, and to then share that
instance across all subsequent tests (whether simultaneous or
sequential). Otherwise, each test would need to initialise and dispose
the CocoonBeanm which would be a waste of resources.
Extension Possibilities
-----------------------
Once the above works, it would not be hard to allow this method to test
specific sections of a pipeline, by supplying the input, as well as
testing the output:
CocoonConversation cocoonConversation = new CocoonConversation();
CocoonRequest request = new
GetMethodCocoonRequest("/samples/hello-world/hello.html");
request.setPipelineXML(1, "<xml>.....</xml>");
CocoonResponse response = webConversation.getResponse(request);
String transformedXML = response.getPipelineXML(2);
assertXpathExists("//para[contains(.,'Cocoon')]", transformedXML);
The above sets the input to the second stage of the pipeline, which the
DOMGatheringTransformer (now a UnitTestTransformer) uses instead of the
incoming SAX stream.
Conclusion
----------
Unit testing helps to provide reliable code. Unit testing in a Web
environment is not always straightforward as your 'units' aren't always
clearly defined. With Cocoon, however, stages in a pipeline can be
considered to be 'units', and therefore could be tested independently of
each other. Adding a CocoonUnit to the Cocoon codebase would further
extend Cocoon's ability to be used in substantial web environments where
automated testing is crucial to success.
What do you all think?
Upayavira
Re: [RT] Unit testing and CocoonUnit
Posted by Upayavira <uv...@upaya.co.uk>.
David Crossley wrote:
>Upayavira wrote:
>
>
>>Dear All,
>>
>>After reading of Steve K's efforts on this list, and reading "Extreme
>>Programming Java Cookbook", I would like to propose a Cocoon pipeline
>>unit testing framework, based arount HTTPUnit, XMLUnit and the CocoonBean.
>>
>>
><snip/>
>
>
>>What do you all think?
>>
>>
>
>That you should go away more often and come back with such
>wonderful ideas.
>
>
Actually, I wrote it before I went, and had to exercise great self
control to not send it until I got back. But I agree with the sentiment
that I should go away more often!
Regards, Upayavira
Re: [RT] Unit testing and CocoonUnit
Posted by David Crossley <cr...@indexgeo.com.au>.
Upayavira wrote:
> Dear All,
>
> After reading of Steve K's efforts on this list, and reading "Extreme
> Programming Java Cookbook", I would like to propose a Cocoon pipeline
> unit testing framework, based arount HTTPUnit, XMLUnit and the CocoonBean.
<snip/>
> What do you all think?
That you should go away more often and come back with such
wonderful ideas.
--David
Re: [RT] Unit testing and CocoonUnit
Posted by Upayavira <uv...@upaya.co.uk>.
Bertrand Delacretaz wrote:
> Le Vendredi, 7 nov 2003, à 08:17 Europe/Zurich, Upayavira a écrit :
>
>> ...3) The remaining thing I dislike is the need to have some code in
>> the sitemap to accept the requests from CocoonUnit)
>>
>> Thinking wildly about point 3, maybe I could write a unit test
>> servlet and set it up in web.xml to answer requests to /cocoon-unit.
>> Then you wouldn't need code in your sitemap. Hmm....
>
>
> Why not use sitemap definitions to let CocoonUnitClient (HttpUnit
> extension) talk to CocoonUnitServer (Cocoon Generator)?
>
> <map:match pattern="/CocoonUnitControl/*/*">
> <map:generate type="cocoon-unit">
> <map:parameter name="test-context-id" value="{1}"/>
> <map:parameter name="operation" value="{2}"/>
> </map:generate>
> <map:serialize type="xml"/>
> </map:match>
>
> The Generator would then take care of all interactions between the
> client-side CocoonUnit classes and the server-side instrumentation
> stuff: creating test contexts, retrieving pipeline stages, etc, in a
> RESTish way.
>
> -Bertrand
Yup, that's what I had in mind. But I don't like having to add something
to the sitemap for testing purposes.
At least I can make the generator return <testing-not-available/> if the
cocoon-unit-enable is set to off.
Regards, Upayavira
Re: [RT] Unit testing and CocoonUnit
Posted by Bertrand Delacretaz <bd...@apache.org>.
Le Vendredi, 7 nov 2003, à 08:17 Europe/Zurich, Upayavira a écrit :
> ...3) The remaining thing I dislike is the need to have some code in
> the sitemap to accept the requests from CocoonUnit)
>
> Thinking wildly about point 3, maybe I could write a unit test servlet
> and set it up in web.xml to answer requests to /cocoon-unit. Then you
> wouldn't need code in your sitemap. Hmm....
Why not use sitemap definitions to let CocoonUnitClient (HttpUnit
extension) talk to CocoonUnitServer (Cocoon Generator)?
<map:match pattern="/CocoonUnitControl/*/*">
<map:generate type="cocoon-unit">
<map:parameter name="test-context-id" value="{1}"/>
<map:parameter name="operation" value="{2}"/>
</map:generate>
<map:serialize type="xml"/>
</map:match>
The Generator would then take care of all interactions between the
client-side CocoonUnit classes and the server-side instrumentation
stuff: creating test contexts, retrieving pipeline stages, etc, in a
RESTish way.
-Bertrand
Re: [RT] Unit testing and CocoonUnit
Posted by Upayavira <uv...@upaya.co.uk>.
Bertrand,
Thanks for this response. I like it, and it seems to remove almost all
of the things I dislike about this approach.
So, my comments:
1) The idea of passing an ID with the webRequest is brilliant. It gets
around a major disadvantage. I would send the ID as a request header
rather than as a request parameter, though.
2) I agree that the 'cocoon-unit-enable' parameter should be 'off' by
default. That is what I was proposing, but your name is better.
3) The remaining thing I dislike is the need to have some code in the
sitemap to accept the requests from CocoonUnit)
Thinking wildly about point 3, maybe I could write a unit test servlet
and set it up in web.xml to answer requests to /cocoon-unit. Then you
wouldn't need code in your sitemap. Hmm.
Thanks for this. I think I'm now close to ready to attempt implementing it.
Regards, Upayavira
Bertrand Delacretaz wrote:
> Hi Upayavira,
>
> I like the idea of reusing HTTPUnit as much as possible - here's a
> slightly different proposal which could allow multiple simultaneous
> unit tests.
>
>> ...
>> CocoonUnit
>> ------------
>> public class MyXMLTestCase extends XMLTestCase {
>> public MyXMLTestCase(String name) {
>> super(name);
>> }
>> public void testForEquality() throws Exception {
>> CocoonUnit cocoon = new
>> CocoonUnit("http://localhost:8080/unit-manager");
>
>
> Now putting this the other way round so that the WebRequest can pass a
> test context ID, and using a setup object instead of multiple option
> calls on CocoonUnit:
>
> CocoonUnitContext ctx = new CocoonUnitContext();
> ctx.setCollectStageXML(1);
> ctx.setCollectStageXML(2);
>
> // here the cocoon server allocates an internal test context,
> returns its ID
> final String testContextId = cocoon.prepareTest(ctx);
>
>> WebConversation webConversation = new WebConversation();
>> WebRequest request = new
>> GetMethodWebRequest("/samples/hello-world/hello.html");
>
>
> // test ID must be passed to server, can be reused several times
> // for stress testing, as long as only one request with this
> particular ID is
> // active a a given time
> request.setParameter("cocoon-unit-id",testContextId);
>
>> WebResponse response = webConversation.getResponse(request);
>> ...
>
>
> etc...
>
>> ...
>> Cocoon Servlet Changes
>> ---------------------
>> The cocoon servlet is extended with a single extra init parameter
>> "test" which is set to either 'on' or 'off'. (This could equally be
>> in cocoon.xconf).
>
>
> IMO, for security reasons, this must be accepted only if a
> configurable "cocoon-unit-enable" parameter is set to on, and it must
> be set to off by default.
>
>> When the cocoon.informServer() method is called, an HTTP post is sent
>> to the server, encapsulating details of the stages for which XML must
>> be gathered....
>
>
> In the above scenario, this must return a test ID string.
>
>> ...When the webConversation.getResponse() is called, the server, if
>> the test input parameter is set to on, looks to see if this object is
>> present in the persistent store....
>
>
> This would then use the test ID string.
>
>> ...The only downside of this as I currently see it is that only one
>> person must be testing a site at any one time, as they cannot share
>> their transient store objects.
>
>
> That's a big downside as IIUC it would prevent one from stress-testing
> a pipeline using multiple client threads or hosts.
>
> But of course, if allowing multiple simultaneous unit tests
> complicates your implementation too much, feel free to ignore this!
>
> -Bertrand
>
>
Re: [RT] Unit testing and CocoonUnit
Posted by Bertrand Delacretaz <bd...@apache.org>.
Hi Upayavira,
I like the idea of reusing HTTPUnit as much as possible - here's a
slightly different proposal which could allow multiple simultaneous
unit tests.
> ...
> CocoonUnit
> ------------
> public class MyXMLTestCase extends XMLTestCase {
> public MyXMLTestCase(String name) {
> super(name);
> }
> public void testForEquality() throws Exception {
> CocoonUnit cocoon = new
> CocoonUnit("http://localhost:8080/unit-manager");
Now putting this the other way round so that the WebRequest can pass a
test context ID, and using a setup object instead of multiple option
calls on CocoonUnit:
CocoonUnitContext ctx = new CocoonUnitContext();
ctx.setCollectStageXML(1);
ctx.setCollectStageXML(2);
// here the cocoon server allocates an internal test context,
returns its ID
final String testContextId = cocoon.prepareTest(ctx);
> WebConversation webConversation = new WebConversation();
> WebRequest request = new
> GetMethodWebRequest("/samples/hello-world/hello.html");
// test ID must be passed to server, can be reused several times
// for stress testing, as long as only one request with this
particular ID is
// active a a given time
request.setParameter("cocoon-unit-id",testContextId);
> WebResponse response = webConversation.getResponse(request);
> ...
etc...
> ...
> Cocoon Servlet Changes
> ---------------------
> The cocoon servlet is extended with a single extra init parameter
> "test" which is set to either 'on' or 'off'. (This could equally be in
> cocoon.xconf).
IMO, for security reasons, this must be accepted only if a configurable
"cocoon-unit-enable" parameter is set to on, and it must be set to off
by default.
> When the cocoon.informServer() method is called, an HTTP post is sent
> to the server, encapsulating details of the stages for which XML must
> be gathered....
In the above scenario, this must return a test ID string.
> ...When the webConversation.getResponse() is called, the server, if
> the test input parameter is set to on, looks to see if this object is
> present in the persistent store....
This would then use the test ID string.
> ...The only downside of this as I currently see it is that only one
> person must be testing a site at any one time, as they cannot share
> their transient store objects.
That's a big downside as IIUC it would prevent one from stress-testing
a pipeline using multiple client threads or hosts.
But of course, if allowing multiple simultaneous unit tests complicates
your implementation too much, feel free to ignore this!
-Bertrand
Re: [RT] Unit testing and CocoonUnit
Posted by Upayavira <uv...@upaya.co.uk>.
Upayavira wrote:
> Giacomo Pati wrote:
>
>> Upayavira wrote:
>>
>>> Steve K wrote:
>>>
>>>> Upayavira --
>>>> - Jakarta Cactus (http://jakarta.apache.org/cactus/) -- This takes
>>>> a similar approach by setting up a mock servlet environment to test
>>>> your servlets in. However, it seemed way to big and complicated to
>>>> do what I thought would be very simple. Also, I think there is a
>>>> lot to be gained by unit testing at the pipeline level as you
>>>> suggest, rather than treating Cocoon like just another servlet.
>>>
>>>
>>> I skimmed it too, but it didn't seem to fit.
>>
>>
>> Have you guys also looked at Anteater http://aft.sf.net?
>>
>> My personal experience with HTTPUnit is that is somehow inflexible
>> and way too programmatic, not very declarative as Anteater is.
>
>
> I've looked into Anteater now, and I'm starting to see how I could
> make a Cocoon unit test tool with it.
> ...
I copied this email to Jeff Turner (who is part responsible for
Anteater). He said that it was, in his opinion, pretty much dead, as it
did what it set out to do, but needed more higher level functionality.
He now uses jWebUnit, which extends HttpUnit.
So, how does this proposal read:
CocoonUnit extention to HttpUnit and jWebUnit
-----------------------------------------------
My original proposal was to use the CocoonBean to access Cocoon.
However, hacking HttpUnit and then jWebUnit to use the new access method
would be a real pain. So....
A new CocoonUnit class can be used alongside HttpUnit or jWebUnit.
HttpUnit or jWebUnit continue to make their tests using HTTP to a
servlet container.
CocoonUnit
------------
public class MyXMLTestCase extends XMLTestCase {
public MyXMLTestCase(String name) {
super(name);
}
public void testForEquality() throws Exception {
CocoonUnit cocoon = new
CocoonUnit("http://localhost:8080/unit-manager");
WebConversation webConversation = new WebConversation();
WebRequest request = new
GetMethodWebRequest("/samples/hello-world/hello.html");
cocoon.setCollectStageXML(1);
cocoon.setCollectStageXML(2);
cocoon.informServer();
WebResponse response = webConversation.getResponse(request);
String generatorXML = cocoon.getPipelineXML(1);
String transformedXML = cocoon.getPipelineXML(2);
assertXpathExists("//para[contains(.,'Cocoon')]", generatorXML);
assertXpathNotExists("//para[contains(.,'Active Server Pages')]",
generatorXML);
// further tests on the transformedXML
}
}
In this way, HttpUnit requires no changes - all instructions to the
server are made through this CocoonUnit object.
Cocoon Servlet Changes
---------------------
The cocoon servlet is extended with a single extra init parameter "test"
which is set to either 'on' or 'off'. (This could equally be in
cocoon.xconf).
When the cocoon.informServer() method is called, an HTTP post is sent to
the server, encapsulating details of the stages for which XML must be
gathered. A couple of extra matchers placed in a pipeline will use an
extension of the StreamGenerator to write the contents of this XML into
an object in the transient store.
When the webConversation.getResponse() is called, the server, if the
test input parameter is set to on, looks to see if this object is
present in the persistent store. If it is, it inserts transformers into
the pipeline as required to gather XML. This XML is then stored in the
transient store, and the normal response is sent back to the user.
When the cocoon.getPipelineXML(1) method is called, an HTTP request is
sent to the server, and the server returns the relevant XML out of the
transient store.
The only downside of this as I currently see it is that only one person
must be testing a site at any one time, as they cannot share their
transient store objects.
This is by far the easiest solution to implement that I've come up with
so far. It is not the most elegant internally, but from the user's point
of view should be relatively easy to understand.
What do you think?
Regards,. Upayavira
Re: [RT] Unit testing and CocoonUnit
Posted by Upayavira <uv...@upaya.co.uk>.
Giacomo Pati wrote:
> Upayavira wrote:
>
>> Steve K wrote:
>>
>>> Upayavira --
>>> - Jakarta Cactus (http://jakarta.apache.org/cactus/) -- This takes a
>>> similar approach by setting up a mock servlet environment to test
>>> your servlets in. However, it seemed way to big and complicated to
>>> do what I thought would be very simple. Also, I think there is a
>>> lot to be gained by unit testing at the pipeline level as you
>>> suggest, rather than treating Cocoon like just another servlet.
>>
>> I skimmed it too, but it didn't seem to fit.
>
> Have you guys also looked at Anteater http://aft.sf.net?
>
> My personal experience with HTTPUnit is that is somehow inflexible and
> way too programmatic, not very declarative as Anteater is.
I've looked into Anteater now, and I'm starting to see how I could make
a Cocoon unit test tool with it.
I do prefer the idea of a declarative (as opposed to programmatic)
method for specifying tests, expecially given that a user who has never
coded Java may want to establish tests for their site, and requiring
Java to test a site that does not include any Java code of their own
would be unfortunate.
I think I would extend Anteater to do something like this (extracted
from the calc.xml Anteater test in CocoonCVS):
<target name="calc">
<property name="calc" value="${cocoon}/samples/flow/calc/"/>
<cocoonRequest href="${calc}/"
description="Test the 'calc' JavaScript implementation">
<pipelineMatch stage="1">
<xpath select="page/content/form/@method" value="post"/>
</pipelineMatch>
<pipelineMatch stage="last">
<xpath select="html/body//form/@action" assign="cont1"/>
</pipelineMatch>
</cocoonRequest>
</etc>
</target>
With this, instead of an <httpRequest> you have a <cocoonRequest> which
passes the request direct to Cocoon. Then, instead of a <match> you have
a <pipelineMatch>, which chooses a particular stage from a pipeline.
Stages can be reffered to by number, or with the keywords "last" or
"penultimate".
With this, you can use the full range of test that are available to
Anteater: xpath, regexp, contentEquals, etc.
With code like the following:
<cocoonRequest href="database.html">
<pipelineStage stage="1" filename="esql-output.xml"/>
<pipelineMatch stage="2">
<xpath select="....."/>
...
</cocoonRequest>
The user can state that they want the output of stage 1 to be replaced
with the contents of the file esql-output.xml. Imagine you're testing a
site that is driven using an ESQL XSP page. To test this, you want a
static XML, rather than having to actually access the database. Here,
whilst the pipeline will still hit the database and the XSP will still
generate XML, this generated XML will be ignored and the contents of
eqsl-output.xml will be passed on to the next stage of the pipeline,
creating a managable way to test a single stage in a pipeline.
The remainining unanswered question is how I could initialise a Cocoon
instance (probably via a modified CocoonBean) at test startup, and then
have that Cocoon available to all subsequent cocoonRequest objects.
Anyone got any ideas about this?
Where the code for this goes, I'm not sure either. Is it part of
Anteater, or a part of Cocoon? Or neither?
What do you all think? Is an Anteater approach better than a
programmatic HTTPUnit one?
Regards, Upayavira
Re: [RT] Unit testing and CocoonUnit
Posted by Upayavira <uv...@upaya.co.uk>.
Giacomo Pati wrote:
>
>
> Upayavira wrote:
>
>> Steve K wrote:
>>
>>> Upayavira --
>>> - Jakarta Cactus (http://jakarta.apache.org/cactus/) -- This takes a
>>> similar approach by setting up a mock servlet environment to test
>>> your servlets in. However, it seemed way to big and complicated to
>>> do what I thought would be very simple. Also, I think there is a
>>> lot to be gained by unit testing at the pipeline level as you
>>> suggest, rather than treating Cocoon like just another servlet.
>>
>>
>>
>> I skimmed it too, but it didn't seem to fit.
>
>
> Have you guys also looked at Anteater http://aft.sf.net?
>
> My personal experience with HTTPUnit is that is somehow inflexible and
> way too programmatic, not very declarative as Anteater is.
Interesting.
I had thought about a more declarative approach, and there are already
some Anteater tests in the Cocoon codebase.
Looking at it, I suspect we could get away with extending/modifying the
HTTPRequest and replacing the HTTPClient with a CocoonClient. I can see
how it could be done. Worst case, I would just replace HTTPRequest with
CocoonRequest and HTTPClient with CocoonClient.
I haven't yet worked out how to allow xpaths on intermediate pipeline
XMLs - I haven't found the docs on Anteater that easy to understand. The
best thing I found were the anteater tests within Cocoon
(src/test/anteater).
So it is certainly possible, and I'll look into it more.
Regards, Upayavira
Re: [RT] Unit testing and CocoonUnit
Posted by Giacomo Pati <gi...@apache.org>.
Upayavira wrote:
> Steve K wrote:
>
>> Upayavira --
>> - Jakarta Cactus (http://jakarta.apache.org/cactus/) -- This takes a
>> similar approach by setting up a mock servlet environment to test your
>> servlets in. However, it seemed way to big and complicated to do what
>> I thought would be very simple. Also, I think there is a lot to be
>> gained by unit testing at the pipeline level as you suggest, rather
>> than treating Cocoon like just another servlet.
>
>
> I skimmed it too, but it didn't seem to fit.
Have you guys also looked at Anteater http://aft.sf.net?
My personal experience with HTTPUnit is that is somehow inflexible and
way too programmatic, not very declarative as Anteater is.
--
Giacomo Pati
Otego AG, Switzerland - http://www.otego.com
Orixo, the XML business alliance - http://www.orixo.com
Re: [RT] Unit testing and CocoonUnit
Posted by Upayavira <uv...@upaya.co.uk>.
Steve K wrote:
> Upayavira --
>
> This all looks great. I've also been hacking at this problem and have
> a working solution that does not involve any open pipeline surgery or
> triple-pipeline-bypasses. It is a bit of a hack and I do believe your
> solution is much more appropriate in the long term, however, maybe
> what I've done will give you some more ideas.
I suspect that your and my ideas potentially complement each other.
Let's see...
> Originally I was using the CocoonBean to test pipelines, but I had
> some new pipelines that needed access to HTTP headers and cookies --
> and I could not find a way to pass them in using the interfaces
> provided by the bean. So I did the following:
It would probably be my intention to extend the bean to be able to
handle an HTTPEnvironment of some sort, which would make this possible.
> - I extended the ExcaliburTestCase to create a CocoonTestCase. In my
> new class I copied the initalize() and getClassPath() methods from the
> CocoonWrapper and modified it all enough so on JUnit setUp() I could
> instantiate an instance of Cocoon and stick it in a member variable.
> The reason I extended ExcaliburTestCase is that I have several Avalon
> components that my application uses that I also use to set up the test
> cases.
Effectively you wrote your own tool to invoke Cocoon, which is fair
enough. I'd like to see this sharing as much of the existing code as
possible, though.
> - To send a request to Cocoon, I create an HttpEnvironment object
> using mock servlet objects -- specifically the objects from
> http://strutstestcase.sourceforge.net/.
Useful to know about.
> When I get the request back, I parse it into XML and use the XMLUnit
> style asserts on it.
Good that we share on something!
> Here is what the code looks like:
>
> ----
>
> // set up the mock objects
>
> ServletContextSimulator sctx = new ServletContextSimulator();
> HttpContext hc = new HttpContext((ServletContext) sctx);
> HttpServletRequestSimulator req = new HttpServletRequestSimulator(sctx);
> HttpServletResponseSimulator res = new HttpServletResponseSimulator();
>
> // set up the HTTP request
>
> req.setHeader("Accept", text/xml,application/xml");
> req.setHeader("Accept-Language", "en");
> req.setParameterValue("cocoon-view", new String [] {"content"});
>
> // output goes to baos
>
> ByteArrayOutputStream baos = new ByteArrayOutputStream();
> res.setOutputStream(out);
>
> // set up HttpEnvironment to get the uri "foo/bar"
>
> HttpEnvironment env = new HttpEnvironment(
> "foo/bar",
> "path/to/webapp",
> (HttpServletRequest) req,
> (HttpServletResponse) res,
> (ServletContext) sctx,
> hc,
> "",
> "");
> env.enableLogging(new LogKitLogger(getLogger()));
>
> // call process() on the cocoon member variable
>
> boolean rv = cocoon.process(env);
> baos.flush();
>
> // parse the response into XML
>
> SAXParser parser = (SAXParser) this.manager.lookup(SAXParser.ROLE);
> DOMBuilder builder = new DOMBuilder();
> parser.parse(new InputSource(
> new ByteArrayInputStream(baos.toByteArray())),
> new WhitespaceFilter(builder),
> builder);
> Document xml = builder.getDocument();
> this.manager.release((Component) parser);
>
> // test the xml
>
> assertXPathTrue(xml, "/blah");
>
> ----
Very interesting.
The thing I liked about HTTPUnit was the relatively high level way in
which a testing script was described. I would like to see the testing
code being somewhat higher level than the code you've got. For example,
the parsing of XML, and creation of an environment done behind the
scenes. But I suspect that what I'm talking about is merely a wrapper
around what you've done already.
To extend your solution to do the pipeline inspection I was talking
about, you'd need to create an HTTPTestEnvironment, which puts an empty
ArrayList into the ObjectModel (not hard). Then the pipeline code would
pick up on this and insert the UnitTestTransformer in between each stage
of the pipeline. Once you've done cocoon.process(env), you can get at
the DOM objects stored in this ArrayList via the environment's object
model. Make sense?
> This has worked pretty well so far.
Great.
> I think extending the ExaliburTestCase and using mock servlet objects
> might be better than extedning HTTPUnit since it gives us more
> flexibility and more access to the raw Coccon request/response.
From my brief reading up on HTTPUnit, it offers two ways to run tests,
using an actuall HTTP request, or by running a servlet within its own
simulated servlet context. My proposed extension to HTTPUnit would be to
add another method - via Cocoon itself. The code to do that would be
much the same as you have it for your ExcaliburTestCase, and would offer
the user access to Cocoon data if they wanted it (but it would be hidden
unless needed).
What I like about HTTPUnit is its rich language for describing web
conversations. And that is just as relevant to Cocoon as it is servlets
or HTTP testing, it is just the access method that is different.
> Some other things I've investigated on the way:
>
> - Jakarta Cactus (http://jakarta.apache.org/cactus/) -- This takes a
> similar approach by setting up a mock servlet environment to test your
> servlets in. However, it seemed way to big and complicated to do what
> I thought would be very simple. Also, I think there is a lot to be
> gained by unit testing at the pipeline level as you suggest, rather
> than treating Cocoon like just another servlet.
I skimmed it too, but it didn't seem to fit.
> - In my search for a mock implementation of the servlet classes, I
> came along MockObjects (http://www.mockobjects.com). They have mock
> objects for much of the Java API, however, their approach didn't
> really fit into what I was trying to do. Basically, you are supposed
> to pre-program the mock objects to expect certain method calls (e.g.
> getHeader should be called with such and such parameters, then
> getSession should be called, etc). Since I just wanted mock objects
> good enough to masquerade as a servlet container and get the output
> from Cocoon, this was not very helpful. However, perhaps a similar
> methodology could be adapted where you could pass in the expected XML
> for each pipeline step -- I think you had this idea in our previous
> exchanges.
I haven't thought about this side of things that much - I guess I would
create my own objects that implement o.a.c.environment.Request/Response,
etc, e.g. HTTPUnitTestEnvironment, rather than using mocks.
> And finally, on a somewhat unrelated subject, one thing that I've
> always wanted Cocoon to do may be possible if support for collecting
> the XML at each pipeline step is added. To aid in debugging, I think
> it would be very helpful to switch on some kind of debug mode, that
> would cause a trace of what pipeline steps where executed and the
> state of the XML at each step to be printed out at the bottom of each
> page you output to the browser. This way it is easy for a developer
> to see the path though the pipelines the request took, as well as a
> snapshot of the XML each step of the way.
I've never looked at it (and probably should), but I think the
logTransformer does that, although it isn't transparent, which is what
I'm proposing.
Thanks for taking the time to respond to my message. I suspect that if
we work together on this one, we could come up with something really neat.
Regards, Upayavira
Re: Profiling Pipeline [was Re: [RT] Unit testing and CocoonUnit]
Posted by Vadim Gritsenko <va...@verizon.net>.
Nicola Ken Barozzi wrote:
> Berin Loritsch wrote:
> ...
>
>>
>>
>> I have a feeling the timings are useless largely because of the
>> granularity
>> of the System.getTimeInMillis() method. On Windows the granularity
>> is 10 ms.
>> If the timing is less than that it registers as 0. Add enough zeros
>> and it
>> will always be less than the total time for the timing.
>
>
> There is a way of getting around this by using JNI and calling the
> correct stuff. If you Google round you will sure find something.
> Probably there is something at www.javaworld.com.
It's JavaPRO [1]. Any takers? :)
Vadim
[1]
http://www.fawcette.com/archives/premier/mgznarch/javapro/2001/08aug01/km0108/km0108-2.asp
Re: Profiling Pipeline [was Re: [RT] Unit testing and CocoonUnit]
Posted by Thor Heinrichs-Wolpert <th...@lunartek.com>.
If you try to profile on a Unix environment based on timings you have
to do multiple passes and get a statistical average to be meaningful.
The timer functionality in the kernel or user space is at such a low
priority that effectively it jumps between values ... that's why when
tuning using tools like sar you have to set a decent sample size. This
is pretty much the case for any current non-realtime OS. For long
running tasks the time is probably OK, for small times ( < 10 ms or
even .1s timings ) on a busy system the times are just erroneous.
Cheers,
Thor HW
On 3-Nov-03, at 9:36 AM, Nicola Ken Barozzi wrote:
> Berin Loritsch wrote:
> ...
>> I have a feeling the timings are useless largely because of the
>> granularity
>> of the System.getTimeInMillis() method. On Windows the granularity
>> is 10 ms.
>> If the timing is less than that it registers as 0. Add enough zeros
>> and it
>> will always be less than the total time for the timing.
>
> There is a way of getting around this by using JNI and calling the
> correct stuff. If you Google round you will sure find something.
> Probably there is something at www.javaworld.com.
>
> --
> Nicola Ken Barozzi nicolaken@apache.org
> - verba volant, scripta manent -
> (discussions get forgotten, just code remains)
> ---------------------------------------------------------------------
>
>
>
Re: Profiling Pipeline [was Re: [RT] Unit testing and CocoonUnit]
Posted by Nicola Ken Barozzi <ni...@apache.org>.
Berin Loritsch wrote:
...
>
>
> I have a feeling the timings are useless largely because of the granularity
> of the System.getTimeInMillis() method. On Windows the granularity is
> 10 ms.
> If the timing is less than that it registers as 0. Add enough zeros and it
> will always be less than the total time for the timing.
There is a way of getting around this by using JNI and calling the
correct stuff. If you Google round you will sure find something.
Probably there is something at www.javaworld.com.
--
Nicola Ken Barozzi nicolaken@apache.org
- verba volant, scripta manent -
(discussions get forgotten, just code remains)
---------------------------------------------------------------------
Re: Profiling Pipeline [was Re: [RT] Unit testing and CocoonUnit]
Posted by Berin Loritsch <bl...@apache.org>.
Antonio Gallardo wrote:
> Stefano Mazzocchi dijo:
>
>>On Saturday, Nov 1, 2003, at 23:29 Europe/Rome, Steve K wrote:
>>
>>
>>>And finally, on a somewhat unrelated subject, one thing that I've
>>>always wanted Cocoon to do may be possible if support for collecting
>>>the XML at each pipeline step is added. To aid in debugging, I think
>>>it would be very helpful to switch on some kind of debug mode, that
>>>would cause a trace of what pipeline steps where executed and the
>>>state of the XML at each step to be printed out at the bottom of each
>>>page you output to the browser. This way it is easy for a developer
>>>to see the path though the pipelines the request took, as well as a
>>>snapshot of the XML each step of the way.
>>
>>This is already there, althought somewhat hidden, check into the
>>"profiler" block.
>>
>>BTW, there is something that always bugged me about the profiler: the
>>time that gives you is almost totally useless, while the exposed view
>>of the pipeline internals is a *great* debugging tool (some people do
>>it with views, but sometimes you don't know where the problem is so you
>>might want to see it all).
>>
>>I propose two changes here:
>>
>>1) rename the "profiling" pipeline into "debug"
>>2) remove the timings (they don't make any sense)
>>3) move the whole thing into core
>
>
>>WTYT?
>
>
> +1 on 1)
>
> +- ? 2) What are the timings? Sorry, I don't understand this point.
I have a feeling the timings are useless largely because of the granularity
of the System.getTimeInMillis() method. On Windows the granularity is 10 ms.
If the timing is less than that it registers as 0. Add enough zeros and it
will always be less than the total time for the timing.
--
"They that give up essential liberty to obtain a little temporary safety
deserve neither liberty nor safety."
- Benjamin Franklin
Re: Profiling Pipeline [was Re: [RT] Unit testing and CocoonUnit]
Posted by Antonio Gallardo <ag...@agsoftware.dnsalias.com>.
Stefano Mazzocchi dijo:
>
> On Saturday, Nov 1, 2003, at 23:29 Europe/Rome, Steve K wrote:
>
>> And finally, on a somewhat unrelated subject, one thing that I've
>> always wanted Cocoon to do may be possible if support for collecting
>> the XML at each pipeline step is added. To aid in debugging, I think
>> it would be very helpful to switch on some kind of debug mode, that
>> would cause a trace of what pipeline steps where executed and the
>> state of the XML at each step to be printed out at the bottom of each
>> page you output to the browser. This way it is easy for a developer
>> to see the path though the pipelines the request took, as well as a
>> snapshot of the XML each step of the way.
>
> This is already there, althought somewhat hidden, check into the
> "profiler" block.
>
> BTW, there is something that always bugged me about the profiler: the
> time that gives you is almost totally useless, while the exposed view
> of the pipeline internals is a *great* debugging tool (some people do
> it with views, but sometimes you don't know where the problem is so you
> might want to see it all).
>
> I propose two changes here:
>
> 1) rename the "profiling" pipeline into "debug"
> 2) remove the timings (they don't make any sense)
> 3) move the whole thing into core
> WTYT?
+1 on 1)
+- ? 2) What are the timings? Sorry, I don't understand this point.
-1 on 3) It is desired to have a very light core. If the core can live
without the profiler please don't include them on every distribution. I
mean development and production.
Best Regards,
Antonio Gallardo
Re: Profiling Pipeline [was Re: [RT] Unit testing and CocoonUnit]
Posted by Antonio Gallardo <ag...@agsoftware.dnsalias.com>.
Stefano Mazzocchi dijo:
>>>> I propose two changes here:
>>>>
>>>> 1) rename the "profiling" pipeline into "debug"
>>>>
>>>
>>> +1
>>>
>>
>> -0. Current name suits me well.
>
> I see, but people that look for debugging the pipeline will not find it
> (since profiling is not the same as debugging).
>
> Maybe "pipe-instrumentation"?
what about:
cocoon-instrumentation
Because if we will include there also the current instrumentation then we
are talking also about components. Not only about pipes.
>>>> 3) move the whole thing into core
>>>>
>>>
>>> -1, the core should only contain necessary components.
>>>
>>
>> +1, I wouldn't mind seeing it in core. It's like instrumentation
>> manager which is also in core.
>
> which I would like to be moved away as well... maybe an
> "instrumentation" block with both? I don't know. WDYT?
+1
Best Regards,
Antonio Gallardo
Re: Profiling Pipeline [was Re: [RT] Unit testing and CocoonUnit]
Posted by Torsten Curdt <tc...@vafer.org>.
>> "instrumentation" block can be a good place for both.
>
>
> The problem is that the instrumentation code is not (easily)
> modularizable as it's intermixed with a bunch of places (in short, is an
> aspect, not an object).
I haven't looked into this yet but... Doesn't instrumentation also work
through java bytecode transformation?
So having a special classloader inside an intrumenation block
that transforms the classes should be enough... IMO
...or what am I here missing?
cheers
--
Torsten
Re: Profiling Pipeline [was Re: [RT] Unit testing and CocoonUnit]
Posted by Stefano Mazzocchi <st...@apache.org>.
On Sunday, Nov 2, 2003, at 20:32 Europe/Rome, Vadim Gritsenko wrote:
> Stefano Mazzocchi wrote:
>
>>>>> 3) move the whole thing into core
>>>>>
>>>>
>>>> -1, the core should only contain necessary components.
>>>
>>>
>>> +1, I wouldn't mind seeing it in core. It's like instrumentation
>>> manager which is also in core.
>>
>>
>> which I would like to be moved away as well... maybe an
>> "instrumentation" block with both? I don't know. WDYT?
>
>
> "instrumentation" block can be a good place for both.
The problem is that the instrumentation code is not (easily)
modularizable as it's intermixed with a bunch of places (in short, is
an aspect, not an object).
This is a much bigger issue that is seems at first sight :-/
--
Stefano.
Re: Profiling Pipeline [was Re: [RT] Unit testing and CocoonUnit]
Posted by Vadim Gritsenko <va...@verizon.net>.
Stefano Mazzocchi wrote:
>>>> 3) move the whole thing into core
>>>>
>>>
>>> -1, the core should only contain necessary components.
>>
>>
>> +1, I wouldn't mind seeing it in core. It's like instrumentation
>> manager which is also in core.
>
>
> which I would like to be moved away as well... maybe an
> "instrumentation" block with both? I don't know. WDYT?
"instrumentation" block can be a good place for both.
Vadim
Re: Profiling Pipeline [was Re: [RT] Unit testing and CocoonUnit]
Posted by Stefano Mazzocchi <st...@apache.org>.
On Monday, Nov 3, 2003, at 11:53 Europe/Rome, Stephan Michels wrote:
>
>
> On Sun, 2 Nov 2003, Stefano Mazzocchi wrote:
>
>>
>> On Sunday, Nov 2, 2003, at 15:57 Europe/Rome, Vadim Gritsenko wrote:
>>
>>> Stephan Michels wrote:
>>>
>>>> On Sun, 2 Nov 2003, Stefano Mazzocchi wrote:
>>>>
>>>>
>>>>> On Saturday, Nov 1, 2003, at 23:29 Europe/Rome, Steve K wrote:
>>>>>
>>>>>
>>>>>> And finally, on a somewhat unrelated subject, one thing that I've
>>>>>> always wanted Cocoon to do may be possible if support for
>>>>>> collecting
>>>>>> the XML at each pipeline step is added. To aid in debugging, I
>>>>>> think
>>>>>> it would be very helpful to switch on some kind of debug mode,
>>>>>> that
>>>>>> would cause a trace of what pipeline steps where executed and the
>>>>>> state of the XML at each step to be printed out at the bottom of
>>>>>> each
>>>>>> page you output to the browser. This way it is easy for a
>>>>>> developer
>>>>>> to see the path though the pipelines the request took, as well as
>>>>>> a
>>>>>> snapshot of the XML each step of the way.
>>>>>>
>>>>> This is already there, althought somewhat hidden, check into the
>>>>> "profiler" block.
>>>>>
>>>>> BTW, there is something that always bugged me about the profiler:
>>>>> the
>>>>> time that gives you is almost totally useless,
>>>>>
>>>
>>> (IIRC) Stephan did a lot of work on how profiler counts time, and now
>>> it's not useless at all.
>>
>> ah, didn't know that! I used it a while ago and it seemed to me that
>> the numbers were totally silly.
>>
>> Stephan, can you explain the changes (or point me to a message that
>> explains it)? thanks.
>
> All I made was to use XMLByteStreamInterpreter/Compiler as a buffer,
> measure each sitemap component separate.
>
> G -> Buffer1
>
> Buffer1 -> T1 -> Buffer2
>
> etc.
>
> The side effect is that you can use these buffers to get intermediate
> document as debug information.
>
> (Buffer(n) -> Buffer(n+1) take in most cases less than a 1ms)
>
> I also measure the time for setup(), which needs in some special cases
> 400 times more time.
Ok, great. Makes perfect sense. Sorry for the noise.
--
Stefano.
Re: Profiling Pipeline [was Re: [RT] Unit testing and CocoonUnit]
Posted by Stephan Michels <st...@apache.org>.
On Sun, 2 Nov 2003, Stefano Mazzocchi wrote:
>
> On Sunday, Nov 2, 2003, at 15:57 Europe/Rome, Vadim Gritsenko wrote:
>
> > Stephan Michels wrote:
> >
> >> On Sun, 2 Nov 2003, Stefano Mazzocchi wrote:
> >>
> >>
> >>> On Saturday, Nov 1, 2003, at 23:29 Europe/Rome, Steve K wrote:
> >>>
> >>>
> >>>> And finally, on a somewhat unrelated subject, one thing that I've
> >>>> always wanted Cocoon to do may be possible if support for collecting
> >>>> the XML at each pipeline step is added. To aid in debugging, I
> >>>> think
> >>>> it would be very helpful to switch on some kind of debug mode, that
> >>>> would cause a trace of what pipeline steps where executed and the
> >>>> state of the XML at each step to be printed out at the bottom of
> >>>> each
> >>>> page you output to the browser. This way it is easy for a developer
> >>>> to see the path though the pipelines the request took, as well as a
> >>>> snapshot of the XML each step of the way.
> >>>>
> >>> This is already there, althought somewhat hidden, check into the
> >>> "profiler" block.
> >>>
> >>> BTW, there is something that always bugged me about the profiler: the
> >>> time that gives you is almost totally useless,
> >>>
> >
> > (IIRC) Stephan did a lot of work on how profiler counts time, and now
> > it's not useless at all.
>
> ah, didn't know that! I used it a while ago and it seemed to me that
> the numbers were totally silly.
>
> Stephan, can you explain the changes (or point me to a message that
> explains it)? thanks.
All I made was to use XMLByteStreamInterpreter/Compiler as a buffer,
measure each sitemap component separate.
G -> Buffer1
Buffer1 -> T1 -> Buffer2
etc.
The side effect is that you can use these buffers to get intermediate
document as debug information.
(Buffer(n) -> Buffer(n+1) take in most cases less than a 1ms)
I also measure the time for setup(), which needs in some special cases
400 times more time.
Stephan.
Re: Profiling Pipeline [was Re: [RT] Unit testing and CocoonUnit]
Posted by Stefano Mazzocchi <st...@apache.org>.
On Sunday, Nov 2, 2003, at 15:57 Europe/Rome, Vadim Gritsenko wrote:
> Stephan Michels wrote:
>
>> On Sun, 2 Nov 2003, Stefano Mazzocchi wrote:
>>
>>
>>> On Saturday, Nov 1, 2003, at 23:29 Europe/Rome, Steve K wrote:
>>>
>>>
>>>> And finally, on a somewhat unrelated subject, one thing that I've
>>>> always wanted Cocoon to do may be possible if support for collecting
>>>> the XML at each pipeline step is added. To aid in debugging, I
>>>> think
>>>> it would be very helpful to switch on some kind of debug mode, that
>>>> would cause a trace of what pipeline steps where executed and the
>>>> state of the XML at each step to be printed out at the bottom of
>>>> each
>>>> page you output to the browser. This way it is easy for a developer
>>>> to see the path though the pipelines the request took, as well as a
>>>> snapshot of the XML each step of the way.
>>>>
>>> This is already there, althought somewhat hidden, check into the
>>> "profiler" block.
>>>
>>> BTW, there is something that always bugged me about the profiler: the
>>> time that gives you is almost totally useless,
>>>
>
> (IIRC) Stephan did a lot of work on how profiler counts time, and now
> it's not useless at all.
ah, didn't know that! I used it a while ago and it seemed to me that
the numbers were totally silly.
Stephan, can you explain the changes (or point me to a message that
explains it)? thanks.
> For example, I recently used profiler with different XSLT engines and
> found out that XSLTC finally became a tad faster than Saxon. Also, I
> found couple of stylesheets which were rewritten as transformers to
> gain some speed improvements.
Very cool. Then I remove my proposal.
>>> while the exposed view
>>> of the pipeline internals is a *great* debugging tool
>>>
>
> Never tried it (yet)! :)
>
>
>>> (some people do
>>> it with views, but sometimes you don't know where the problem is so
>>> you
>>> might want to see it all).
>>>
>>> I propose two changes here:
>>>
>>> 1) rename the "profiling" pipeline into "debug"
>>>
>>
>> +1
>>
>
> -0. Current name suits me well.
I see, but people that look for debugging the pipeline will not find it
(since profiling is not the same as debugging).
Maybe "pipe-instrumentation"?
>>> 2) remove the timings (they don't make any sense)
>>>
>>
>> -1, why do you think the timings are useless?! I done a lot of
>> profiling
>> with it in the past, and found for example the problem with the
>> use-store
>> paramter of the TaxTransformer.
>>
>
> -1, it's useful to me.
I remove my proposal after your comments above.
>>> 3) move the whole thing into core
>>>
>>
>> -1, the core should only contain necessary components.
>>
>
> +1, I wouldn't mind seeing it in core. It's like instrumentation
> manager which is also in core.
which I would like to be moved away as well... maybe an
"instrumentation" block with both? I don't know. WDYT?
--
Stefano.
Re: Profiling Pipeline [was Re: [RT] Unit testing and CocoonUnit]
Posted by Vadim Gritsenko <va...@verizon.net>.
Stephan Michels wrote:
>On Sun, 2 Nov 2003, Stefano Mazzocchi wrote:
>
>
>
>>On Saturday, Nov 1, 2003, at 23:29 Europe/Rome, Steve K wrote:
>>
>>
>>
>>>And finally, on a somewhat unrelated subject, one thing that I've
>>>always wanted Cocoon to do may be possible if support for collecting
>>>the XML at each pipeline step is added. To aid in debugging, I think
>>>it would be very helpful to switch on some kind of debug mode, that
>>>would cause a trace of what pipeline steps where executed and the
>>>state of the XML at each step to be printed out at the bottom of each
>>>page you output to the browser. This way it is easy for a developer
>>>to see the path though the pipelines the request took, as well as a
>>>snapshot of the XML each step of the way.
>>>
>>>
>>This is already there, althought somewhat hidden, check into the
>>"profiler" block.
>>
>>BTW, there is something that always bugged me about the profiler: the
>>time that gives you is almost totally useless,
>>
(IIRC) Stephan did a lot of work on how profiler counts time, and now
it's not useless at all. For example, I recently used profiler with
different XSLT engines and found out that XSLTC finally became a tad
faster than Saxon. Also, I found couple of stylesheets which were
rewritten as transformers to gain some speed improvements.
>>while the exposed view
>>of the pipeline internals is a *great* debugging tool
>>
Never tried it (yet)! :)
>>(some people do
>>it with views, but sometimes you don't know where the problem is so you
>>might want to see it all).
>>
>>I propose two changes here:
>>
>>1) rename the "profiling" pipeline into "debug"
>>
>>
>
>+1
>
>
-0. Current name suits me well.
>>2) remove the timings (they don't make any sense)
>>
>>
>
>-1, why do you think the timings are useless?! I done a lot of profiling
>with it in the past, and found for example the problem with the use-store
>paramter of the TaxTransformer.
>
>
-1, it's useful to me.
>>3) move the whole thing into core
>>
>>
>
>-1, the core should only contain necessary components.
>
>
+1, I wouldn't mind seeing it in core. It's like instrumentation manager
which is also in core.
Vadim
Re: Profiling Pipeline [was Re: [RT] Unit testing and CocoonUnit]
Posted by Stephan Michels <st...@apache.org>.
On Sun, 2 Nov 2003, Stefano Mazzocchi wrote:
>
> On Saturday, Nov 1, 2003, at 23:29 Europe/Rome, Steve K wrote:
>
> > And finally, on a somewhat unrelated subject, one thing that I've
> > always wanted Cocoon to do may be possible if support for collecting
> > the XML at each pipeline step is added. To aid in debugging, I think
> > it would be very helpful to switch on some kind of debug mode, that
> > would cause a trace of what pipeline steps where executed and the
> > state of the XML at each step to be printed out at the bottom of each
> > page you output to the browser. This way it is easy for a developer
> > to see the path though the pipelines the request took, as well as a
> > snapshot of the XML each step of the way.
>
> This is already there, althought somewhat hidden, check into the
> "profiler" block.
>
> BTW, there is something that always bugged me about the profiler: the
> time that gives you is almost totally useless, while the exposed view
> of the pipeline internals is a *great* debugging tool (some people do
> it with views, but sometimes you don't know where the problem is so you
> might want to see it all).
>
> I propose two changes here:
>
> 1) rename the "profiling" pipeline into "debug"
+1
> 2) remove the timings (they don't make any sense)
-1, why do you think the timings are useless?! I done a lot of profiling
with it in the past, and found for example the problem with the use-store
paramter of the TaxTransformer.
> 3) move the whole thing into core
-1, the core should only contain necessary components.
Stephan.
Profiling Pipeline [was Re: [RT] Unit testing and CocoonUnit]
Posted by Stefano Mazzocchi <st...@apache.org>.
On Saturday, Nov 1, 2003, at 23:29 Europe/Rome, Steve K wrote:
> And finally, on a somewhat unrelated subject, one thing that I've
> always wanted Cocoon to do may be possible if support for collecting
> the XML at each pipeline step is added. To aid in debugging, I think
> it would be very helpful to switch on some kind of debug mode, that
> would cause a trace of what pipeline steps where executed and the
> state of the XML at each step to be printed out at the bottom of each
> page you output to the browser. This way it is easy for a developer
> to see the path though the pipelines the request took, as well as a
> snapshot of the XML each step of the way.
This is already there, althought somewhat hidden, check into the
"profiler" block.
BTW, there is something that always bugged me about the profiler: the
time that gives you is almost totally useless, while the exposed view
of the pipeline internals is a *great* debugging tool (some people do
it with views, but sometimes you don't know where the problem is so you
might want to see it all).
I propose two changes here:
1) rename the "profiling" pipeline into "debug"
2) remove the timings (they don't make any sense)
3) move the whole thing into core
WTYT?
--
Stefano.
Re: [RT] Unit testing and CocoonUnit
Posted by Steve K <sh...@mm.st>.
Upayavira --
This all looks great. I've also been hacking at this problem and have a
working solution that does not involve any open pipeline surgery or
triple-pipeline-bypasses. It is a bit of a hack and I do believe your
solution is much more appropriate in the long term, however, maybe what
I've done will give you some more ideas.
Originally I was using the CocoonBean to test pipelines, but I had some
new pipelines that needed access to HTTP headers and cookies -- and I
could not find a way to pass them in using the interfaces provided by
the bean. So I did the following:
- I extended the ExcaliburTestCase to create a CocoonTestCase. In my
new class I copied the initalize() and getClassPath() methods from the
CocoonWrapper and modified it all enough so on JUnit setUp() I could
instantiate an instance of Cocoon and stick it in a member variable.
The reason I extended ExcaliburTestCase is that I have several Avalon
components that my application uses that I also use to set up the test
cases.
- To send a request to Cocoon, I create an HttpEnvironment object using
mock servlet objects -- specifically the objects from
http://strutstestcase.sourceforge.net/. When I get the request back, I
parse it into XML and use the XMLUnit style asserts on it. Here is what
the code looks like:
----
// set up the mock objects
ServletContextSimulator sctx = new ServletContextSimulator();
HttpContext hc = new HttpContext((ServletContext) sctx);
HttpServletRequestSimulator req = new HttpServletRequestSimulator(sctx);
HttpServletResponseSimulator res = new HttpServletResponseSimulator();
// set up the HTTP request
req.setHeader("Accept", text/xml,application/xml");
req.setHeader("Accept-Language", "en");
req.setParameterValue("cocoon-view", new String [] {"content"});
// output goes to baos
ByteArrayOutputStream baos = new ByteArrayOutputStream();
res.setOutputStream(out);
// set up HttpEnvironment to get the uri "foo/bar"
HttpEnvironment env = new HttpEnvironment(
"foo/bar",
"path/to/webapp",
(HttpServletRequest) req,
(HttpServletResponse) res,
(ServletContext) sctx,
hc,
"",
"");
env.enableLogging(new LogKitLogger(getLogger()));
// call process() on the cocoon member variable
boolean rv = cocoon.process(env);
baos.flush();
// parse the response into XML
SAXParser parser = (SAXParser) this.manager.lookup(SAXParser.ROLE);
DOMBuilder builder = new DOMBuilder();
parser.parse(new InputSource(
new ByteArrayInputStream(baos.toByteArray())),
new WhitespaceFilter(builder),
builder);
Document xml = builder.getDocument();
this.manager.release((Component) parser);
// test the xml
assertXPathTrue(xml, "/blah");
----
This has worked pretty well so far. I think extending the
ExaliburTestCase and using mock servlet objects might be better than
extedning HTTPUnit since it gives us more flexibility and more access to
the raw Coccon request/response.
Some other things I've investigated on the way:
- Jakarta Cactus (http://jakarta.apache.org/cactus/) -- This takes a
similar approach by setting up a mock servlet environment to test your
servlets in. However, it seemed way to big and complicated to do what I
thought would be very simple. Also, I think there is a lot to be gained
by unit testing at the pipeline level as you suggest, rather than
treating Cocoon like just another servlet.
- In my search for a mock implementation of the servlet classes, I came
along MockObjects (http://www.mockobjects.com). They have mock objects
for much of the Java API, however, their approach didn't really fit into
what I was trying to do. Basically, you are supposed to pre-program the
mock objects to expect certain method calls (e.g. getHeader should be
called with such and such parameters, then getSession should be called,
etc). Since I just wanted mock objects good enough to masquerade as a
servlet container and get the output from Cocoon, this was not very
helpful. However, perhaps a similar methodology could be adapted where
you could pass in the expected XML for each pipeline step -- I think you
had this idea in our previous exchanges.
And finally, on a somewhat unrelated subject, one thing that I've always
wanted Cocoon to do may be possible if support for collecting the XML at
each pipeline step is added. To aid in debugging, I think it would be
very helpful to switch on some kind of debug mode, that would cause a
trace of what pipeline steps where executed and the state of the XML at
each step to be printed out at the bottom of each page you output to the
browser. This way it is easy for a developer to see the path though the
pipelines the request took, as well as a snapshot of the XML each step
of the way.
cheers,
-steve
Re: [RT] Unit testing and CocoonUnit
Posted by Upayavira <uv...@upaya.co.uk>.
Bertrand Delacretaz wrote:
> Hi Upayavira,
>
> This looks real good, having a specialized testing framework will
> certainly help improving our test traceability!
Great.
>
> I just have one small thing to add:
>
>> ...CocoonUnit would use the CocoonBean to pass requests to Cocoon.
>> The bean would prepare an Environment (maybe even an
>> HTTPEnvironment), and would place an empty ArrayList into the object
>> model with a key of UNIT_TEST_XML...
>
>
> Instead of an ArrayList you might want to put a custom object there
> (CocoonUnitContext?), for example to allow the testing code to specify
> options later on, like "don't store steps 1 and 4 of the pipeline".
That makes sense. I'll do it that way.
>> ...Conclusion
>> ----------
>> Unit testing helps to provide reliable code. Unit testing in a Web
>> environment is not always straightforward as your 'units' aren't
>> always clearly defined. With Cocoon, however, stages in a pipeline
>> can be considered to be 'units', and therefore could be tested
>> independently of each other. Adding a CocoonUnit to the Cocoon
>> codebase would further extend Cocoon's ability to be used in
>> substantial web environments where automated testing is crucial to
>> success.
>
>
> +1000
>
>> What do you all think?
>
>
> +1000 again!
Glad you like it. Now all I've got to do is get it working!
Regards, Upayavira
Re: [RT] Unit testing and CocoonUnit
Posted by Bertrand Delacretaz <bd...@apache.org>.
Hi Upayavira,
This looks real good, having a specialized testing framework will
certainly help improving our test traceability!
I just have one small thing to add:
> ...CocoonUnit would use the CocoonBean to pass requests to Cocoon. The
> bean would prepare an Environment (maybe even an HTTPEnvironment), and
> would place an empty ArrayList into the object model with a key of
> UNIT_TEST_XML...
Instead of an ArrayList you might want to put a custom object there
(CocoonUnitContext?), for example to allow the testing code to specify
options later on, like "don't store steps 1 and 4 of the pipeline".
> ...Conclusion
> ----------
> Unit testing helps to provide reliable code. Unit testing in a Web
> environment is not always straightforward as your 'units' aren't
> always clearly defined. With Cocoon, however, stages in a pipeline can
> be considered to be 'units', and therefore could be tested
> independently of each other. Adding a CocoonUnit to the Cocoon
> codebase would further extend Cocoon's ability to be used in
> substantial web environments where automated testing is crucial to
> success.
+1000
> What do you all think?
+1000 again!
-Bertrand
Re: [RT] Unit testing and CocoonUnit
Posted by Upayavira <uv...@upaya.co.uk>.
Jeremy Quinn wrote:
>
> On Saturday, November 1, 2003, at 05:21 PM, Upayavira wrote:
>
> <snip wot="really interesting stuff">
>
>> String generatorXML = response.getPipelineXML(0);
>> String transformedXML = response.getPipelineXML(1);
>>
>
> If I understand this correctly .... you are referring to elements on
> the pipeline via an index.
>
> Are you sure that is such a great idea?
>
> If you re-implement a pipeline, you would have to re-write your tests
> .... which seems to be contrary to the whole point of pipeline testing
> .....
The point here is 'unit' testing, i.e. testing just one bit. Therefore,
if you do too much reworking of your pipeline, you've probably changed
your 'units' and thus would need new tests.
Also, whether indexes are wise or not, they would be tremendously easy
to implement.
> Would it not be better to refer to parts of a pipeline by their
> view-label?
I did mull on that one. Firstly, not all pipeline stages necessarily
have labels - so you would have to add labels for each pipeline stage,
which would require a change to your sitemap, and I'm keen to avoid
that. Also, I don't know how I would get at the view label from within
the AbstractProcessingPipeline. I suspect that the point I want to make
my change (so as to minimise the impact upon an important bit of Cocoon)
no longer has access to the view information - all that is left is a
generator, some transformers and a serializer.
However, I do think that offering both indexes and view labels would be
good, if I were able to work out how to identify the label.
Regards, Upayavira
Re: [RT] Unit testing and CocoonUnit
Posted by Jeremy Quinn <je...@media.demon.co.uk>.
On Saturday, November 1, 2003, at 05:21 PM, Upayavira wrote:
<snip wot="really interesting stuff">
> String generatorXML = response.getPipelineXML(0);
> String transformedXML = response.getPipelineXML(1);
>
If I understand this correctly .... you are referring to elements on
the pipeline via an index.
Are you sure that is such a great idea?
If you re-implement a pipeline, you would have to re-write your tests
.... which seems to be contrary to the whole point of pipeline testing
.....
Would it not be better to refer to parts of a pipeline by their
view-label?
regards Jeremy