You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tuscany.apache.org by Jeremy Boynes <jb...@apache.org> on 2006/08/03 18:18:26 UTC
SCA test framework
We've talked a lot recently about issues booting the server inside
the build environment in order to be able to test application code. I
think one way we might be able to solve this is through a client-side
test harness that can be integrated into Maven and Ant.
An essential observation here is that users will typically be
exposing services in the server rather than applications. Therefore
we should have a test harness capable of interacting with those
services rather than use application testing harnesses like HTMLUnit.
We can do this with an extensible SCA test harness that allows people
to easily access the services on the server. This can be an SCA
"client" environment containing just bindings, combined with some
utility functions to deploy applications to the server.
In a Maven environment, it would work like this (based on Maven build
phases):
compile: compile the application code
test: run unit tests on the application code inside Maven - no SCA
functions available
package: package up the application composite
pre-integration-test:
deploy the package to a Tuscany server (potentially booting
the remote server)
boot the test-harness Tuscany client with the appropriate
bindings
integration-test:
run integration/functional test against the exposed services
the user's test code would access the server using SCA
References
post-integration-test:
shutdown the test-harness client
undeploy the application (and possibly stop the remote server)
We would do a similar thing for Ant with different task definitions
for the various stages.
Seem reasonable?
--
Jeremy
---------------------------------------------------------------------
To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: tuscany-dev-help@ws.apache.org
Re: SCA test framework
Posted by Jeremy Boynes <jb...@apache.org>.
On Aug 3, 2006, at 3:03 PM, Ken Tam wrote:
> On 8/3/06, Jeremy Boynes <jb...@apache.org> wrote:
>> ...
>> An essential observation here is that users will typically be
>> exposing services in the server rather than applications.
>
> Quick terminology question -- you're classifying "applications" and
> "user-written services" separately? In my mind, it's all
> application/business logic -- whether it's exposed via an HTML UI or
> as a REST service/WS.
I think it's all application/business logic but the difference is the
client - for HTML and other it's a human and needs to look good and
work in the face of human errors (bum on keyboard problem), for
services the client is another application and you have different
problems (data overrun, bad format). You may well use different tools.
> How do you think bindings/extensions fit into this? For example, I
> see bindings as having some unit tests (that don't require the binding
> to be running in the context of an SCA system), but then there should
> also be tests to validate that the binding actually works when
> deployed as an extension. One way to do that would be to have a
> "user-written service" or "application" that used that binding, and
> run some integration tests against that.
I would expect that the extension provider would write copious unit
tests on the extension's implementation code and would write
functional tests that it did the right kind of thing when deployed
into a variety of servers.
I would not expect the user to repeat that kind of testing, just like
they don't test that a servlet container works (until it doesn't). I
would expect user testing to focus on business function and on
integration in their environment (e.g. their server profile with
extensions of their choice).
>> Therefore
>> we should have a test harness capable of interacting with those
>> services rather than use application testing harnesses like HTMLUnit.
>
> I like this idea, but it's a little fuzzy to me. Let's consider a
> service exposed using the Axis2 binding. How would the harness help
> with writting tests against that service? Are plain Java tests that
> just use the "native" Axis2 client APIs enough? (certainly those
> should work). Is there an equivalent to this snippet that's common to
> sample testing today:
>
> CompositeContext context = CurrentCompositeContext.getContext
> ();
> helloWorldService =
> context.locateService(HelloWorldService.class, "HelloWorldComponent");
Yes, they could definitely use the raw Axis2 client (or JAX-WS or
Celtix) to test the service.
The SCA harness would allow them to use an SCA client environment
which should make life easier for them (e.g. simple API, declarative
policies, ...)
They would write code similar to that but unlike today's code it
would be running against a remote service rather than a local component.
>> We can do this with an extensible SCA test harness that allows people
>> to easily access the services on the server. This can be an SCA
>> "client" environment containing just bindings, combined with some
>> utility functions to deploy applications to the server.
>
> I can see the SCATestCase morphing from booting up a full-blown SCA
> server environment (which is the path it's been on), to booting up
> just enough SCA "client" infrastructure where it could take an
> application SCDL that wires some tests up to the services exposed on a
> running server. Going back to the example of a service "Foo" exposed
> using the Axis2 binding -- one set of test cases would be in the form
> of an application SCDL with a <reference> element that was wired by
> the SCATestCase to the service "Foo" (which itself was exposed using a
> <service> element), and the test case logic itself would call out
> through the SCA reference.
Yes. With a cheffed version even being able to automatically generate
the client environment (e.g. it would know what services were being
tested and could automatically wire to them).
>>
>> In a Maven environment, it would work like this (based on Maven build
>> phases):
>> compile: compile the application code
>> test: run unit tests on the application code inside Maven - no SCA
>> functions available
>> package: package up the application composite
>> pre-integration-test:
>> deploy the package to a Tuscany server (potentially booting
>> the remote server)
>> boot the test-harness Tuscany client with the appropriate
>> bindings
>> integration-test:
>> run integration/functional test against the exposed
>> services
>> the user's test code would access the server using SCA
>> References
>> post-integration-test:
>> shutdown the test-harness client
>> undeploy the application (and possibly stop the remote
>> server)
>>
>> We would do a similar thing for Ant with different task definitions
>> for the various stages.
>>
>> Seem reasonable?
>
> +1 in principal -- I'd like to hear what you think the user's test
> code would look like given this infrastructure.. that would help me
> understand the proposal a lot better.
Something like:
import static o.a.t.test.SCATest.*;
public class HelloTestCase {
private final HelloService service;
public HelloTestCase(HelloService service) {
this.service = service;
}
// test the service returns what we expect
public void testGreeting() {
assertEquals("Hello World", service.getGreeting("World"));
}
}
if have our own framework or
public class HelloTestCase extends TestCase {
private HelloService service;
// test the service returns what we expect
public void testGreeting() {
assertEquals("Hello World", service.getGreeting("World"));
}
protected void setUp() {
service = CurrentCompositeContext.getContext().lookupService
(HelloService.class, "HelloService");
}
}
if the user wanted to use junit (as an integration test runner).
--
Jeremy
---------------------------------------------------------------------
To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: tuscany-dev-help@ws.apache.org
Re: SCA test framework
Posted by Ken Tam <ke...@gmail.com>.
On 8/3/06, Jeremy Boynes <jb...@apache.org> wrote:
> ...
> An essential observation here is that users will typically be
> exposing services in the server rather than applications.
Quick terminology question -- you're classifying "applications" and
"user-written services" separately? In my mind, it's all
application/business logic -- whether it's exposed via an HTML UI or
as a REST service/WS.
How do you think bindings/extensions fit into this? For example, I
see bindings as having some unit tests (that don't require the binding
to be running in the context of an SCA system), but then there should
also be tests to validate that the binding actually works when
deployed as an extension. One way to do that would be to have a
"user-written service" or "application" that used that binding, and
run some integration tests against that.
> Therefore
> we should have a test harness capable of interacting with those
> services rather than use application testing harnesses like HTMLUnit.
I like this idea, but it's a little fuzzy to me. Let's consider a
service exposed using the Axis2 binding. How would the harness help
with writting tests against that service? Are plain Java tests that
just use the "native" Axis2 client APIs enough? (certainly those
should work). Is there an equivalent to this snippet that's common to
sample testing today:
CompositeContext context = CurrentCompositeContext.getContext();
helloWorldService =
context.locateService(HelloWorldService.class, "HelloWorldComponent");
> We can do this with an extensible SCA test harness that allows people
> to easily access the services on the server. This can be an SCA
> "client" environment containing just bindings, combined with some
> utility functions to deploy applications to the server.
I can see the SCATestCase morphing from booting up a full-blown SCA
server environment (which is the path it's been on), to booting up
just enough SCA "client" infrastructure where it could take an
application SCDL that wires some tests up to the services exposed on a
running server. Going back to the example of a service "Foo" exposed
using the Axis2 binding -- one set of test cases would be in the form
of an application SCDL with a <reference> element that was wired by
the SCATestCase to the service "Foo" (which itself was exposed using a
<service> element), and the test case logic itself would call out
through the SCA reference.
>
> In a Maven environment, it would work like this (based on Maven build
> phases):
> compile: compile the application code
> test: run unit tests on the application code inside Maven - no SCA
> functions available
> package: package up the application composite
> pre-integration-test:
> deploy the package to a Tuscany server (potentially booting
> the remote server)
> boot the test-harness Tuscany client with the appropriate
> bindings
> integration-test:
> run integration/functional test against the exposed services
> the user's test code would access the server using SCA
> References
> post-integration-test:
> shutdown the test-harness client
> undeploy the application (and possibly stop the remote server)
>
> We would do a similar thing for Ant with different task definitions
> for the various stages.
>
> Seem reasonable?
+1 in principal -- I'd like to hear what you think the user's test
code would look like given this infrastructure.. that would help me
understand the proposal a lot better.
---------------------------------------------------------------------
To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: tuscany-dev-help@ws.apache.org
Re: SCA test framework
Posted by Jim Marino <jm...@myromatours.com>.
I think it sounds reasonable. A couple of things I would add:
1. We may want to look at making this into a generic deploy harness
(in addition to some type of "native" deployment on host platforms,
e.g. a J2EE server or OSGi container)
2. I think we also need to clearly establish best practices around
testing. For example, as you mentioned below, this harness should be
used for integration testing and specifically to ensure componentry
works correctly when deployed to the runtime. Its purpose is not to
unit test application code. Unit testing of application code should
be done without any Tuscany jars on the classpath. To help promote
this, we should follow these guidelines in our sample applications.
Jim
On Aug 3, 2006, at 9:18 AM, Jeremy Boynes wrote:
> We've talked a lot recently about issues booting the server inside
> the build environment in order to be able to test application code.
> I think one way we might be able to solve this is through a client-
> side test harness that can be integrated into Maven and Ant.
>
> An essential observation here is that users will typically be
> exposing services in the server rather than applications. Therefore
> we should have a test harness capable of interacting with those
> services rather than use application testing harnesses like HTMLUnit.
>
> We can do this with an extensible SCA test harness that allows
> people to easily access the services on the server. This can be an
> SCA "client" environment containing just bindings, combined with
> some utility functions to deploy applications to the server.
>
> In a Maven environment, it would work like this (based on Maven
> build phases):
> compile: compile the application code
> test: run unit tests on the application code inside Maven - no
> SCA functions available
> package: package up the application composite
> pre-integration-test:
> deploy the package to a Tuscany server (potentially
> booting the remote server)
> boot the test-harness Tuscany client with the appropriate
> bindings
> integration-test:
> run integration/functional test against the exposed services
> the user's test code would access the server using SCA
> References
> post-integration-test:
> shutdown the test-harness client
> undeploy the application (and possibly stop the remote
> server)
>
> We would do a similar thing for Ant with different task definitions
> for the various stages.
>
> Seem reasonable?
> --
> Jeremy
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
> For additional commands, e-mail: tuscany-dev-help@ws.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: tuscany-dev-help@ws.apache.org