You are viewing a plain text version of this content. The canonical link for it is here.
Posted to derby-dev@db.apache.org by Manjula Kutty <ma...@gmail.com> on 2007/06/20 21:23:49 UTC
Running junit tests with n/w server on a different machine other than the localhost
Hi
Is there a way to give the hostname other than the localhost for the junit
tests? If is it given in the java file can I edit that file and then run the
junit tests on a client machine and the server is started on another
machine?
--
Thanks,
Manjula.
Re: Running the junit testsuite with a remote server
Posted by Vemund Ostgaard <Ve...@Sun.COM>.
Myrna van Lunteren wrote:
> Starting to reason from the tests, I think part of the issue is that
> there are tests that test the correct starting and shutting down of
> the network server; or that need the server to be started with
> specific properties as opposed to the 'normal' mode. And as far as I
> know we cannot start and shutdown a server on a remote machine, and
> thus these tests cannot run with a remote server.
>
So there is a group of tests that can only be run with the server on the
localhost and/or only make sense to run with the server on the localhost.
Probably there is also a group of tests that only make sense to run with
the server on a remote machine, for instance verifying that you are not
allowed to shut it down?
Finally there is the group of tests that make sense to run both with a
remote server and with a local server, and possibly also in embedded mode.
> Theoretically, all tests that do not need to test starting/shutting
> down, or that do not test the server with specific properties set,
> should, be 'runnable' with remote server that is already running.
> However, is that really necessary? What things could go wrong when
> connecting to a remote server versus a server on localhost? I would
> think mostly things to do with connections, permissions,
> import/export. So, maybe it makes sense after all to have a dedicated
> set of tests for remote server testing.
>
From reading about the junit tests on the wiki and taking a look at the
code, I understand that the logic for controlling which test fixtures
are run with an embedded database and which fixtures are run with a
network server, is in the suite() method of the various test classes. I
guess this is the right place to put logic that will
> Note that with remote server testing, trouble brews in the area of
> cleanup; I believe the current junit framework attempts to delete
> files that get created by the server; and I think deleting of files on
> a remote machine should be impossible. I wasted considerable time with
> the old test harness trying to identify tests that would run cleanly
> in sequence; only to find that a previously working set was made to
> fail because a uncooperative test/testcase was added. It may be better
> to create a suite of tests, normally part of suites.All, which
> can/should be run against a remote server as well.
>
> For the desire to be able to run all client tests; that is more an
> efficiency issue useful for someone working on client only changes;
> when one is certain there is no need to run the tests with embedded. I
> think maybe a class that travels through the test suites and
> identifies the ones that run with DerbyNetClient would cover that
> requirement. The converse could also be true (run only embedded
> tests), but I believe this can be achieved by excluding
> derbyclient.jar from the classpath.
>
> I have a hard time to imagine how to arrange for a configurable port
> number and host without using properties in some manner. But I imagine
> some default values, and/or a scheme to identify existing servers
> running could be devised. Maybe someone else has good suggestions.
>
> Myrna
>
Re: Running the junit testsuite with a remote server
Posted by Vemund Ostgaard <Ve...@Sun.COM>.
Oops, hit the send button by accident in the middle of writing, so
resending this mail.
Myrna van Lunteren wrote:
> Starting to reason from the tests, I think part of the issue is that
> there are tests that test the correct starting and shutting down of
> the network server; or that need the server to be started with
> specific properties as opposed to the 'normal' mode. And as far as I
> know we cannot start and shutdown a server on a remote machine, and
> thus these tests cannot run with a remote server.
>
So there is a group of tests that can only be run with the server on the
localhost and/or only make sense to run with the server on the localhost.
Probably there is also a group of tests that only make sense to run with
the server on a remote machine, for instance verifying that you are not
allowed to shut it down?
Finally there is the group of tests that make sense to run both with a
remote server and with a local server, and possibly also in embedded mode.
> Theoretically, all tests that do not need to test starting/shutting
> down, or that do not test the server with specific properties set,
> should, be 'runnable' with remote server that is already running.
> However, is that really necessary? What things could go wrong when
> connecting to a remote server versus a server on localhost? I would
> think mostly things to do with connections, permissions,
> import/export. So, maybe it makes sense after all to have a dedicated
> set of tests for remote server testing.
>
From reading about the junit tests on the wiki and taking a look at the
code, I understand that the logic for controlling which test fixtures
are run with an embedded database and which fixtures are run with a
network server, is in the suite() method of the various test classes. I
guess this is the place to put logic that will give you a suite with the
right fixtures in it and with the right decorators, depending on whether
you want to run tests with a local server, a remote server, or embedded.
It should be possible to run only the fixtures for the modes that you
want to test, but ideally it should also be possible to run all the
fixtures in all the modes that they support. So I guess what is needed
in terms of functionality for the test framework is to be able to say
* I do/don't want to run the "embedded tests"
* I do/don't want to run the "local server tests"
* I do/don't want to run the "remote server tests"
The default behavior could be as it is today, to run the tests for all
modes, provided that the framework has been instructed about the
whereabouts of a remote server. If not, skip the "remote server tests".
> Note that with remote server testing, trouble brews in the area of
> cleanup; I believe the current junit framework attempts to delete
> files that get created by the server; and I think deleting of files on
> a remote machine should be impossible. I wasted considerable time with
> the old test harness trying to identify tests that would run cleanly
> in sequence; only to find that a previously working set was made to
> fail because a uncooperative test/testcase was added.
Is it necessary to delete files to be able to clean properly between
test fixtures? It wouldn't be sufficient in most cases to just drop
everything with SQL?
It may be better
> to create a suite of tests, normally part of suites.All, which
> can/should be run against a remote server as well.
I am concerned about a top level suite knowing about which tests are
supposed to run in this or that mode. I think the way it is now, that
this knowledge is contained in each testcase class, makes sense.
>
> For the desire to be able to run all client tests; that is more an
> efficiency issue useful for someone working on client only changes;
> when one is certain there is no need to run the tests with embedded. I
> think maybe a class that travels through the test suites and
> identifies the ones that run with DerbyNetClient would cover that
> requirement. The converse could also be true (run only embedded
> tests), but I believe this can be achieved by excluding
> derbyclient.jar from the classpath.
>
Another reason to be able to run only the "embedded tests" or "client
tests" would be to limit which jars/classes that are in the classpath to
the ones that are strictly required in that mode and test that nothing
breaks because of that.
Vemund
Re: Running the junit testsuite with a remote server
Posted by Myrna van Lunteren <m....@gmail.com>.
On 9/17/07, Vemund Ostgaard <Ve...@sun.com> wrote:
> I'm interested in running the junit tests with a remote server, but as
> far as I can see there is no easy way to do so yet.
>
> I found two relevant jiras:
> https://issues.apache.org/jira/browse/DERBY-1973 (Support running JUnit
> tests directly with a remote server)
> http://issues.apache.org/jira/browse/DERBY-2638 (Create an option for
> junit tests to run only client tests)
>
> As these are currently unassigned I was thinking of taking a closer look
> myself.
>
> A couple of questions:
>
> What interface should be used to instruct the test framework that you
> want to use a network server (probably started manually) running on a
> given host & portnumber, instead of the default localhost location? And
> along the same lines, what is the best way to instruct the test
> framework that you want to run only the "client tests" (or only the
> "embedded tests")?
>
> For choosing to run only a subset of tests, like the "client tests", I
> see a few possibilities:
> * Create some new top level suite, for instance "Suites.client" that
> runs only the "client tests".
> * Set a property (on the commandline) when starting Suites.All (or any
> other subsuite) that instructs the framework to only run the "client tests".
> * Make it possible to do both of the above.
>
> For instructing the framework to use a specific hostname and portnumber
> instead of the defaults I don't have any good ideas except using
> properties. The DERBY-1973 report says "Ideally this would be through a
> test decorator and not setting properties on the command line.", though.
> Any suggestions on the best way to solve this? The TestConfiguration
> class seems to read system properties for framework (embedded or
> client), hostname and port, but is this code there only to make the
> tests run under the old harness and intended to be removed when all the
> tests are converted?
>
> Is it likely that all "client tests" will work with a remote server, or
> will some tests be written in a way that requires the server(s) to run
> on the same host as the test framework? Perhaps some tests have to start
> the networkserver as part of the test, and because of this it has to run
> on the same host. In this case it would actually be three groups of
> tests: embedded, local networkserver and remote networkserver.
>
> If anyone has given all of this some thought I'd be happy to hear.
>
> Vemund
>
Coming from attempting to implement remote server testing in the 'old'
test harness, I have some limited input, and little to offer in the
way of solutions...
Starting to reason from the tests, I think part of the issue is that
there are tests that test the correct starting and shutting down of
the network server; or that need the server to be started with
specific properties as opposed to the 'normal' mode. And as far as I
know we cannot start and shutdown a server on a remote machine, and
thus these tests cannot run with a remote server.
Theoretically, all tests that do not need to test starting/shutting
down, or that do not test the server with specific properties set,
should, be 'runnable' with remote server that is already running.
However, is that really necessary? What things could go wrong when
connecting to a remote server versus a server on localhost? I would
think mostly things to do with connections, permissions,
import/export. So, maybe it makes sense after all to have a dedicated
set of tests for remote server testing.
Note that with remote server testing, trouble brews in the area of
cleanup; I believe the current junit framework attempts to delete
files that get created by the server; and I think deleting of files on
a remote machine should be impossible. I wasted considerable time with
the old test harness trying to identify tests that would run cleanly
in sequence; only to find that a previously working set was made to
fail because a uncooperative test/testcase was added. It may be better
to create a suite of tests, normally part of suites.All, which
can/should be run against a remote server as well.
For the desire to be able to run all client tests; that is more an
efficiency issue useful for someone working on client only changes;
when one is certain there is no need to run the tests with embedded. I
think maybe a class that travels through the test suites and
identifies the ones that run with DerbyNetClient would cover that
requirement. The converse could also be true (run only embedded
tests), but I believe this can be achieved by excluding
derbyclient.jar from the classpath.
I have a hard time to imagine how to arrange for a configurable port
number and host without using properties in some manner. But I imagine
some default values, and/or a scheme to identify existing servers
running could be devised. Maybe someone else has good suggestions.
Myrna
Running the junit testsuite with a remote server
Posted by Vemund Ostgaard <Ve...@Sun.COM>.
I'm interested in running the junit tests with a remote server, but as
far as I can see there is no easy way to do so yet.
I found two relevant jiras:
https://issues.apache.org/jira/browse/DERBY-1973 (Support running JUnit
tests directly with a remote server)
http://issues.apache.org/jira/browse/DERBY-2638 (Create an option for
junit tests to run only client tests)
As these are currently unassigned I was thinking of taking a closer look
myself.
A couple of questions:
What interface should be used to instruct the test framework that you
want to use a network server (probably started manually) running on a
given host & portnumber, instead of the default localhost location? And
along the same lines, what is the best way to instruct the test
framework that you want to run only the "client tests" (or only the
"embedded tests")?
For choosing to run only a subset of tests, like the "client tests", I
see a few possibilities:
* Create some new top level suite, for instance "Suites.client" that
runs only the "client tests".
* Set a property (on the commandline) when starting Suites.All (or any
other subsuite) that instructs the framework to only run the "client tests".
* Make it possible to do both of the above.
For instructing the framework to use a specific hostname and portnumber
instead of the defaults I don't have any good ideas except using
properties. The DERBY-1973 report says "Ideally this would be through a
test decorator and not setting properties on the command line.", though.
Any suggestions on the best way to solve this? The TestConfiguration
class seems to read system properties for framework (embedded or
client), hostname and port, but is this code there only to make the
tests run under the old harness and intended to be removed when all the
tests are converted?
Is it likely that all "client tests" will work with a remote server, or
will some tests be written in a way that requires the server(s) to run
on the same host as the test framework? Perhaps some tests have to start
the networkserver as part of the test, and because of this it has to run
on the same host. In this case it would actually be three groups of
tests: embedded, local networkserver and remote networkserver.
If anyone has given all of this some thought I'd be happy to hear.
Vemund
Re: Running junit tests with n/w server on a different machine other than the localhost
Posted by Manjula Kutty <ma...@gmail.com>.
yes. But I don't mind starting the server manually.
On 6/20/07, John H. Embretsen <Jo...@sun.com> wrote:
>
> Manjula Kutty wrote:
> >
> > Hi
> >
> > Is there a way to give the hostname other than the localhost for the
> > junit tests? If is it given in the java file can I edit that file and
> > then run the junit tests on a client machine and the server is started
> > on another machine?
>
> I believe you are asking for a solution to DERBY-1973:
>
> Support running JUnit tests directly with a remote server
> ---------------------------------------------------------
> URL: https://issues.apache.org/jira/browse/DERBY-1973
>
> A first step would probably have to be to have the client tests run
> against an already running server on the remote host. Having the server
> start automatically on a remote host requires a bit more of an effort.
>
> --
> John
>
>
>
>
--
Thanks,
Manjula.
Re: Running junit tests with n/w server on a different machine other
than the localhost
Posted by "John H. Embretsen" <Jo...@Sun.COM>.
Manjula Kutty wrote:
>
> Hi
>
> Is there a way to give the hostname other than the localhost for the
> junit tests? If is it given in the java file can I edit that file and
> then run the junit tests on a client machine and the server is started
> on another machine?
I believe you are asking for a solution to DERBY-1973:
Support running JUnit tests directly with a remote server
---------------------------------------------------------
URL: https://issues.apache.org/jira/browse/DERBY-1973
A first step would probably have to be to have the client tests run
against an already running server on the remote host. Having the server
start automatically on a remote host requires a bit more of an effort.
--
John