You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@qpid.apache.org by ja...@mail.ooc.nf.ca on 2007/03/22 04:13:21 UTC
.NET Client Unit Test Failures
Hi all,
I just discovered that several of the unit tests for the .NET client are failing or have been disabled. I've only started running these recently (past day or two) so I don't know how long they have been in this state. I wanted to bring this up just in case it would affect the release in any way. The tests are as follows:
Qpid.Client.Tests.ServiceRequestingClient: disabled
Qpid.Client.Tests.failover.FailoverTest: disabled
Qpid.Client.Tests.failover.FailoverTxTest: disabled
Qpid.Client.Tests.ServiceProvidingClient: disabled
Qpid.Client.Tests.HeadersExchangeTest: 3/4 tests fail
Comments appreciated.
Cheers,
Jon
Re: .NET Client Unit Test Failures
Posted by Jonathan Anstey <ja...@iona.com>.
Tomas,
Now it makes sense why the tests were failing! If I find any real bugs
I'll file them in JIRA.
Thanks a million!
Jon
PS. Your shameless plug has been duly noted. :)
Tomas Restrepo wrote:
> Jon,
>
>
>
>> I just discovered that several of the unit tests for the .NET client
>> are failing or have been disabled. I've only started running these
>> recently (past day or two) so I don't know how long they have been in
>> this state. I wanted to bring this up just in case it would affect the
>> release in any way. The tests are as follows:
>>
>> Qpid.Client.Tests.ServiceRequestingClient: disabled
>> Qpid.Client.Tests.failover.FailoverTest: disabled
>> Qpid.Client.Tests.failover.FailoverTxTest: disabled
>> Qpid.Client.Tests.ServiceProvidingClient: disabled
>> Qpid.Client.Tests.HeadersExchangeTest: 3/4 tests fail
>>
>
> I haven't checked on the status of all of them, but here's the crux of the
> issue:
>
> Right now, the test suites in the .NET client are a mixture of independent,
> real unit tests and a bunch of integration-level tests that require certain
> environment support to be available.
>
> For example, the Qpid.Buffer tests are your usual standalone unit tests, but
> the most of the tests in Qpid.Client.Tests require a running broker to
> connect to.
>
> One of the issues here is that some of the tests required even more. For
> example, I think the failover tests required you to be running a broker to
> failover to, as well as fail the primary broker manually at the right time.
> I don't think many of us are running such an environment and are willing to
> manually do those tricks every time the test suite run :)
>
> I'm not sure about the other tests, but I think the ServiceProvidingClient
> and ServiceRequestingClient tests were meant to be run at the same time to
> talk to one another (which would 90% of the time not be the case of how one
> would run them). The end result was that all the stuff like this just caused
> the tests to hang for a long time if you made the mistake of running the
> entire test suite at once. I think Rupert may know a bit more about these
> than I do.
>
> I'll be the first to admit that the current state of affairs is not good,
> and we definitely need to solve this. From one side, I think that tests such
> as ServiceProvidingClient and ServiceRequestingClient would be better off
> moving to the new interop testing infrastructure that Rupert Smith has been
> working on (though work on implementing the necessary support for the .NET
> client has not begun yet).
>
>
> Shameless Plug: The truth is the .NET client still needs a *lot* of work to
> reach the level of maturity the Java client has. We've done tremendous
> progress of the past couple of months, but we need to do more, and any help
> in doing so would be gladly welcome. If you'd like to contribute code and
> patches, they'd be really appreciated, though even just bringing up any
> issues you run into and filing bugs is of significant help!
>
>
> Tomas Restrepo
> http://www.winterdom.com/weblog/
>
>
>
>
>
>
RE: .NET Client Unit Test Failures
Posted by Tomas Restrepo <to...@devdeo.com>.
Jon,
> I just discovered that several of the unit tests for the .NET client
> are failing or have been disabled. I've only started running these
> recently (past day or two) so I don't know how long they have been in
> this state. I wanted to bring this up just in case it would affect the
> release in any way. The tests are as follows:
>
> Qpid.Client.Tests.ServiceRequestingClient: disabled
> Qpid.Client.Tests.failover.FailoverTest: disabled
> Qpid.Client.Tests.failover.FailoverTxTest: disabled
> Qpid.Client.Tests.ServiceProvidingClient: disabled
> Qpid.Client.Tests.HeadersExchangeTest: 3/4 tests fail
I haven't checked on the status of all of them, but here's the crux of the
issue:
Right now, the test suites in the .NET client are a mixture of independent,
real unit tests and a bunch of integration-level tests that require certain
environment support to be available.
For example, the Qpid.Buffer tests are your usual standalone unit tests, but
the most of the tests in Qpid.Client.Tests require a running broker to
connect to.
One of the issues here is that some of the tests required even more. For
example, I think the failover tests required you to be running a broker to
failover to, as well as fail the primary broker manually at the right time.
I don't think many of us are running such an environment and are willing to
manually do those tricks every time the test suite run :)
I'm not sure about the other tests, but I think the ServiceProvidingClient
and ServiceRequestingClient tests were meant to be run at the same time to
talk to one another (which would 90% of the time not be the case of how one
would run them). The end result was that all the stuff like this just caused
the tests to hang for a long time if you made the mistake of running the
entire test suite at once. I think Rupert may know a bit more about these
than I do.
I'll be the first to admit that the current state of affairs is not good,
and we definitely need to solve this. From one side, I think that tests such
as ServiceProvidingClient and ServiceRequestingClient would be better off
moving to the new interop testing infrastructure that Rupert Smith has been
working on (though work on implementing the necessary support for the .NET
client has not begun yet).
Shameless Plug: The truth is the .NET client still needs a *lot* of work to
reach the level of maturity the Java client has. We've done tremendous
progress of the past couple of months, but we need to do more, and any help
in doing so would be gladly welcome. If you'd like to contribute code and
patches, they'd be really appreciated, though even just bringing up any
issues you run into and filing bugs is of significant help!
Tomas Restrepo
http://www.winterdom.com/weblog/
Re: .NET Client Unit Test Failures
Posted by Jonathan Anstey <ja...@iona.com>.
Yeah, I agree. The .NET client has really taken a back seat compared to
the Java client unfortunately... I believe there are several Java
continuous builds throughout the Qpid community - no .NET ones that I
know about. If I get a chance in the next century I may set that up ;)
Cheers,
Jon
Rupert Smith wrote:
> That is unfortunate as they were working a while back. I've been
> meaning to
> get these tests (or similar) running regularly in order to keep an eye on
> the state of the .Net. Continous build + interop tests would be the
> thing to
> do IMO. That particular goal is taking a back seat behind more immediate
> priorities for internal clients at the moment.
>
> Rupert
>
> On 22/03/07, Jonathan Anstey <ja...@iona.com> wrote:
>>
>> Rupert,
>>
>> Unfortunately the header exchange tests still fail when a broker is
>> started up. :( The tests appear to connect to the broker and set up the
>> exchange fine, but fail in actually sending the message through the
>> exchange to the consumer...
>>
>> Cheers,
>> Jon
>>
>> Rupert Smith wrote:
>> > As Thomas pointed out, these tests were not unit tests, but more like
>> > integration tests. I removed/commented some of them out in an effort
>> > to try
>> > and get down to just those that would run in a fully automated way.
>> >
>> > They should really be moved into an integration test package.
>> >
>> > Service requesting/providing need to be run as a pair. To try and
>> get a
>> > better automated tests suite, they should really be combined together
>> > into a
>> > single NUnit test that does both ends.
>> >
>> > Have you tried running them with a broker started on default
>> > localhost/port?
>> > I think headers exchange tests ought to pass like that.
>> >
>> > Rupert
>> >
>> > On 22/03/07, Tomas Restrepo <to...@devdeo.com> wrote:
>> >>
>> >> Jon,
>> >>
>> >>
>> >> > I just discovered that several of the unit tests for the .NET
>> client
>> >> > are failing or have been disabled. I've only started running these
>> >> > recently (past day or two) so I don't know how long they have
>> been in
>> >> > this state. I wanted to bring this up just in case it would affect
>> the
>> >> > release in any way. The tests are as follows:
>> >> >
>> >> > Qpid.Client.Tests.ServiceRequestingClient: disabled
>> >> > Qpid.Client.Tests.failover.FailoverTest: disabled
>> >> > Qpid.Client.Tests.failover.FailoverTxTest: disabled
>> >> > Qpid.Client.Tests.ServiceProvidingClient: disabled
>> >> > Qpid.Client.Tests.HeadersExchangeTest: 3/4 tests fail
>> >>
>> >> I haven't checked on the status of all of them, but here's the crux
>> >> of the
>> >> issue:
>> >>
>> >> Right now, the test suites in the .NET client are a mixture of
>> >> independent,
>> >> real unit tests and a bunch of integration-level tests that require
>> >> certain
>> >> environment support to be available.
>> >>
>> >> For example, the Qpid.Buffer tests are your usual standalone unit
>> tests,
>> >> but
>> >> the most of the tests in Qpid.Client.Tests require a running
>> broker to
>> >> connect to.
>> >>
>> >> One of the issues here is that some of the tests required even more.
>> For
>> >> example, I think the failover tests required you to be running a
>> >> broker to
>> >> failover to, as well as fail the primary broker manually at the right
>> >> time.
>> >> I don't think many of us are running such an environment and are
>> willing
>> >> to
>> >> manually do those tricks every time the test suite run :)
>> >>
>> >> I'm not sure about the other tests, but I think the
>> >> ServiceProvidingClient
>> >> and ServiceRequestingClient tests were meant to be run at the same
>> >> time to
>> >> talk to one another (which would 90% of the time not be the case
>> of how
>> >> one
>> >> would run them). The end result was that all the stuff like this just
>> >> caused
>> >> the tests to hang for a long time if you made the mistake of running
>> the
>> >> entire test suite at once. I think Rupert may know a bit more about
>> >> these
>> >> than I do.
>> >>
>> >> I'll be the first to admit that the current state of affairs is not
>> >> good,
>> >> and we definitely need to solve this. From one side, I think that
>> tests
>> >> such
>> >> as ServiceProvidingClient and ServiceRequestingClient would be better
>> >> off
>> >> moving to the new interop testing infrastructure that Rupert Smith
>> has
>> >> been
>> >> working on (though work on implementing the necessary support for the
>> >> .NET
>> >> client has not begun yet).
>> >>
>> >>
>> >> Shameless Plug: The truth is the .NET client still needs a *lot* of
>> work
>> >> to
>> >> reach the level of maturity the Java client has. We've done
>> tremendous
>> >> progress of the past couple of months, but we need to do more, and
>> any
>> >> help
>> >> in doing so would be gladly welcome. If you'd like to contribute code
>> >> and
>> >> patches, they'd be really appreciated, though even just bringing
>> up any
>> >> issues you run into and filing bugs is of significant help!
>> >>
>> >>
>> >> Tomas Restrepo
>> >> http://www.winterdom.com/weblog/
>> >>
>> >>
>> >>
>> >>
>> >>
>> >
>>
>>
>
Re: .NET Client Unit Test Failures
Posted by Rupert Smith <ru...@googlemail.com>.
That is unfortunate as they were working a while back. I've been meaning to
get these tests (or similar) running regularly in order to keep an eye on
the state of the .Net. Continous build + interop tests would be the thing to
do IMO. That particular goal is taking a back seat behind more immediate
priorities for internal clients at the moment.
Rupert
On 22/03/07, Jonathan Anstey <ja...@iona.com> wrote:
>
> Rupert,
>
> Unfortunately the header exchange tests still fail when a broker is
> started up. :( The tests appear to connect to the broker and set up the
> exchange fine, but fail in actually sending the message through the
> exchange to the consumer...
>
> Cheers,
> Jon
>
> Rupert Smith wrote:
> > As Thomas pointed out, these tests were not unit tests, but more like
> > integration tests. I removed/commented some of them out in an effort
> > to try
> > and get down to just those that would run in a fully automated way.
> >
> > They should really be moved into an integration test package.
> >
> > Service requesting/providing need to be run as a pair. To try and get a
> > better automated tests suite, they should really be combined together
> > into a
> > single NUnit test that does both ends.
> >
> > Have you tried running them with a broker started on default
> > localhost/port?
> > I think headers exchange tests ought to pass like that.
> >
> > Rupert
> >
> > On 22/03/07, Tomas Restrepo <to...@devdeo.com> wrote:
> >>
> >> Jon,
> >>
> >>
> >> > I just discovered that several of the unit tests for the .NET client
> >> > are failing or have been disabled. I've only started running these
> >> > recently (past day or two) so I don't know how long they have been in
> >> > this state. I wanted to bring this up just in case it would affect
> the
> >> > release in any way. The tests are as follows:
> >> >
> >> > Qpid.Client.Tests.ServiceRequestingClient: disabled
> >> > Qpid.Client.Tests.failover.FailoverTest: disabled
> >> > Qpid.Client.Tests.failover.FailoverTxTest: disabled
> >> > Qpid.Client.Tests.ServiceProvidingClient: disabled
> >> > Qpid.Client.Tests.HeadersExchangeTest: 3/4 tests fail
> >>
> >> I haven't checked on the status of all of them, but here's the crux
> >> of the
> >> issue:
> >>
> >> Right now, the test suites in the .NET client are a mixture of
> >> independent,
> >> real unit tests and a bunch of integration-level tests that require
> >> certain
> >> environment support to be available.
> >>
> >> For example, the Qpid.Buffer tests are your usual standalone unit
> tests,
> >> but
> >> the most of the tests in Qpid.Client.Tests require a running broker to
> >> connect to.
> >>
> >> One of the issues here is that some of the tests required even more.
> For
> >> example, I think the failover tests required you to be running a
> >> broker to
> >> failover to, as well as fail the primary broker manually at the right
> >> time.
> >> I don't think many of us are running such an environment and are
> willing
> >> to
> >> manually do those tricks every time the test suite run :)
> >>
> >> I'm not sure about the other tests, but I think the
> >> ServiceProvidingClient
> >> and ServiceRequestingClient tests were meant to be run at the same
> >> time to
> >> talk to one another (which would 90% of the time not be the case of how
> >> one
> >> would run them). The end result was that all the stuff like this just
> >> caused
> >> the tests to hang for a long time if you made the mistake of running
> the
> >> entire test suite at once. I think Rupert may know a bit more about
> >> these
> >> than I do.
> >>
> >> I'll be the first to admit that the current state of affairs is not
> >> good,
> >> and we definitely need to solve this. From one side, I think that tests
> >> such
> >> as ServiceProvidingClient and ServiceRequestingClient would be better
> >> off
> >> moving to the new interop testing infrastructure that Rupert Smith has
> >> been
> >> working on (though work on implementing the necessary support for the
> >> .NET
> >> client has not begun yet).
> >>
> >>
> >> Shameless Plug: The truth is the .NET client still needs a *lot* of
> work
> >> to
> >> reach the level of maturity the Java client has. We've done tremendous
> >> progress of the past couple of months, but we need to do more, and any
> >> help
> >> in doing so would be gladly welcome. If you'd like to contribute code
> >> and
> >> patches, they'd be really appreciated, though even just bringing up any
> >> issues you run into and filing bugs is of significant help!
> >>
> >>
> >> Tomas Restrepo
> >> http://www.winterdom.com/weblog/
> >>
> >>
> >>
> >>
> >>
> >
>
>
Re: .NET Client Unit Test Failures
Posted by Jonathan Anstey <ja...@iona.com>.
Rupert,
Unfortunately the header exchange tests still fail when a broker is
started up. :( The tests appear to connect to the broker and set up the
exchange fine, but fail in actually sending the message through the
exchange to the consumer...
Cheers,
Jon
Rupert Smith wrote:
> As Thomas pointed out, these tests were not unit tests, but more like
> integration tests. I removed/commented some of them out in an effort
> to try
> and get down to just those that would run in a fully automated way.
>
> They should really be moved into an integration test package.
>
> Service requesting/providing need to be run as a pair. To try and get a
> better automated tests suite, they should really be combined together
> into a
> single NUnit test that does both ends.
>
> Have you tried running them with a broker started on default
> localhost/port?
> I think headers exchange tests ought to pass like that.
>
> Rupert
>
> On 22/03/07, Tomas Restrepo <to...@devdeo.com> wrote:
>>
>> Jon,
>>
>>
>> > I just discovered that several of the unit tests for the .NET client
>> > are failing or have been disabled. I've only started running these
>> > recently (past day or two) so I don't know how long they have been in
>> > this state. I wanted to bring this up just in case it would affect the
>> > release in any way. The tests are as follows:
>> >
>> > Qpid.Client.Tests.ServiceRequestingClient: disabled
>> > Qpid.Client.Tests.failover.FailoverTest: disabled
>> > Qpid.Client.Tests.failover.FailoverTxTest: disabled
>> > Qpid.Client.Tests.ServiceProvidingClient: disabled
>> > Qpid.Client.Tests.HeadersExchangeTest: 3/4 tests fail
>>
>> I haven't checked on the status of all of them, but here's the crux
>> of the
>> issue:
>>
>> Right now, the test suites in the .NET client are a mixture of
>> independent,
>> real unit tests and a bunch of integration-level tests that require
>> certain
>> environment support to be available.
>>
>> For example, the Qpid.Buffer tests are your usual standalone unit tests,
>> but
>> the most of the tests in Qpid.Client.Tests require a running broker to
>> connect to.
>>
>> One of the issues here is that some of the tests required even more. For
>> example, I think the failover tests required you to be running a
>> broker to
>> failover to, as well as fail the primary broker manually at the right
>> time.
>> I don't think many of us are running such an environment and are willing
>> to
>> manually do those tricks every time the test suite run :)
>>
>> I'm not sure about the other tests, but I think the
>> ServiceProvidingClient
>> and ServiceRequestingClient tests were meant to be run at the same
>> time to
>> talk to one another (which would 90% of the time not be the case of how
>> one
>> would run them). The end result was that all the stuff like this just
>> caused
>> the tests to hang for a long time if you made the mistake of running the
>> entire test suite at once. I think Rupert may know a bit more about
>> these
>> than I do.
>>
>> I'll be the first to admit that the current state of affairs is not
>> good,
>> and we definitely need to solve this. From one side, I think that tests
>> such
>> as ServiceProvidingClient and ServiceRequestingClient would be better
>> off
>> moving to the new interop testing infrastructure that Rupert Smith has
>> been
>> working on (though work on implementing the necessary support for the
>> .NET
>> client has not begun yet).
>>
>>
>> Shameless Plug: The truth is the .NET client still needs a *lot* of work
>> to
>> reach the level of maturity the Java client has. We've done tremendous
>> progress of the past couple of months, but we need to do more, and any
>> help
>> in doing so would be gladly welcome. If you'd like to contribute code
>> and
>> patches, they'd be really appreciated, though even just bringing up any
>> issues you run into and filing bugs is of significant help!
>>
>>
>> Tomas Restrepo
>> http://www.winterdom.com/weblog/
>>
>>
>>
>>
>>
>
Re: .NET Client Unit Test Failures
Posted by Rupert Smith <ru...@googlemail.com>.
As Thomas pointed out, these tests were not unit tests, but more like
integration tests. I removed/commented some of them out in an effort to try
and get down to just those that would run in a fully automated way.
They should really be moved into an integration test package.
Service requesting/providing need to be run as a pair. To try and get a
better automated tests suite, they should really be combined together into a
single NUnit test that does both ends.
Have you tried running them with a broker started on default localhost/port?
I think headers exchange tests ought to pass like that.
Rupert
On 22/03/07, Tomas Restrepo <to...@devdeo.com> wrote:
>
> Jon,
>
>
> > I just discovered that several of the unit tests for the .NET client
> > are failing or have been disabled. I've only started running these
> > recently (past day or two) so I don't know how long they have been in
> > this state. I wanted to bring this up just in case it would affect the
> > release in any way. The tests are as follows:
> >
> > Qpid.Client.Tests.ServiceRequestingClient: disabled
> > Qpid.Client.Tests.failover.FailoverTest: disabled
> > Qpid.Client.Tests.failover.FailoverTxTest: disabled
> > Qpid.Client.Tests.ServiceProvidingClient: disabled
> > Qpid.Client.Tests.HeadersExchangeTest: 3/4 tests fail
>
> I haven't checked on the status of all of them, but here's the crux of the
> issue:
>
> Right now, the test suites in the .NET client are a mixture of
> independent,
> real unit tests and a bunch of integration-level tests that require
> certain
> environment support to be available.
>
> For example, the Qpid.Buffer tests are your usual standalone unit tests,
> but
> the most of the tests in Qpid.Client.Tests require a running broker to
> connect to.
>
> One of the issues here is that some of the tests required even more. For
> example, I think the failover tests required you to be running a broker to
> failover to, as well as fail the primary broker manually at the right
> time.
> I don't think many of us are running such an environment and are willing
> to
> manually do those tricks every time the test suite run :)
>
> I'm not sure about the other tests, but I think the ServiceProvidingClient
> and ServiceRequestingClient tests were meant to be run at the same time to
> talk to one another (which would 90% of the time not be the case of how
> one
> would run them). The end result was that all the stuff like this just
> caused
> the tests to hang for a long time if you made the mistake of running the
> entire test suite at once. I think Rupert may know a bit more about these
> than I do.
>
> I'll be the first to admit that the current state of affairs is not good,
> and we definitely need to solve this. From one side, I think that tests
> such
> as ServiceProvidingClient and ServiceRequestingClient would be better off
> moving to the new interop testing infrastructure that Rupert Smith has
> been
> working on (though work on implementing the necessary support for the .NET
> client has not begun yet).
>
>
> Shameless Plug: The truth is the .NET client still needs a *lot* of work
> to
> reach the level of maturity the Java client has. We've done tremendous
> progress of the past couple of months, but we need to do more, and any
> help
> in doing so would be gladly welcome. If you'd like to contribute code and
> patches, they'd be really appreciated, though even just bringing up any
> issues you run into and filing bugs is of significant help!
>
>
> Tomas Restrepo
> http://www.winterdom.com/weblog/
>
>
>
>
>