You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@qpid.apache.org by Keith W <ke...@gmail.com> on 2012/05/24 09:59:32 UTC

Apache CI Python against CPP (Qpid-Python-Cpp-Test) tests failing for the last few days

Hello

The Qpid-Python-Cpp-Test started to fail on May 21st.   Many tests are
failing and others are begin skipped.  I turned on DEBUG logging from the
Python tests, and logging from the CPP Broker, but it is no obvious to me
what has gone wrong.    The Python/Java tests continue to pass.

Totals: 506 tests, 197 passed, 302 skipped, 0 ignored, 7 failed

The first failing test is:

qpid_tests.broker_0_10.alternate_exchange.AlternateExchangeTests. \
    test_del_alternate_to_exchange ...................................... fail
Error during test:  Traceback (most recent call last):
    File "/home/jenkins/jenkins-slave/workspace/Qpid-Python-Cpp-Test/trunk/qpid/python/qpid-python-test",
line 340, in run
      phase()
    File "/home/jenkins/jenkins-slave/workspace/Qpid-Python-Cpp-Test/trunk/qpid/tests/src/py/qpid_tests/broker_0_10/alternate_exchange.py",
line 186, in test_del_alternate_to_exchange
      session.exchange_declare(exchange="alt1", type="direct")
    File "/home/jenkins/jenkins-slave/workspace/Qpid-Python-Cpp-Test/trunk/qpid/python/qpid/generator.py",
line 25, in <lambda>
      method = lambda self, *args, **kwargs: self.invoke(op, args, kwargs)
    File "/home/jenkins/jenkins-slave/workspace/Qpid-Python-Cpp-Test/trunk/qpid/python/qpid/session.py",
line 138, in invoke
      return self.do_invoke(op, args, kwargs)
    File "/home/jenkins/jenkins-slave/workspace/Qpid-Python-Cpp-Test/trunk/qpid/python/qpid/session.py",
line 183, in do_invoke
      self.sync(self.timeout)
    File "/home/jenkins/jenkins-slave/workspace/Qpid-Python-Cpp-Test/trunk/qpid/python/qpid/session.py",
line 103, in sync
      raise SessionException(self.error())
  SessionException: (None, '[Errno 104] Connection reset by peer')

Test .out
https://builds.apache.org/view/M-R/view/Qpid/job/Qpid-Python-Cpp-Test/ws/trunk/qpid/python/build/results/results.out/*view*/

The Broker is logging only:

2012-05-24 04:08:11 notice SASL disabled: No Authentication Performed
2012-05-24 04:08:11 notice Listening on TCP/TCP6 port 35672
2012-05-24 04:08:11 notice Broker running
2012-05-24 04:08:13 error Execution exception: not-allowed: Exchange
declared with alternate-exchange <nonexistent>, requested alt1
(qpid/broker/SessionAdapter.cpp:133)
2012-05-24 04:08:14 error Connection 127.0.0.1:35672-127.0.0.1:44003
No protocol received closing

Broker logs:

https://builds.apache.org/view/M-R/view/Qpid/job/Qpid-Python-Cpp-Test/ws/trunk/qpid/python/build/results/qpidd.log/*view*/

Does someone know what has happened?

cheers, Keith

Re: Apache CI Python against CPP (Qpid-Python-Cpp-Test) tests failing for the last few days

Posted by Gordon Sim <gs...@redhat.com>.
On 05/25/2012 05:08 PM, Andrew Stitcher wrote:
> On Fri, 2012-05-25 at 16:39 +0100, Gordon Sim wrote:
>> ...
>
>> FWIW I really don't like that code.
>
> Neither do I, possibly for different reasons.
>
>>
>> It doesn't actually protect from badly behaved client code anyway, only
>> from one specific case. Provided you send a valid AMQP header you can
>> still use up all the connections without doing anything further and
>> without authenticating.
>
> Fair point. It doesn't protect you from all badly behaved code, but it
> is a step wise improvement from the previous state.

I don't think it is, really. It means instead of using a telnet client 
to use up connections you need to write a few lines of code. What 
benefit do you see from that?

>> Rather than having a maximum time to negotiate the protocol version what
>> is really needed is a maximum time to authenticate.
>
> I agree.
>
> I'll see if there is an obviously equally safe place to detect we've
> authenticated.
>
> Of course this still won't protect you from a massive DDoS.

Agreed.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org


Re: Apache CI Python against CPP (Qpid-Python-Cpp-Test) tests failing for the last few days

Posted by Andrew Stitcher <as...@redhat.com>.
On Fri, 2012-05-25 at 16:39 +0100, Gordon Sim wrote:
> ...

> FWIW I really don't like that code.

Neither do I, possibly for different reasons.

> 
> It doesn't actually protect from badly behaved client code anyway, only 
> from one specific case. Provided you send a valid AMQP header you can 
> still use up all the connections without doing anything further and 
> without authenticating.

Fair point. It doesn't protect you from all badly behaved code, but it
is a step wise improvement from the previous state.

> 
> Rather than having a maximum time to negotiate the protocol version what 
> is really needed is a maximum time to authenticate.

I agree.

I'll see if there is an obviously equally safe place to detect we've
authenticated.

Of course this still won't protect you from a massive DDoS.

Andrew



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org


Re: Apache CI Python against CPP (Qpid-Python-Cpp-Test) tests failing for the last few days

Posted by Gordon Sim <gs...@redhat.com>.
On 05/25/2012 03:59 PM, Andrew Stitcher wrote:
> On Thu, 2012-05-24 at 08:59 +0100, Keith W wrote:
>> Hello
>>
>> The Qpid-Python-Cpp-Test started to fail on May 21st.   Many tests are
>> failing and others are begin skipped.  I turned on DEBUG logging from the
>> Python tests, and logging from the CPP Broker, but it is no obvious to me
>> what has gone wrong.    The Python/Java tests continue to pass.
>> ...
>
>> The Broker is logging only:
>>
>> 2012-05-24 04:08:11 notice SASL disabled: No Authentication Performed
>> 2012-05-24 04:08:11 notice Listening on TCP/TCP6 port 35672
>> 2012-05-24 04:08:11 notice Broker running
>> 2012-05-24 04:08:13 error Execution exception: not-allowed: Exchange
>> declared with alternate-exchange<nonexistent>, requested alt1
>> (qpid/broker/SessionAdapter.cpp:133)
>> 2012-05-24 04:08:14 error Connection 127.0.0.1:35672-127.0.0.1:44003
>> No protocol received closing
>
> This message comes from the recent "DoS prevention code" (well badly
> behaved client code prevention really) - If a client has not sent the
> initial "AMQP" + version bytes within a reasonable time (2s by default)
> a connection is closed peremptorily.

FWIW I really don't like that code.

It doesn't actually protect from badly behaved client code anyway, only 
from one specific case. Provided you send a valid AMQP header you can 
still use up all the connections without doing anything further and 
without authenticating.

Rather than having a maximum time to negotiate the protocol version what 
is really needed is a maximum time to authenticate.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org


Re: Apache CI Python against CPP (Qpid-Python-Cpp-Test) tests failing for the last few days

Posted by Andrew Stitcher <as...@redhat.com>.
On Thu, 2012-05-24 at 08:59 +0100, Keith W wrote:
> Hello
> 
> The Qpid-Python-Cpp-Test started to fail on May 21st.   Many tests are
> failing and others are begin skipped.  I turned on DEBUG logging from the
> Python tests, and logging from the CPP Broker, but it is no obvious to me
> what has gone wrong.    The Python/Java tests continue to pass.
> ...

> The Broker is logging only:
> 
> 2012-05-24 04:08:11 notice SASL disabled: No Authentication Performed
> 2012-05-24 04:08:11 notice Listening on TCP/TCP6 port 35672
> 2012-05-24 04:08:11 notice Broker running
> 2012-05-24 04:08:13 error Execution exception: not-allowed: Exchange
> declared with alternate-exchange <nonexistent>, requested alt1
> (qpid/broker/SessionAdapter.cpp:133)
> 2012-05-24 04:08:14 error Connection 127.0.0.1:35672-127.0.0.1:44003
> No protocol received closing

This message comes from the recent "DoS prevention code" (well badly
behaved client code prevention really) - If a client has not sent the
initial "AMQP" + version bytes within a reasonable time (2s by default)
a connection is closed peremptorily. So somehow in this test the test
client does not succeed in completing protocol negotiation within 2s.

I suppose this could also happen if the machine is highly loaded too.

Andrew

  


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org