You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@qpid.apache.org by Adel Boutros <Ad...@live.com> on 2017/01/04 11:27:17 UTC
[Proton 0.16.0] [Solaris] Unit tests hanging when destroying
connection_driver
Hello,
I am compiling Proton 0.16.0 on Solaris using SunStudio compiler. The compiler is successful however unit tests are hanging (cpp_thread_safe_test, cpp_container_test, c-refcount-tests, c-reactor-tests, , cpp-example-container).
All tests except cpp-example-container seem to have the same failure. I am willing to debug this further if needed but maybe you guys already have some hints at why it might be hanging (I tend to think there is an object not added or destroyed correctly and this its reference is not being incremented or decremented).
Debugging analysis so far:
* c-refcount-tests: The hanging is occurring when calling the first drain in test_collector_connection_transport ( drain(collector) )
* cpp_thread_safe_test: The hanging is occurring when calling the last assert in test_new ( ASSERT_EQUAL(1, pn_refcount(c)) )
Failures with stack (pstack $PID | c++filt)
=================================
c-refcounts-tests
------------------------
fffffd7fff2efd87 pn_collector_next () + 2f
00000000004023e2 drain () + 1e
0000000000402464 test_collector_connection_transport () + 74
00000000004027f5 main () + 45
00000000004014bc ???????? ()
c-reactor-tests
------------------------
fffffd7fff2ef8f1 pn_collector_drain () + 3d
fffffd7fff2efbd7 pn_collector_release () + 57
fffffd7fff30025a pn_reactor_run () + 6e
0000000000402b4b test_reactor_event_root () + 63
00000000004038d2 main () + e
00000000004023ac ???????? ()
c-event-tests
------------------------
fffffd7fff2efda1 pn_collector_next () + 49
fffffd7fff2ef8f9 pn_collector_drain () + 45
fffffd7fff2ef9a5 pn_collector_finalize () + 11
fffffd7fff2efa7d pn_collector_finalize_cast () + 9
fffffd7fff2ded19 pn_class_decref () + 6d
fffffd7fff2dedee pn_class_free () + aa
fffffd7fff2df1c7 pn_free () + 13
00000000004011ad test_collector_put () + 65
00000000004014ee main () + e
0000000000400fbc ???????? ()
cpp_thread_safe_test
------------------------
fffffd7fff27fd95 pn_collector_next () + 3d
fffffd7fff27f8f9 pn_collector_drain () + 45
fffffd7fff27fbd7 pn_collector_release () + 57
fffffd7fff27fb68 pn_collector_free () + 38
fffffd7fff27a448 pn_connection_driver_destroy () + 44
fffffd7fff3139d9 proton::io::connection_driver::~connection_driver #Nvariant 1() () + d
0000000000403c79 void __unnamed_KkRMcLMNbYEgR::test_new() () + 4a5
00000000004048f2 main () + 3a
000000000040362c ???????? ()
cpp_container_test
------------------------
fffffd7fff27fd95 pn_collector_next () + 3d
fffffd7fff27f8f9 pn_collector_drain () + 45
fffffd7fff27fbd7 pn_collector_release () + 57
fffffd7fff29025a pn_reactor_run () + 6e
fffffd7fff31a674 void proton::reactor::run() () + c
fffffd7fff30aa97 void proton::container::impl::run() () + 17
fffffd7fff308560 void proton::container::run() () + c
00000000004061b1 int __unnamed_KkRMcLdNbYEzd::test_container_vhost() () + 1ad
0000000000407bb8 main () + 18
0000000000405bec ???????? ()
cpp-example-container
------------------------
fffffd7ffef12dca waitid (0, 2099, fffffd7fffdfe3b0, 3)
fffffd7ffef00f5b waitpid () + 7b
fffffd7fff2c7b76 posix_waitpid () + 46
fffffd7fff285aa3 PyEval_EvalFrameEx () + 5643
fffffd7fff287eba PyEval_EvalCodeEx () + 78a
fffffd7fff2863bc PyEval_EvalFrameEx () + 5f5c
fffffd7fff286e50 PyEval_EvalFrameEx () + 69f0
fffffd7fff287eba PyEval_EvalCodeEx () + 78a
fffffd7fff2863bc PyEval_EvalFrameEx () + 5f5c
fffffd7fff287eba PyEval_EvalCodeEx () + 78a
fffffd7fff288066 PyEval_EvalCode () + 36
fffffd7fff2aa8df PyRun_FileExFlags () + 9f
fffffd7fff2ab800 PyRun_SimpleFileExFlags () + 1c0
fffffd7fff2c0710 Py_Main () + a90
0000000000400a0c _start () + 6c
Regards,
Adel
Re: [Proton 0.16.0] [Solaris] Unit tests hanging when destroying
connection_driver
Posted by Andrew Stitcher <as...@redhat.com>.
On Mon, 2017-01-09 at 11:37 -0500, Alan Conway wrote:
> ...
> They're not used often because people tend to be too busy to think
> about struct packing, but they are an important perf tuning tool. It
> can be the difference between a single bool packing into 1 bit at
> best�
> vs 64 at worst. That can easily double the size of a small structure
> that gets allocated by the millions, or force cache misses on the
> critical path. It often is not very important but sometimes it really
> is.
>
While this may be true in some cases, more often there are negative
effects of bitfields - namely that they cannot be atomic, as the usual
limit to atomicity in modern architectures is the word (or at least the
byte).
This means that every bitfield access in memory must be read-modify-
write (if the compiler can keep the packed struct in registers then
this gets much better).
It also means that thread atomic bools can't be single bits anyway -
they need a mutex as well. So they may as well just use the minimal
atomic int size.
So as a general rule, unless you have really large numbers of bools in
your struct - like more than 64 of them (which might be more than a
cache line on some archs if you map bools -> bytes) I think it's better
to use non bitfield bools.
However like all general advice it is subject to actually measuring the
thing you care about (performance or memory or whatever).
My previous experience has been that the major use of bitfields has
been in mapping low level hardware register layouts into structs.
Oviously in conjunction with controlling packing and big vs little
endian knowledge.
Andrew
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: [Proton 0.16.0] [Solaris] Unit tests hanging when destroying
connection_driver
Posted by Adel Boutros <Ad...@live.com>.
My mail went too fast..
Output
-----------
* Bool case
---> Bool: dummy=1
---> Bool: headReturned = 0
* Int case
---> Int: dummy=1
---> Int: headReturned = 1
________________________________
From: Adel Boutros
Sent: Monday, January 9, 2017 6:17 PM
To: users@qpid.apache.org
Subject: Re: [Proton 0.16.0] [Solaris] Unit tests hanging when destroying connection_driver
This bug is actually fixed on SunStudio 12.4. However validating a new compiler takes some time. So we have to stick with 12.1 for now (We hope to be able to switch to the newer one in the coming months).
Of course a compiler bug shouldn't be a cause for not using the best practices.
There are many workarounds possible. It's just about finding the best one.
I confirm that int instead of bool works perfectly.
--------------
Bool test
--------------
#include <stdio.h>
#include <stdbool.h>
struct myStruct {
bool dummy:1;
bool headReturned:1;
};
int main() {
struct myStruct s;
s.headReturned = true;
fprintf(stderr, "---> Bool: dummy=%d\n", s.dummy);
fprintf(stderr, "---> Bool: headReturned = %d\n", s.headReturned);
return 0;
}
--------------
Int test
---------------
#include <stdio.h>
#include <stdbool.h>
struct myStruct {
int dummy:1;
int headReturned:1;
};
int main() {
struct myStruct s;
s.headReturned = true;
fprintf(stderr, "---> Int: dummy=%d\n", s.dummy);
fprintf(stderr, "---> Int: headReturned = %d\n", s.headReturned);
return 0;
}
-----------
output
-----------
* Bool case
---> dummy=1
---> headReturned = 1
Regards,
Adel
________________________________
From: Alan Conway <ac...@redhat.com>
Sent: Monday, January 9, 2017 6:10:20 PM
To: users@qpid.apache.org
Subject: Re: [Proton 0.16.0] [Solaris] Unit tests hanging when destroying connection_driver
On Mon, 2017-01-09 at 16:51 +0000, Adel Boutros wrote:
> Hello Alan,
>
>
> I had provided in one of my previous emails in this discussion a
> simple test case which fails. What we noticed is that this failure
> occurs when we have 2 bool fields. It doesn't occur with a single
> bool.
>
> It seems that on Solaris with SunStudio 12.1 the wrong byte is
> changed when setting one of the bools when 2 consecutive bool fields
> have bitfields activated.
>
>
> If you really find bit fields important, we can try to separate the 2
> bools (Having another field between them) and it should work.
I'm not worried about this case, your fix is fine. I'm concerned about
the broader question of ceasing to use bit fields always and forever in
my professional life case Solaris explodes :)
Separating bitfields defeats the purpose which is to get the compiler
to pack multiple boolean flags or very small integer values into a
single byte/word/long word.
Is the Solaris problem just with bool? Traditional C use would be
int flag:1;
does that work on Solaris? I'm guessing it does.
I used bool for stylistic reasons, but I'll switch to the more
tranditional int now that I know it's a possible portability issue.
bool is a recent introduction to C and the degree to which bool is
"like" and "not like" the traditional int and char types has caused no
end of confusion.
>
>
> Regards,
>
> Adel
>
> ________________________________
> From: Alan Conway <ac...@redhat.com>
> Sent: Monday, January 9, 2017 5:37:27 PM
> To: users@qpid.apache.org
> Subject: Re: [Proton 0.16.0] [Solaris] Unit tests hanging when
> destroying connection_driver
>
> On Wed, 2017-01-04 at 10:39 -0500, Andrew Stitcher wrote:
> > On Wed, 2017-01-04 at 14:47 +0000, Adel Boutros wrote:
> > > I found the issue!!
> > >
> > >
> > > It is a Solaris compiler bug:
> > >
> > > Actually Solaris doesn't handle correctly bit fields in C. This
> > > is
> > > the case of the struct pn_collector_t (bool head_returned:1).
>
> Huh? What?? Bitfields have been a standard part of C since forever. I
> am astonished.
>
> > We probably should just never be using bifields in this case - it
> > is
> > pretty unusual in my experience (even if it seems like it would
> > save
> > space in the struct).
> >
>
> They're not used often because people tend to be too busy to think
> about struct packing, but they are an important perf tuning tool. It
> can be the difference between a single bool packing into 1 bit at
> best
> vs 64 at worst. That can easily double the size of a small structure
> that gets allocated by the millions, or force cache misses on the
> critical path. It often is not very important but sometimes it really
> is.
>
> > So I, for one, would definitely accept a patch to remove all uses
> > of
> > bool bitfields.
>
> If existing bit fields can be removed without performance impact to
> fix
> a bug then I'm OK with that too, but a blanket ban on bit fields
> seems
> unreasonable. Can we narrow the scope of the actual Solaris problem
> to
> something smaller than "bitfields are broken, don't ever use them"?
> I'm
> pretty sure that can't be the case.
>
> Cheers,
> Alan.
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: [Proton 0.16.0] [Solaris] Unit tests hanging when destroying
connection_driver
Posted by Adel Boutros <Ad...@live.com>.
This bug is actually fixed on SunStudio 12.4. However validating a new compiler takes some time. So we have to stick with 12.1 for now (We hope to be able to switch to the newer one in the coming months).
Of course a compiler bug shouldn't be a cause for not using the best practices.
There are many workarounds possible. It's just about finding the best one.
I confirm that int instead of bool works perfectly.
--------------
Bool test
--------------
#include <stdio.h>
#include <stdbool.h>
struct myStruct {
bool dummy:1;
bool headReturned:1;
};
int main() {
struct myStruct s;
s.headReturned = true;
fprintf(stderr, "---> Bool: dummy=%d\n", s.dummy);
fprintf(stderr, "---> Bool: headReturned = %d\n", s.headReturned);
return 0;
}
--------------
Int test
---------------
#include <stdio.h>
#include <stdbool.h>
struct myStruct {
int dummy:1;
int headReturned:1;
};
int main() {
struct myStruct s;
s.headReturned = true;
fprintf(stderr, "---> Int: dummy=%d\n", s.dummy);
fprintf(stderr, "---> Int: headReturned = %d\n", s.headReturned);
return 0;
}
-----------
output
-----------
* Bool case
---> dummy=1
---> headReturned = 1
Regards,
Adel
________________________________
From: Alan Conway <ac...@redhat.com>
Sent: Monday, January 9, 2017 6:10:20 PM
To: users@qpid.apache.org
Subject: Re: [Proton 0.16.0] [Solaris] Unit tests hanging when destroying connection_driver
On Mon, 2017-01-09 at 16:51 +0000, Adel Boutros wrote:
> Hello Alan,
>
>
> I had provided in one of my previous emails in this discussion a
> simple test case which fails. What we noticed is that this failure
> occurs when we have 2 bool fields. It doesn't occur with a single
> bool.
>
> It seems that on Solaris with SunStudio 12.1 the wrong byte is
> changed when setting one of the bools when 2 consecutive bool fields
> have bitfields activated.
>
>
> If you really find bit fields important, we can try to separate the 2
> bools (Having another field between them) and it should work.
I'm not worried about this case, your fix is fine. I'm concerned about
the broader question of ceasing to use bit fields always and forever in
my professional life case Solaris explodes :)
Separating bitfields defeats the purpose which is to get the compiler
to pack multiple boolean flags or very small integer values into a
single byte/word/long word.
Is the Solaris problem just with bool? Traditional C use would be
int flag:1;
does that work on Solaris? I'm guessing it does.
I used bool for stylistic reasons, but I'll switch to the more
tranditional int now that I know it's a possible portability issue.
bool is a recent introduction to C and the degree to which bool is
"like" and "not like" the traditional int and char types has caused no
end of confusion.
>
>
> Regards,
>
> Adel
>
> ________________________________
> From: Alan Conway <ac...@redhat.com>
> Sent: Monday, January 9, 2017 5:37:27 PM
> To: users@qpid.apache.org
> Subject: Re: [Proton 0.16.0] [Solaris] Unit tests hanging when
> destroying connection_driver
>
> On Wed, 2017-01-04 at 10:39 -0500, Andrew Stitcher wrote:
> > On Wed, 2017-01-04 at 14:47 +0000, Adel Boutros wrote:
> > > I found the issue!!
> > >
> > >
> > > It is a Solaris compiler bug:
> > >
> > > Actually Solaris doesn't handle correctly bit fields in C. This
> > > is
> > > the case of the struct pn_collector_t (bool head_returned:1).
>
> Huh? What?? Bitfields have been a standard part of C since forever. I
> am astonished.
>
> > We probably should just never be using bifields in this case - it
> > is
> > pretty unusual in my experience (even if it seems like it would
> > save
> > space in the struct).
> >
>
> They're not used often because people tend to be too busy to think
> about struct packing, but they are an important perf tuning tool. It
> can be the difference between a single bool packing into 1 bit at
> best
> vs 64 at worst. That can easily double the size of a small structure
> that gets allocated by the millions, or force cache misses on the
> critical path. It often is not very important but sometimes it really
> is.
>
> > So I, for one, would definitely accept a patch to remove all uses
> > of
> > bool bitfields.
>
> If existing bit fields can be removed without performance impact to
> fix
> a bug then I'm OK with that too, but a blanket ban on bit fields
> seems
> unreasonable. Can we narrow the scope of the actual Solaris problem
> to
> something smaller than "bitfields are broken, don't ever use them"?
> I'm
> pretty sure that can't be the case.
>
> Cheers,
> Alan.
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: [Proton 0.16.0] [Solaris] Unit tests hanging when destroying
connection_driver
Posted by Alan Conway <ac...@redhat.com>.
On Mon, 2017-01-09 at 16:51 +0000, Adel Boutros wrote:
> Hello Alan,
>
>
> I had provided in one of my previous emails in this discussion a
> simple test case which fails. What we noticed is that this failure
> occurs when we have 2 bool fields. It doesn't occur with a single
> bool.
>
> It seems that on Solaris with SunStudio 12.1 the wrong byte is
> changed when setting one of the bools when 2 consecutive bool fields
> have bitfields activated.
>
>
> If you really find bit fields important, we can try to separate the 2
> bools (Having another field between them) and it should work.
I'm not worried about this case, your fix is fine. I'm concerned about
the broader question of ceasing to use bit fields always and forever in
my professional life case Solaris explodes :)
Separating bitfields defeats the purpose which is to get the compiler
to pack multiple boolean flags or very small integer values into a
single byte/word/long word.
Is the Solaris problem just with bool? Traditional C use would be
int flag:1;
does that work on Solaris? I'm guessing it does.
I used bool for stylistic reasons, but I'll switch to the more
tranditional int now that I know it's a possible portability issue.
bool is a recent introduction to C and the degree to which bool is
"like" and "not like" the traditional int and char types has caused no
end of confusion.
>
>
> Regards,
>
> Adel
>
> ________________________________
> From: Alan Conway <ac...@redhat.com>
> Sent: Monday, January 9, 2017 5:37:27 PM
> To: users@qpid.apache.org
> Subject: Re: [Proton 0.16.0] [Solaris] Unit tests hanging when
> destroying connection_driver
>
> On Wed, 2017-01-04 at 10:39 -0500, Andrew Stitcher wrote:
> > On Wed, 2017-01-04 at 14:47 +0000, Adel Boutros wrote:
> > > I found the issue!!
> > >
> > >
> > > It is a Solaris compiler bug:
> > >
> > > Actually Solaris doesn't handle correctly bit fields in C. This
> > > is
> > > the case of the struct pn_collector_t (bool head_returned:1).
>
> Huh? What?? Bitfields have been a standard part of C since forever. I
> am astonished.
>
> > We probably should just never be using bifields in this case - it
> > is
> > pretty unusual in my experience (even if it seems like it would
> > save
> > space in the struct).
> >
>
> They're not used often because people tend to be too busy to think
> about struct packing, but they are an important perf tuning tool. It
> can be the difference between a single bool packing into 1 bit at
> best
> vs 64 at worst. That can easily double the size of a small structure
> that gets allocated by the millions, or force cache misses on the
> critical path. It often is not very important but sometimes it really
> is.
>
> > So I, for one, would definitely accept a patch to remove all uses
> > of
> > bool bitfields.
>
> If existing bit fields can be removed without performance impact to
> fix
> a bug then I'm OK with that too, but a blanket ban on bit fields
> seems
> unreasonable. Can we narrow the scope of the actual Solaris problem
> to
> something smaller than "bitfields are broken, don't ever use them"?
> I'm
> pretty sure that can't be the case.
>
> Cheers,
> Alan.
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: [Proton 0.16.0] [Solaris] Unit tests hanging when destroying
connection_driver
Posted by Adel Boutros <Ad...@live.com>.
Hello Alan,
I had provided in one of my previous emails in this discussion a simple test case which fails. What we noticed is that this failure occurs when we have 2 bool fields. It doesn't occur with a single bool.
It seems that on Solaris with SunStudio 12.1 the wrong byte is changed when setting one of the bools when 2 consecutive bool fields have bitfields activated.
If you really find bit fields important, we can try to separate the 2 bools (Having another field between them) and it should work.
Regards,
Adel
________________________________
From: Alan Conway <ac...@redhat.com>
Sent: Monday, January 9, 2017 5:37:27 PM
To: users@qpid.apache.org
Subject: Re: [Proton 0.16.0] [Solaris] Unit tests hanging when destroying connection_driver
On Wed, 2017-01-04 at 10:39 -0500, Andrew Stitcher wrote:
> On Wed, 2017-01-04 at 14:47 +0000, Adel Boutros wrote:
> > I found the issue!!
> >
> >
> > It is a Solaris compiler bug:
> >
> > Actually Solaris doesn't handle correctly bit fields in C. This is
> > the case of the struct pn_collector_t (bool head_returned:1).
>
Huh? What?? Bitfields have been a standard part of C since forever. I
am astonished.
> We probably should just never be using bifields in this case - it is
> pretty unusual in my experience (even if it seems like it would save
> space in the struct).
>
They're not used often because people tend to be too busy to think
about struct packing, but they are an important perf tuning tool. It
can be the difference between a single bool packing into 1 bit at best
vs 64 at worst. That can easily double the size of a small structure
that gets allocated by the millions, or force cache misses on the
critical path. It often is not very important but sometimes it really
is.
> So I, for one, would definitely accept a patch to remove all uses of
> bool bitfields.
If existing bit fields can be removed without performance impact to fix
a bug then I'm OK with that too, but a blanket ban on bit fields seems
unreasonable. Can we narrow the scope of the actual Solaris problem to
something smaller than "bitfields are broken, don't ever use them"? I'm
pretty sure that can't be the case.
Cheers,
Alan.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: [Proton 0.16.0] [Solaris] Unit tests hanging when destroying
connection_driver
Posted by Alan Conway <ac...@redhat.com>.
On Wed, 2017-01-04 at 10:39 -0500, Andrew Stitcher wrote:
> On Wed, 2017-01-04 at 14:47 +0000, Adel Boutros wrote:
> > I found the issue!!
> >
> >
> > It is a Solaris compiler bug:
> >
> > Actually Solaris doesn't handle correctly bit fields in C. This is
> > the case of the struct pn_collector_t (bool head_returned:1).
>
Huh? What?? Bitfields have been a standard part of C since forever. I
am astonished.
> We probably should just never be using bifields in this case - it is
> pretty unusual in my experience (even if it seems like it would save
> space in the struct).
>
They're not used often because people tend to be too busy to think
about struct packing, but they are an important perf tuning tool. It
can be the difference between a single bool packing into 1 bit at best
vs 64 at worst. That can easily double the size of a small structure
that gets allocated by the millions, or force cache misses on the
critical path. It often is not very important but sometimes it really
is.
> So I, for one, would definitely accept a patch to remove all uses of
> bool bitfields.
If existing bit fields can be removed without performance impact to fix
a bug then I'm OK with that too, but a blanket ban on bit fields seems
unreasonable. Can we narrow the scope of the actual Solaris problem to
something smaller than "bitfields are broken, don't ever use them"? I'm
pretty sure that can't be the case.
Cheers,
Alan.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: [Proton 0.16.0] [Solaris] Unit tests hanging when destroying
connection_driver
Posted by Alan Conway <ac...@redhat.com>.
On Thu, 2017-01-05 at 08:35 +0000, Adel Boutros wrote:
> Hello Andrew,
>
>
> Issue reported and patch provided at head of master here: https://iss
> ues.apache.org/jira/browse/PROTON-1382
>
> 2 problems fixed:
>
> * Remove bit fields initialization to fix hanging tests on Solaris
>
> * Add missing include to fix incomplete type error on Solaris
> (message.cpp, Error: Function "proton::link::error() const" with
> incomplete return type "proton::error_condition", compiler cannot
> generate virtual table in this situation..)
>
>
> Could this be considered as a regression because Proton was compiling
> on Solaris at 0.14.0?
Yep. I made the change because I was touching the struct anyway, in
this particular case the use of bit-fields makes no difference so I'm
fine with your fix.
> Would it be possible to consider a 0.16.1 release in that case if
> 0.17.0 is a bit far from being released?
Sounds reasonable to me.
>
> Regards,
>
> Adel
>
> ________________________________
> From: Andrew Stitcher <as...@redhat.com>
> Sent: Wednesday, January 4, 2017 4:39:27 PM
> To: users@qpid.apache.org
> Subject: Re: [Proton 0.16.0] [Solaris] Unit tests hanging when
> destroying connection_driver
>
> On Wed, 2017-01-04 at 14:47 +0000, Adel Boutros wrote:
> > I found the issue!!
> >
> >
> > It is a Solaris compiler bug:
> >
> > Actually Solaris doesn't handle correctly bit fields in C. This is
> > the case of the struct pn_collector_t (bool head_returned:1).
>
> We probably should just never be using bifields in this case - it is
> pretty unusual in my experience (even if it seems like it would save
> space in the struct).
>
> So I, for one, would definitely accept a patch to remove all uses of
> bool bitfields.
>
> Andrew
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: [Proton 0.16.0] [Solaris] Unit tests hanging when destroying
connection_driver
Posted by Adel Boutros <Ad...@live.com>.
Hello Andrew,
Issue reported and patch provided at head of master here: https://issues.apache.org/jira/browse/PROTON-1382
2 problems fixed:
* Remove bit fields initialization to fix hanging tests on Solaris
* Add missing include to fix incomplete type error on Solaris (message.cpp, Error: Function "proton::link::error() const" with incomplete return type "proton::error_condition", compiler cannot generate virtual table in this situation..)
Could this be considered as a regression because Proton was compiling on Solaris at 0.14.0?
Would it be possible to consider a 0.16.1 release in that case if 0.17.0 is a bit far from being released?
Regards,
Adel
________________________________
From: Andrew Stitcher <as...@redhat.com>
Sent: Wednesday, January 4, 2017 4:39:27 PM
To: users@qpid.apache.org
Subject: Re: [Proton 0.16.0] [Solaris] Unit tests hanging when destroying connection_driver
On Wed, 2017-01-04 at 14:47 +0000, Adel Boutros wrote:
> I found the issue!!
>
>
> It is a Solaris compiler bug:
>
> Actually Solaris doesn't handle correctly bit fields in C. This is
> the case of the struct pn_collector_t (bool head_returned:1).
We probably should just never be using bifields in this case - it is
pretty unusual in my experience (even if it seems like it would save
space in the struct).
So I, for one, would definitely accept a patch to remove all uses of
bool bitfields.
Andrew
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: [Proton 0.16.0] [Solaris] Unit tests hanging when destroying
connection_driver
Posted by Andrew Stitcher <as...@redhat.com>.
On Wed, 2017-01-04 at 14:47 +0000, Adel Boutros wrote:
> I found the issue!!
>
>
> It is a Solaris compiler bug:
>
> Actually Solaris doesn't handle correctly bit fields in C. This is
> the case of the struct pn_collector_t (bool head_returned:1).
We probably should just never be using bifields in this case - it is
pretty unusual in my experience (even if it seems like it would save
space in the struct).
So I, for one, would definitely accept a patch to remove all uses of
bool bitfields.
Andrew
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: [Proton 0.16.0] [Solaris] Unit tests hanging when destroying
connection_driver
Posted by Adel Boutros <Ad...@live.com>.
I found the issue!!
It is a Solaris compiler bug:
Actually Solaris doesn't handle correctly bit fields in C. This is the case of the struct pn_collector_t (bool head_returned:1).
This caused the "collector->head_returned" to never be set to true in pn_collector_next.
By removing the ":1", all the tests are green.
I will submit a pretty patch for the Solaris case once I have it finished. Should I expect the same problem in other classes/structs?
Simple code to reproduce
----------------------------------
#include <stdio.h>
#include <stdbool.h>
struct myStruct {
bool dummy:1;
bool headReturned:1;
};
main() {
struct myStruct s;
s.headReturned = true;
fprintf(stderr, "---> headReturned = %d\n", s.headReturned);
}
Output : ---> headReturned = 0
Regards,
Adel
________________________________
From: Adel Boutros <Ad...@live.com>
Sent: Wednesday, January 4, 2017 12:31:53 PM
To: users@qpid.apache.org
Subject: Re: [Proton 0.16.0] [Solaris] Unit tests hanging when destroying connection_driver
PS: The title of the mail is a bit misleading. It can be resumed as follows "Unit tests hanging in proton-c"
________________________________
From: Adel Boutros <Ad...@live.com>
Sent: Wednesday, January 4, 2017 12:27:17 PM
To: users@qpid.apache.org
Subject: [Proton 0.16.0] [Solaris] Unit tests hanging when destroying connection_driver
Hello,
I am compiling Proton 0.16.0 on Solaris using SunStudio compiler. The compiler is successful however unit tests are hanging (cpp_thread_safe_test, cpp_container_test, c-refcount-tests, c-reactor-tests, , cpp-example-container).
All tests except cpp-example-container seem to have the same failure. I am willing to debug this further if needed but maybe you guys already have some hints at why it might be hanging (I tend to think there is an object not added or destroyed correctly and this its reference is not being incremented or decremented).
Debugging analysis so far:
* c-refcount-tests: The hanging is occurring when calling the first drain in test_collector_connection_transport ( drain(collector) )
* cpp_thread_safe_test: The hanging is occurring when calling the last assert in test_new ( ASSERT_EQUAL(1, pn_refcount(c)) )
Failures with stack (pstack $PID | c++filt)
=================================
c-refcounts-tests
------------------------
fffffd7fff2efd87 pn_collector_next () + 2f
00000000004023e2 drain () + 1e
0000000000402464 test_collector_connection_transport () + 74
00000000004027f5 main () + 45
00000000004014bc ???????? ()
c-reactor-tests
------------------------
fffffd7fff2ef8f1 pn_collector_drain () + 3d
fffffd7fff2efbd7 pn_collector_release () + 57
fffffd7fff30025a pn_reactor_run () + 6e
0000000000402b4b test_reactor_event_root () + 63
00000000004038d2 main () + e
00000000004023ac ???????? ()
c-event-tests
------------------------
fffffd7fff2efda1 pn_collector_next () + 49
fffffd7fff2ef8f9 pn_collector_drain () + 45
fffffd7fff2ef9a5 pn_collector_finalize () + 11
fffffd7fff2efa7d pn_collector_finalize_cast () + 9
fffffd7fff2ded19 pn_class_decref () + 6d
fffffd7fff2dedee pn_class_free () + aa
fffffd7fff2df1c7 pn_free () + 13
00000000004011ad test_collector_put () + 65
00000000004014ee main () + e
0000000000400fbc ???????? ()
cpp_thread_safe_test
------------------------
fffffd7fff27fd95 pn_collector_next () + 3d
fffffd7fff27f8f9 pn_collector_drain () + 45
fffffd7fff27fbd7 pn_collector_release () + 57
fffffd7fff27fb68 pn_collector_free () + 38
fffffd7fff27a448 pn_connection_driver_destroy () + 44
fffffd7fff3139d9 proton::io::connection_driver::~connection_driver #Nvariant 1() () + d
0000000000403c79 void __unnamed_KkRMcLMNbYEgR::test_new() () + 4a5
00000000004048f2 main () + 3a
000000000040362c ???????? ()
cpp_container_test
------------------------
fffffd7fff27fd95 pn_collector_next () + 3d
fffffd7fff27f8f9 pn_collector_drain () + 45
fffffd7fff27fbd7 pn_collector_release () + 57
fffffd7fff29025a pn_reactor_run () + 6e
fffffd7fff31a674 void proton::reactor::run() () + c
fffffd7fff30aa97 void proton::container::impl::run() () + 17
fffffd7fff308560 void proton::container::run() () + c
00000000004061b1 int __unnamed_KkRMcLdNbYEzd::test_container_vhost() () + 1ad
0000000000407bb8 main () + 18
0000000000405bec ???????? ()
cpp-example-container
------------------------
fffffd7ffef12dca waitid (0, 2099, fffffd7fffdfe3b0, 3)
fffffd7ffef00f5b waitpid () + 7b
fffffd7fff2c7b76 posix_waitpid () + 46
fffffd7fff285aa3 PyEval_EvalFrameEx () + 5643
fffffd7fff287eba PyEval_EvalCodeEx () + 78a
fffffd7fff2863bc PyEval_EvalFrameEx () + 5f5c
fffffd7fff286e50 PyEval_EvalFrameEx () + 69f0
fffffd7fff287eba PyEval_EvalCodeEx () + 78a
fffffd7fff2863bc PyEval_EvalFrameEx () + 5f5c
fffffd7fff287eba PyEval_EvalCodeEx () + 78a
fffffd7fff288066 PyEval_EvalCode () + 36
fffffd7fff2aa8df PyRun_FileExFlags () + 9f
fffffd7fff2ab800 PyRun_SimpleFileExFlags () + 1c0
fffffd7fff2c0710 Py_Main () + a90
0000000000400a0c _start () + 6c
Regards,
Adel
Re: [Proton 0.16.0] [Solaris] Unit tests hanging when destroying
connection_driver
Posted by Adel Boutros <Ad...@live.com>.
PS: The title of the mail is a bit misleading. It can be resumed as follows "Unit tests hanging in proton-c"
________________________________
From: Adel Boutros <Ad...@live.com>
Sent: Wednesday, January 4, 2017 12:27:17 PM
To: users@qpid.apache.org
Subject: [Proton 0.16.0] [Solaris] Unit tests hanging when destroying connection_driver
Hello,
I am compiling Proton 0.16.0 on Solaris using SunStudio compiler. The compiler is successful however unit tests are hanging (cpp_thread_safe_test, cpp_container_test, c-refcount-tests, c-reactor-tests, , cpp-example-container).
All tests except cpp-example-container seem to have the same failure. I am willing to debug this further if needed but maybe you guys already have some hints at why it might be hanging (I tend to think there is an object not added or destroyed correctly and this its reference is not being incremented or decremented).
Debugging analysis so far:
* c-refcount-tests: The hanging is occurring when calling the first drain in test_collector_connection_transport ( drain(collector) )
* cpp_thread_safe_test: The hanging is occurring when calling the last assert in test_new ( ASSERT_EQUAL(1, pn_refcount(c)) )
Failures with stack (pstack $PID | c++filt)
=================================
c-refcounts-tests
------------------------
fffffd7fff2efd87 pn_collector_next () + 2f
00000000004023e2 drain () + 1e
0000000000402464 test_collector_connection_transport () + 74
00000000004027f5 main () + 45
00000000004014bc ???????? ()
c-reactor-tests
------------------------
fffffd7fff2ef8f1 pn_collector_drain () + 3d
fffffd7fff2efbd7 pn_collector_release () + 57
fffffd7fff30025a pn_reactor_run () + 6e
0000000000402b4b test_reactor_event_root () + 63
00000000004038d2 main () + e
00000000004023ac ???????? ()
c-event-tests
------------------------
fffffd7fff2efda1 pn_collector_next () + 49
fffffd7fff2ef8f9 pn_collector_drain () + 45
fffffd7fff2ef9a5 pn_collector_finalize () + 11
fffffd7fff2efa7d pn_collector_finalize_cast () + 9
fffffd7fff2ded19 pn_class_decref () + 6d
fffffd7fff2dedee pn_class_free () + aa
fffffd7fff2df1c7 pn_free () + 13
00000000004011ad test_collector_put () + 65
00000000004014ee main () + e
0000000000400fbc ???????? ()
cpp_thread_safe_test
------------------------
fffffd7fff27fd95 pn_collector_next () + 3d
fffffd7fff27f8f9 pn_collector_drain () + 45
fffffd7fff27fbd7 pn_collector_release () + 57
fffffd7fff27fb68 pn_collector_free () + 38
fffffd7fff27a448 pn_connection_driver_destroy () + 44
fffffd7fff3139d9 proton::io::connection_driver::~connection_driver #Nvariant 1() () + d
0000000000403c79 void __unnamed_KkRMcLMNbYEgR::test_new() () + 4a5
00000000004048f2 main () + 3a
000000000040362c ???????? ()
cpp_container_test
------------------------
fffffd7fff27fd95 pn_collector_next () + 3d
fffffd7fff27f8f9 pn_collector_drain () + 45
fffffd7fff27fbd7 pn_collector_release () + 57
fffffd7fff29025a pn_reactor_run () + 6e
fffffd7fff31a674 void proton::reactor::run() () + c
fffffd7fff30aa97 void proton::container::impl::run() () + 17
fffffd7fff308560 void proton::container::run() () + c
00000000004061b1 int __unnamed_KkRMcLdNbYEzd::test_container_vhost() () + 1ad
0000000000407bb8 main () + 18
0000000000405bec ???????? ()
cpp-example-container
------------------------
fffffd7ffef12dca waitid (0, 2099, fffffd7fffdfe3b0, 3)
fffffd7ffef00f5b waitpid () + 7b
fffffd7fff2c7b76 posix_waitpid () + 46
fffffd7fff285aa3 PyEval_EvalFrameEx () + 5643
fffffd7fff287eba PyEval_EvalCodeEx () + 78a
fffffd7fff2863bc PyEval_EvalFrameEx () + 5f5c
fffffd7fff286e50 PyEval_EvalFrameEx () + 69f0
fffffd7fff287eba PyEval_EvalCodeEx () + 78a
fffffd7fff2863bc PyEval_EvalFrameEx () + 5f5c
fffffd7fff287eba PyEval_EvalCodeEx () + 78a
fffffd7fff288066 PyEval_EvalCode () + 36
fffffd7fff2aa8df PyRun_FileExFlags () + 9f
fffffd7fff2ab800 PyRun_SimpleFileExFlags () + 1c0
fffffd7fff2c0710 Py_Main () + a90
0000000000400a0c _start () + 6c
Regards,
Adel