c++ client - Windows port

[Matt Griswold <ma...@devIT.com>]

Hello Steve et al.,

I've been trying to get the c++ client working in windows, fixed a few issues
in the IO layer, and am now getting deadlocks in the state manager.  Since
Steve mentioned something similar in the broker I figured I should check in
and see if anyone has been working on it before looking into it more.

I'd be happy to pull together the patches against the git windows2 for what
I've done so far if no one else has them fixed anywhere.

Thanks,


Matt

Re: c++ client - Windows port

[Alan Conway <ac...@redhat.com>]

On Thu, 2008-09-04 at 08:10 -0500, Matt Griswold wrote:
> 
> 
> * Alan Conway <ac...@redhat.com> [09/04/08 07:51]:
> > So my suggestion: if boost::time solves your problem on windows then
> do
> > a new sys/boost/Time.cpp impl using boost::time, so no impact on
> other
> > code.
> 
> That's been working well for me.  I've been playing around with making
> a boost
> "port" in sys/boost since I'll eventually need an OSX client too and I
> figured

Absolutely! And for any case where the boost port works everywhere we
can promote it to the default implementation. I'm a big fan of
offloading portability issues wherever possible. A boost version of the
threads & locking would be very interesting also - I shied away from it
when I did the initial impl because I had some (untested) performance
concerns but if it can be shown to perform equally well  I'll happily
ditch the posix impl.

Re: c++ client - Windows port

[Andrew Stitcher <as...@redhat.com>]

On Thu, 2008-09-04 at 14:27 -0500, Matt Griswold wrote:
> * Andrew Stitcher <as...@redhat.com> [09/04/08 12:49]:
> > Also pthreads is well documented (in books even), and boost is still
> > subject to change. If these things make it into the C++ standard lib as
> > has been threatened then I'll argue strongly to use that, but I don't
> > see any value in changing from pthread to boost for those platforms that
> > are sufficiently posix.
> 
> I was using sys/boost as a generic base for making the port(s), not
> necessarily as a replacement to native or posix implementations.  I included
> the boost mutex stuff from Steve's windows port, and did time, socket, and
> asyncIO/Poller also from boost lib.  It's a quick and easy way for me to get a
> cross compilable, portable qpidclient, and I figured if/when boost components
> make it to libstdc++ it'll be a bonus.
> 
> That aside, I tend agree with Alan, if it's as fast as the native
> implementations, it seems like it would be easier to just have one version,
> but that's up to you guys :)  In my application performance isn't key for the
> client side, I just need to have it working in Windows and OSX.

That's assuming we are starting from scratch!

Our previous experience with some younger boost libraries is that they
have not been sufficiently stable or functional and we've spent far too
much time compensating for their deficiencies/changes (particularly
program_options). Parts of boost are excellent, particularly the stuff
that has now been accepted into C++0x, but parts are too immature to use
in a project aiming for some stability. We need to distinguish these
bits of boost.

I'm not saying anything about the boost threading library, but we
already have a pthreads solution, lets just use it on platforms that
support it.

Andrew

Re: c++ client - Windows port

[Matt Griswold <ma...@devIT.com>]

* Andrew Stitcher <as...@redhat.com> [09/04/08 12:49]:
> Also pthreads is well documented (in books even), and boost is still
> subject to change. If these things make it into the C++ standard lib as
> has been threatened then I'll argue strongly to use that, but I don't
> see any value in changing from pthread to boost for those platforms that
> are sufficiently posix.

I was using sys/boost as a generic base for making the port(s), not
necessarily as a replacement to native or posix implementations.  I included
the boost mutex stuff from Steve's windows port, and did time, socket, and
asyncIO/Poller also from boost lib.  It's a quick and easy way for me to get a
cross compilable, portable qpidclient, and I figured if/when boost components
make it to libstdc++ it'll be a bonus.

That aside, I tend agree with Alan, if it's as fast as the native
implementations, it seems like it would be easier to just have one version,
but that's up to you guys :)  In my application performance isn't key for the
client side, I just need to have it working in Windows and OSX.

Steve: I made an account on github, and when I get this all working I'll talk
to you about making a branch on there to throw changes up (hope to be done by
the end of the weekend if time allows).  As I said, I used a few of the files
from your windows port, so having both versions up there should be easier to
keep any changes synced.

RE: c++ client - Windows port

[Steve Huston <sh...@riverace.com>]

> On Thu, 2008-09-04 at 11:18 -0400, Steve Huston wrote:
> > > * Alan Conway <ac...@redhat.com> [09/04/08 07:51]:
> > > > So my suggestion: if boost::time solves your problem on 
> > > > windows then do
> > > > a new sys/boost/Time.cpp impl using boost::time, so no 
> > > > impact on other code.
> > 
> > That's essentially the path I've taken... But the first time a
timed
> > condition wait was tried, boost threw an out-of-range time 
> exception.
> > 
> > The Windows Mutex, Condition, etc. are boost-based. So I guess it
> > could be moved to sys/boost.
> 
> At this point I think the pthreads code is well tested and is very
> portable to many platforms - I would be against replacing it with a
> boost implementation. Especially as on these same platforms the
boost
> implementation will just sit on top of the pthreads implementation
> anyway (in most cases).

Right - to be clear, I wasn't looking to replace pthreads with boost.

> Also pthreads is well documented (in books even), and boost is still
> subject to change. If these things make it into the C++ 
> standard lib as
> has been threatened then I'll argue strongly to use that, but I
don't
> see any value in changing from pthread to boost for those 
> platforms that are sufficiently posix.

Right - as long as the platform implements the same Pthreads
standard/draft as RHEL 4/5, great.

> > 
> > > That's been working well for me.  I've been playing around 
> > > with making a boost
> > > "port" in sys/boost since I'll eventually need an OSX client 
> > > too and I figured
> > > it was easiest to let boost handle as much of the native code 
> > > as possible.
> 
> If I'm not mistaken OSX, being unix essentially, will already have
> pthreads.

But which one? I don't know the answer... Just noting that not all
Pthreads are created equal.

-Steve

RE: c++ client - Windows port

[Andrew Stitcher <as...@redhat.com>]

On Thu, 2008-09-04 at 11:18 -0400, Steve Huston wrote:
> > * Alan Conway <ac...@redhat.com> [09/04/08 07:51]:
> > > So my suggestion: if boost::time solves your problem on 
> > > windows then do
> > > a new sys/boost/Time.cpp impl using boost::time, so no 
> > > impact on other code.
> 
> That's essentially the path I've taken... But the first time a timed
> condition wait was tried, boost threw an out-of-range time exception.
> 
> The Windows Mutex, Condition, etc. are boost-based. So I guess it
> could be moved to sys/boost.

At this point I think the pthreads code is well tested and is very
portable to many platforms - I would be against replacing it with a
boost implementation. Especially as on these same platforms the boost
implementation will just sit on top of the pthreads implementation
anyway (in most cases).

Also pthreads is well documented (in books even), and boost is still
subject to change. If these things make it into the C++ standard lib as
has been threatened then I'll argue strongly to use that, but I don't
see any value in changing from pthread to boost for those platforms that
are sufficiently posix.

> 
> > That's been working well for me.  I've been playing around 
> > with making a boost
> > "port" in sys/boost since I'll eventually need an OSX client 
> > too and I figured
> > it was easiest to let boost handle as much of the native code 
> > as possible.

If I'm not mistaken OSX, being unix essentially, will already have
pthreads.


Andrew

RE: c++ client - Windows port

[Steve Huston <sh...@riverace.com>]

> * Alan Conway <ac...@redhat.com> [09/04/08 07:51]:
> > So my suggestion: if boost::time solves your problem on 
> > windows then do
> > a new sys/boost/Time.cpp impl using boost::time, so no 
> > impact on other code.

That's essentially the path I've taken... But the first time a timed
condition wait was tried, boost threw an out-of-range time exception.

The Windows Mutex, Condition, etc. are boost-based. So I guess it
could be moved to sys/boost.

> That's been working well for me.  I've been playing around 
> with making a boost
> "port" in sys/boost since I'll eventually need an OSX client 
> too and I figured
> it was easiest to let boost handle as much of the native code 
> as possible.

-Steve

Re: c++ client - Windows port

[Matt Griswold <ma...@devIT.com>]

* Steve Huston <sh...@riverace.com> [09/03/08 17:53]:
> Thanks, Matt! I've integrated these and a few local fixes to Monitor
> and Condition. The current source is pushed up to qpid-port, windows2
> branch.

Great, I'll check it out.

* Alan Conway <ac...@redhat.com> [09/04/08 07:51]:
> So my suggestion: if boost::time solves your problem on windows then do
> a new sys/boost/Time.cpp impl using boost::time, so no impact on other
> code.

That's been working well for me.  I've been playing around with making a boost
"port" in sys/boost since I'll eventually need an OSX client too and I figured
it was easiest to let boost handle as much of the native code as possible.


-- 
Matt Griswold
dev/IT
matt@devIT.com
608.443.4002

RE: c++ client - Windows port

[Alan Conway <ac...@redhat.com>]

On Wed, 2008-09-03 at 18:53 -0400, Steve Huston wrote:
> Thanks, Matt! I've integrated these and a few local fixes to Monitor
> and Condition. The current source is pushed up to qpid-port, windows2
> branch.
> 
> There's an issue with the way AbsTime is handled and passed to
> condition variables... That's next up on my list - if anyone has
> particular expertise with boost::posix_time, I'd welcome the input.

Things to watch out for: 
 - older boost - had trouble with this and undid some previous minor use
of boost::date_time a while back.
 - performance & resolution: probably fine but the templates were too
labyrinthine for for me to be sure.
 - impact on existing code using AbsTime/Duration

So my suggestion: if boost::time solves your problem on windows then do
a new sys/boost/Time.cpp impl using boost::time, so no impact on other
code.

If the boost impl works ok on linuxen, solaris etc. then we can use it
everywhere. On any platform where we do have a problem (performance
effects, portability, dealing with older boost) we can conditionally use
the old impl. Ideally there are no issues and we ditch the old impl but
I wouldn't assume that in advance.

Cheers,
Alan.

RE: c++ client - Windows port

[Steve Huston <sh...@riverace.com>]

Thanks, Matt! I've integrated these and a few local fixes to Monitor
and Condition. The current source is pushed up to qpid-port, windows2
branch.

There's an issue with the way AbsTime is handled and passed to
condition variables... That's next up on my list - if anyone has
particular expertise with boost::posix_time, I'd welcome the input.

Thanks,
-Steve

> -----Original Message-----
> From: Matt Griswold [mailto:matt@devIT.com] 
> Sent: Thursday, August 28, 2008 5:25 PM
> To: qpid-dev@incubator.apache.org
> Subject: Re: c++ client - Windows port
> 
> 
> * Steve Huston <sh...@riverace.com> [08/28/08 06:38]:
> > That would be helpful - I am hoping to have the windows client
> > completed in the next week or so, so let's work together - you can
> > either send me patches against windows2 or make another 
> branch in the
> > git repo.
> 
> Sounds good.  Here's the stuff I did so far from git diff 
> against windows2,
> let me know if you'd like to me to work on anything else with 
> the client.
> 
> I'm not sure if any of these changes are how you had intended 
> on it working,
> these are just quick fixes I did while trying to get it 
> working (when I
> started on this I for some reason thought that the client was 
> done and working
> or I would have contacted you beforehand).
> 
> Changes:
> 
> * Removed const and subsequent const cast from overlapped()
> * Moved the OVERLAPPED cast back to AsynchResult so it uses 
> the correct offset
> * Changed WSABUF to protected, and moved initialization to 
> read and write
> results.  AsynchWriteResult bufs were getting set to the end 
> of the data and
> sending uninitialized memory.
> 
> 
> Matt
> 

Re: c++ client - Windows port

[Matt Griswold <ma...@devIT.com>]

* Steve Huston <sh...@riverace.com> [08/28/08 06:38]:
> That would be helpful - I am hoping to have the windows client
> completed in the next week or so, so let's work together - you can
> either send me patches against windows2 or make another branch in the
> git repo.

Sounds good.  Here's the stuff I did so far from git diff against windows2,
let me know if you'd like to me to work on anything else with the client.

I'm not sure if any of these changes are how you had intended on it working,
these are just quick fixes I did while trying to get it working (when I
started on this I for some reason thought that the client was done and working
or I would have contacted you beforehand).

Changes:

* Removed const and subsequent const cast from overlapped()
* Moved the OVERLAPPED cast back to AsynchResult so it uses the correct offset
* Changed WSABUF to protected, and moved initialization to read and write
results.  AsynchWriteResult bufs were getting set to the end of the data and
sending uninitialized memory.


Matt

RE: c++ client - Windows port

[Steve Huston <sh...@riverace.com>]

Hi Matt,

> I've been trying to get the c++ client working in windows, 
> fixed a few issues in the IO layer, and am now getting deadlocks
> in the state manager.

Ok.

> Since Steve mentioned something similar in the broker I figured I 
> should check in
> and see if anyone has been working on it before looking into it
more.

Yes, I've been working on it more. 

> I'd be happy to pull together the patches against the git 
> windows2 for what
> I've done so far if no one else has them fixed anywhere.

That would be helpful - I am hoping to have the windows client
completed in the next week or so, so let's work together - you can
either send me patches against windows2 or make another branch in the
git repo.

Thanks!
-Steve