You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@qpid.apache.org by Aaron Hillegass <aa...@bignerdranch.com> on 2009/11/17 14:40:23 UTC
qpid: Thread-local storage on Mac?
qpid Developers,
I'm working on client/server software. Our users will be running a
client on the Mac and talking to a server on linux. I thought it
would be terrific if I could use your qpid project to do this.
Everything seems to work like a dream on Linux (Nice job!), but I also
need to get the qpid C++ client library compiled on the Mac.
After installing Boost and adding #include <sys/types.h> to include/
qpid/framing/FieldValue.h, I can get pretty far in the build of the
version in svn, but I eventually hit this error:
libtool: compile: g++ -DHAVE_CONFIG_H -I. -I../include -I../include -
I. -I=. -Werror -pedantic -Wall -Wextra -Wno-shadow -Wpointer-arith -
Wcast-qual -Wcast-align -Wno-long-long -Wvolatile-register-var -
Winvalid-pch -Wno-system-headers -Woverloaded-virtual -DQPID_EXEC_DIR=
\"/usr/local/libexec/qpid\" -g -O2 -MT qpid/sys/posix/AsynchIO.lo -MD -
MP -MF qpid/sys/posix/.deps/AsynchIO.Tpo -c qpid/sys/posix/
AsynchIO.cpp -fno-common -DPIC -o qpid/sys/posix/.libs/AsynchIO.o
qpid/sys/posix/AsynchIO.cpp:63: error: thread-local storage not
supported for this target
qpid/sys/posix/AsynchIO.cpp:64: error: thread-local storage not
supported for this target
qpid/sys/posix/AsynchIO.cpp:65: error: thread-local storage not
supported for this target
qpid/sys/posix/AsynchIO.cpp:66: error: thread-local storage not
supported for this target
qpid/sys/posix/AsynchIO.cpp:67: error: thread-local storage not
supported for this target
qpid/sys/posix/AsynchIO.cpp:68: error: thread-local storage not
supported for this target
I found this on the Apple list which might be helpful:
> __thread is not supported. The specification is somewhat ELF-
> specific, which makes it a bit of a challenge.
> Thread-local storage is supported, however. Take a look at the man
> page for pthread_getspecific, for example.
That is from http://lists.apple.com/archives/xcode-users/2006/Jun/msg00550.html
Then I wrote Gordon Sims directly, who recommended that send a note to
this list. He also said:
> It looks like it would be possible to replace the __thread
> declarations with the pthread_getspecific/pthread_setspecific on the
> Mac.
>
> In fact most of those thread local variables are not really used.
> You could try removing them to see whether anything else in the
> build breaks once we get this item solved. Only threadMaxRead is
> actually used in any observable way and it is just to balance reads
> v. writes so removing it for an experiment would be fine.
I'll try this, but if anyone else has thoughts or suggestions, please
let me know.
I'd like to get the Mac supported for the C++ version of qpid. Is
there anything I can I do to help? I can lend a Mac to a developer if
that will make a difference.
I'm not on this list, so please CC me if you respond. Thanks again
for all your hard work!
Sincerely,
Aaron Hillegass
Big Nerd Ranch, Inc
http://www.bignerdranch.com/
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
Re: qpid: Thread-local storage on Mac?
Posted by Andrew Stitcher <as...@redhat.com>.
On Tue, 2009-11-17 at 09:16 -0500, Ted Ross wrote:
> ...
> qpid/sys/posix/AsynchIO.cpp should not use __thread but rather
> QPID_TSS. We could then put a Mac-specific implementation of
> thread-local in QPID_TSS.
Unfortunately this won't help - QPID_TSS is merely a platform
independent piece of syntax for __thread on gcc or __declspec(thread) on
Visual Studio.
Aaron is saying that the Mac gcc simply doesn't have this piece of
syntax working and needs to use other pthread facilities to get the same
effect (but using much less easy to use/comprehensible code - believe me
I've done both)
Andrew
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
Re: qpid: Thread-local storage on Mac?
Posted by Ted Ross <tr...@redhat.com>.
On 11/17/2009 08:40 AM, Aaron Hillegass wrote:
> qpid Developers,
>
> I'm working on client/server software. Our users will be running a
> client on the Mac and talking to a server on linux. I thought it
> would be terrific if I could use your qpid project to do this.
>
> Everything seems to work like a dream on Linux (Nice job!), but I also
> need to get the qpid C++ client library compiled on the Mac.
>
> After installing Boost and adding #include <sys/types.h> to
> include/qpid/framing/FieldValue.h, I can get pretty far in the build
> of the version in svn, but I eventually hit this error:
>
> libtool: compile: g++ -DHAVE_CONFIG_H -I. -I../include -I../include
> -I. -I=. -Werror -pedantic -Wall -Wextra -Wno-shadow -Wpointer-arith
> -Wcast-qual -Wcast-align -Wno-long-long -Wvolatile-register-var
> -Winvalid-pch -Wno-system-headers -Woverloaded-virtual
> -DQPID_EXEC_DIR=\"/usr/local/libexec/qpid\" -g -O2 -MT
> qpid/sys/posix/AsynchIO.lo -MD -MP -MF
> qpid/sys/posix/.deps/AsynchIO.Tpo -c qpid/sys/posix/AsynchIO.cpp
> -fno-common -DPIC -o qpid/sys/posix/.libs/AsynchIO.o
> qpid/sys/posix/AsynchIO.cpp:63: error: thread-local storage not
> supported for this target
> qpid/sys/posix/AsynchIO.cpp:64: error: thread-local storage not
> supported for this target
> qpid/sys/posix/AsynchIO.cpp:65: error: thread-local storage not
> supported for this target
> qpid/sys/posix/AsynchIO.cpp:66: error: thread-local storage not
> supported for this target
> qpid/sys/posix/AsynchIO.cpp:67: error: thread-local storage not
> supported for this target
> qpid/sys/posix/AsynchIO.cpp:68: error: thread-local storage not
> supported for this target
>
> I found this on the Apple list which might be helpful:
>> __thread is not supported. The specification is somewhat ELF-
>> specific, which makes it a bit of a challenge.
>> Thread-local storage is supported, however. Take a look at the man
>> page for pthread_getspecific, for example.
> That is from
> http://lists.apple.com/archives/xcode-users/2006/Jun/msg00550.html
>
> Then I wrote Gordon Sims directly, who recommended that send a note to
> this list. He also said:
>> It looks like it would be possible to replace the __thread
>> declarations with the pthread_getspecific/pthread_setspecific on the
>> Mac.
>>
>> In fact most of those thread local variables are not really used. You
>> could try removing them to see whether anything else in the build
>> breaks once we get this item solved. Only threadMaxRead is actually
>> used in any observable way and it is just to balance reads v. writes
>> so removing it for an experiment would be fine.
>
>
> I'll try this, but if anyone else has thoughts or suggestions, please
> let me know.
>
> I'd like to get the Mac supported for the C++ version of qpid. Is
> there anything I can I do to help? I can lend a Mac to a developer if
> that will make a difference.
>
> I'm not on this list, so please CC me if you respond. Thanks again
> for all your hard work!
>
> Sincerely,
> Aaron Hillegass
> Big Nerd Ranch, Inc
> http://www.bignerdranch.com/
>
>
> ---------------------------------------------------------------------
> Apache Qpid - AMQP Messaging Implementation
> Project: http://qpid.apache.org
> Use/Interact: mailto:dev-subscribe@qpid.apache.org
>
There's already an abstraction set up for thread-local storage to
support Linux and Windows. It's called QPID_TSS and is defined in
cpp/include/qpid/sys/Thread.h
qpid/sys/posix/AsynchIO.cpp should not use __thread but rather
QPID_TSS. We could then put a Mac-specific implementation of
thread-local in QPID_TSS.
-Ted
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
RE: qpid: Thread-local storage on Mac?
Posted by Andrew Stitcher <as...@redhat.com>.
On Tue, 2009-11-17 at 10:29 -0500, Steve Huston wrote:
> Hi Aaron,
>
> > -----Original Message-----
> > From: Aaron Hillegass [mailto:aaron@bignerdranch.com]
> > Sent: Tuesday, November 17, 2009 9:56 AM
> > To: dev@qpid.apache.org
> > Subject: Re: qpid: Thread-local storage on Mac?
> >
> >
> > On 2009/11/17, at 14:18, Andrew Stitcher wrote:
> >
> > > The big job which you clearly haven't got to in your compiles yet
> is
> > > porting the Poller implementation. Currently there are 2
> > ports for the
> > > Poller epoll for Linux and ecf for Solaris - the Solaris port is
> > > currently broken.
> > >
> > > I'm not sure which poll primitives are available on Darwin
> > > - if you're lucky you might have a version of kqueue to use.
> >
> > Yes, we have kqueue:
> >
> > http://developer.apple.com/Mac/library/documentation/Darwin/Re
> ference/ManPages/man2/kqueue.2.html
>
> That's good... You have the facilities available to work this, but you
> have a few weeks' worth of work ahead. I've seen your other email as
> well and it seems like you need to work on:
>
> 1. Poller, using kqueue. You can probably leverage a lot of the epoll
> code; at least the way it enables and disables handles and senses
> events.
> 2. TSS - you may be able to get away with not using it, if all you
> need is the client. You'll probably be writing a Mac version of
> AsynchIO.cpp - just don't carry over the TSS variables from the posix
> version. If you do encounter a need to transition to run-time TSS
> you'll need to add a bit if restructuring there as well.
> 3. #includes - rather than #include <sys/types.h> everywhere, make the
> needed adjustments to qpid/sys/IntegerTypes.h
Given that the OSX userspace is mostly BSD I'd expect that point 3.
should work pretty much the same as on Solaris and Linux (ie not very
much to change here, but any changes are in the way of a few missing
#include lines because of implied header -> header in Linux not present
elsewhere).
Point 2. is more problematic as it's counter productive to split the
various POSIX based ports into different files unless really necessary.
In this case any solution using an encapsulated
pthread_getspecific/pthread_setspecific will work on all Posix
platforms.
There are places where thread specific variables are essential - in the
implementation of the Linux EpollPoller for example - and any kqueues
based port will inherit this need as well.
However as Gordon notes the AsyncIO uses of thread based variables are
more suspect in their necessity and should be revisited anyway.
Andrew
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
RE: qpid: Thread-local storage on Mac?
Posted by Steve Huston <sh...@riverace.com>.
Hi Aaron,
> -----Original Message-----
> From: Aaron Hillegass [mailto:aaron@bignerdranch.com]
> Sent: Tuesday, November 17, 2009 9:56 AM
> To: dev@qpid.apache.org
> Subject: Re: qpid: Thread-local storage on Mac?
>
>
> On 2009/11/17, at 14:18, Andrew Stitcher wrote:
>
> > The big job which you clearly haven't got to in your compiles yet
is
> > porting the Poller implementation. Currently there are 2
> ports for the
> > Poller epoll for Linux and ecf for Solaris - the Solaris port is
> > currently broken.
> >
> > I'm not sure which poll primitives are available on Darwin
> > - if you're lucky you might have a version of kqueue to use.
>
> Yes, we have kqueue:
>
> http://developer.apple.com/Mac/library/documentation/Darwin/Re
ference/ManPages/man2/kqueue.2.html
That's good... You have the facilities available to work this, but you
have a few weeks' worth of work ahead. I've seen your other email as
well and it seems like you need to work on:
1. Poller, using kqueue. You can probably leverage a lot of the epoll
code; at least the way it enables and disables handles and senses
events.
2. TSS - you may be able to get away with not using it, if all you
need is the client. You'll probably be writing a Mac version of
AsynchIO.cpp - just don't carry over the TSS variables from the posix
version. If you do encounter a need to transition to run-time TSS
you'll need to add a bit if restructuring there as well.
3. #includes - rather than #include <sys/types.h> everywhere, make the
needed adjustments to qpid/sys/IntegerTypes.h
-Steve
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
Re: qpid: Thread-local storage on Mac?
Posted by Aaron Hillegass <aa...@bignerdranch.com>.
On 2009/11/17, at 14:18, Andrew Stitcher wrote:
> The big job which you clearly haven't got to in your compiles yet is
> porting the Poller implementation. Currently there are 2 ports for the
> Poller epoll for Linux and ecf for Solaris - the Solaris port is
> currently broken.
>
> I'm not sure which poll primitives are available on Darwin - if you're
> lucky you might have a version of kqueue to use.
Yes, we have kqueue:
http://developer.apple.com/Mac/library/documentation/Darwin/Reference/ManPages/man2/kqueue.2.html
- Aaron
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
Re: qpid: Thread-local storage on Mac?
Posted by Bruno Matos <br...@paradigmaxis.pt>.
On 2009/11/17, at 14:18, Andrew Stitcher wrote:
> On Tue, 2009-11-17 at 08:40 -0500, Aaron Hillegass wrote:
>> qpid Developers,
>>
>> I'm working on client/server software. Our users will be running a
>> client on the Mac and talking to a server on linux. I thought it
>> would be terrific if I could use your qpid project to do this.
>>
>> Everything seems to work like a dream on Linux (Nice job!), but I
>> also
>> need to get the qpid C++ client library compiled on the Mac.
>
> The big job which you clearly haven't got to in your compiles yet is
> porting the Poller implementation. Currently there are 2 ports for the
> Poller epoll for Linux and ecf for Solaris - the Solaris port is
> currently broken.
>
> I'm not sure which poll primitives are available on Darwin - if you're
> lucky you might have a version of kqueue to use.
>
> Incidentally there is essential thread specific variable use here too.
>
> "git grep __thread" on my up to date tree returns:
> qpid/cpp/include/qpid/sys/Thread.h:# define QPID_TSS __thread
> qpid/cpp/include/qpid/sys/Thread.h:# define QPID_TSS __thread
> qpid/cpp/src/qpid/sys/DeletionManager.h: static __thread
> ThreadStatus* threadStatus = 0;
> qpid/cpp/src/qpid/sys/epoll/EpollPoller.cpp: static __thread
> PollerHandlePrivate* lastReturnedHandle = 0;
> qpid/cpp/src/qpid/sys/posix/AsynchIO.cpp:__thread int
> threadReadTotal =
> 0;
> qpid/cpp/src/qpid/sys/posix/AsynchIO.cpp:__thread int threadMaxRead
> = 0;
> qpid/cpp/src/qpid/sys/posix/AsynchIO.cpp:__thread int
> threadReadCount =
> 0;
> qpid/cpp/src/qpid/sys/posix/AsynchIO.cpp:__thread int
> threadWriteTotal =
> 0;
> qpid/cpp/src/qpid/sys/posix/AsynchIO.cpp:__thread int
> threadWriteCount =
> 0;
> qpid/cpp/src/qpid/sys/posix/AsynchIO.cpp:__thread int64_t
> threadMaxReadTimeNs = 2 * 1000000; // start at 2ms
> qpid/cpp/src/qpid/sys/rdma/rdma_exception.h: static __thread char
> s[50];
> qpid/cpp/src/qpid/sys/ssl/SslIo.cpp:__thread int threadReadTotal = 0;
> qpid/cpp/src/qpid/sys/ssl/SslIo.cpp:__thread int threadMaxRead = 0;
> qpid/cpp/src/qpid/sys/ssl/SslIo.cpp:__thread int threadReadCount = 0;
> qpid/cpp/src/qpid/sys/ssl/SslIo.cpp:__thread int threadWriteTotal = 0;
> qpid/cpp/src/qpid/sys/ssl/SslIo.cpp:__thread int threadWriteCount = 0;
> qpid/cpp/src/qpid/sys/ssl/SslIo.cpp:__thread int64_t
> threadMaxReadTimeNs
> = 2 * 1000000; // start at 2ms
>
> Andrew
>
>
>
> ---------------------------------------------------------------------
> Apache Qpid - AMQP Messaging Implementation
> Project: http://qpid.apache.org
> Use/Interact: mailto:dev-subscribe@qpid.apache.org
>
>
I would like to help. I reached to the same point.
Regards.
--
Bruno Matos
bruno.matos@paradigmaxis.pt
Re: qpid: Thread-local storage on Mac?
Posted by Andrew Stitcher <as...@redhat.com>.
On Tue, 2009-11-17 at 08:40 -0500, Aaron Hillegass wrote:
> qpid Developers,
>
> I'm working on client/server software. Our users will be running a
> client on the Mac and talking to a server on linux. I thought it
> would be terrific if I could use your qpid project to do this.
>
> Everything seems to work like a dream on Linux (Nice job!), but I also
> need to get the qpid C++ client library compiled on the Mac.
The big job which you clearly haven't got to in your compiles yet is
porting the Poller implementation. Currently there are 2 ports for the
Poller epoll for Linux and ecf for Solaris - the Solaris port is
currently broken.
I'm not sure which poll primitives are available on Darwin - if you're
lucky you might have a version of kqueue to use.
Incidentally there is essential thread specific variable use here too.
"git grep __thread" on my up to date tree returns:
qpid/cpp/include/qpid/sys/Thread.h:# define QPID_TSS __thread
qpid/cpp/include/qpid/sys/Thread.h:# define QPID_TSS __thread
qpid/cpp/src/qpid/sys/DeletionManager.h: static __thread
ThreadStatus* threadStatus = 0;
qpid/cpp/src/qpid/sys/epoll/EpollPoller.cpp: static __thread
PollerHandlePrivate* lastReturnedHandle = 0;
qpid/cpp/src/qpid/sys/posix/AsynchIO.cpp:__thread int threadReadTotal =
0;
qpid/cpp/src/qpid/sys/posix/AsynchIO.cpp:__thread int threadMaxRead = 0;
qpid/cpp/src/qpid/sys/posix/AsynchIO.cpp:__thread int threadReadCount =
0;
qpid/cpp/src/qpid/sys/posix/AsynchIO.cpp:__thread int threadWriteTotal =
0;
qpid/cpp/src/qpid/sys/posix/AsynchIO.cpp:__thread int threadWriteCount =
0;
qpid/cpp/src/qpid/sys/posix/AsynchIO.cpp:__thread int64_t
threadMaxReadTimeNs = 2 * 1000000; // start at 2ms
qpid/cpp/src/qpid/sys/rdma/rdma_exception.h: static __thread char
s[50];
qpid/cpp/src/qpid/sys/ssl/SslIo.cpp:__thread int threadReadTotal = 0;
qpid/cpp/src/qpid/sys/ssl/SslIo.cpp:__thread int threadMaxRead = 0;
qpid/cpp/src/qpid/sys/ssl/SslIo.cpp:__thread int threadReadCount = 0;
qpid/cpp/src/qpid/sys/ssl/SslIo.cpp:__thread int threadWriteTotal = 0;
qpid/cpp/src/qpid/sys/ssl/SslIo.cpp:__thread int threadWriteCount = 0;
qpid/cpp/src/qpid/sys/ssl/SslIo.cpp:__thread int64_t threadMaxReadTimeNs
= 2 * 1000000; // start at 2ms
Andrew
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org