You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@qpid.apache.org by "Darryl L. Pierce" <dp...@redhat.com> on 2012/05/07 23:16:35 UTC
Strange situation with qpid::sys::Thread and fds
In reworking the Ruby 1.8 non-block I/O model, I'm banging my head
against a nasty blocker.
In the new model I have two main types: a Runner and a SynchioCommand.
All of the existing code that was written has been refactored so that
each of the commands (Receiver::Get/Fetch, Sender::Send, etc) are
inherit of SynchioCommand.
The Runner class implements qpid::sys::Runnable and has a queue that
contains instances of SynchioCommand.
When a command is created and the start() method invoked, it adds itself
to the Runner's queue. Runner periodically checks the queue and when it
sees a new command, it pops it off the front of the queue, tells it to
create the file descriptor that it uses to communicate with the ruby
extension.
All of this works fine. I've debugged and verified that the fd is
created and properly returned to the Ruby extension, and that the
SynchioCommand runs. The problem is that, after the command finishes
running and it pushes a byte into the fd (changing it's status to
readable), the Ruby extension never sees this. So the blocking I/O
command finishes, but then Ruby never wakes up.
Any ideas on what would be causing the fd's readable state to not
change? I've tried several different ways to find the problem and
nothing has uncovered the problem...
--
Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc.
Delivering value year after year.
Red Hat ranks #1 in value among software vendors.
http://www.redhat.com/promo/vendor/
Re: Strange situation with qpid::sys::Thread and fds
Posted by "Darryl L. Pierce" <dp...@redhat.com>.
On Wed, May 09, 2012 at 08:33:58AM -0400, Alan Conway wrote:
> On 05/08/2012 10:03 AM, Darryl L. Pierce wrote:
> >On Tue, May 08, 2012 at 08:52:32AM -0500, Steve Huston wrote:
> >>What kind of fd is this? Linux?
> >
> >Yes. There's a class named Prong that creates a one-way pipe for sending
> >a notification to the Ruby thread to wake it up. The pipe is created
> >using the pipe() API, then fcntl() to make the descriptor non-blocking.
> >
> >>The symptom sounds like what would happen if a TCP socket write were done
> >>and TCP buffered the byte. Setting no-delay on the socket would resolve
> >>that.
> >
> >I tried adding O_NDELAY to the flags for the fds and it made no
> >difference:
> >
> >flags = ::fcntl(yourHandle, F_GETFL);
> >if((::fcntl(yourHandle, F_SETFL, flags | O_NONBLOCK | O_NDELAY))== -1)
> > throw MessagingException("Unable to make your handle non-blocking.");
> >
>
> Sounds like you are doing something very similar to
> qpid::sys::PollableQueue or qpid::sys::Condition. You might be able
> to cut/paste some of the code.
>
> PollableCondition exposes an FD to be added to qpid::sys::Poller.
> The FD is readable iff the condition is set.
> PollableQueue is a work queue that uses a PollableCondition so that
> it is pollable iff there is data in the queue.
Looking through the code it looks like the ideal for what I'm going for.
I would like the CommandRunner (renamed the class) to not sleep but to
respond immediately to a new command being enqueued. I'll pull what I
can out of this code and see if that helps.
--
Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc.
Delivering value year after year.
Red Hat ranks #1 in value among software vendors.
http://www.redhat.com/promo/vendor/
Re: Strange situation with qpid::sys::Thread and fds
Posted by Alan Conway <ac...@redhat.com>.
On 05/08/2012 10:03 AM, Darryl L. Pierce wrote:
> On Tue, May 08, 2012 at 08:52:32AM -0500, Steve Huston wrote:
>> What kind of fd is this? Linux?
>
> Yes. There's a class named Prong that creates a one-way pipe for sending
> a notification to the Ruby thread to wake it up. The pipe is created
> using the pipe() API, then fcntl() to make the descriptor non-blocking.
>
>> The symptom sounds like what would happen if a TCP socket write were done
>> and TCP buffered the byte. Setting no-delay on the socket would resolve
>> that.
>
> I tried adding O_NDELAY to the flags for the fds and it made no
> difference:
>
> flags = ::fcntl(yourHandle, F_GETFL);
> if((::fcntl(yourHandle, F_SETFL, flags | O_NONBLOCK | O_NDELAY))== -1)
> throw MessagingException("Unable to make your handle non-blocking.");
>
Sounds like you are doing something very similar to qpid::sys::PollableQueue or
qpid::sys::Condition. You might be able to cut/paste some of the code.
PollableCondition exposes an FD to be added to qpid::sys::Poller. The FD is
readable iff the condition is set.
PollableQueue is a work queue that uses a PollableCondition so that it is
pollable iff there is data in the queue.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org
Re: Strange situation with qpid::sys::Thread and fds
Posted by "Darryl L. Pierce" <dp...@redhat.com>.
On Tue, May 08, 2012 at 12:50:40PM -0500, Steve Huston wrote:
> No, NDELAY wouldn't help a pipe.
>
> Sorry to ask, but are the read/write sides reversed on one end?
No, it's okay to ask. :) And, no, the sides aren't reversed. The code
for communicating between threads was working fine prior to this new
model.
--
Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc.
Delivering value year after year.
Red Hat ranks #1 in value among software vendors.
http://www.redhat.com/promo/vendor/
RE: Strange situation with qpid::sys::Thread and fds
Posted by Steve Huston <sh...@riverace.com>.
No, NDELAY wouldn't help a pipe.
Sorry to ask, but are the read/write sides reversed on one end?
> -----Original Message-----
> From: Darryl L. Pierce [mailto:dpierce@redhat.com]
> Sent: Tuesday, May 08, 2012 10:03 AM
> To: dev@qpid.apache.org
> Subject: Re: Strange situation with qpid::sys::Thread and fds
>
> On Tue, May 08, 2012 at 08:52:32AM -0500, Steve Huston wrote:
> > What kind of fd is this? Linux?
>
> Yes. There's a class named Prong that creates a one-way pipe for sending
a
> notification to the Ruby thread to wake it up. The pipe is created using
the
> pipe() API, then fcntl() to make the descriptor non-blocking.
>
> > The symptom sounds like what would happen if a TCP socket write were
> > done and TCP buffered the byte. Setting no-delay on the socket would
> > resolve that.
>
> I tried adding O_NDELAY to the flags for the fds and it made no
> difference:
>
> flags = ::fcntl(yourHandle, F_GETFL);
> if((::fcntl(yourHandle, F_SETFL, flags | O_NONBLOCK | O_NDELAY))== -1)
> throw MessagingException("Unable to make your handle non-blocking.");
>
> --
> Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc.
> Delivering value year after year.
> Red Hat ranks #1 in value among software vendors.
> http://www.redhat.com/promo/vendor/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org
Re: Strange situation with qpid::sys::Thread and fds
Posted by "Darryl L. Pierce" <dp...@redhat.com>.
On Tue, May 08, 2012 at 08:52:32AM -0500, Steve Huston wrote:
> What kind of fd is this? Linux?
Yes. There's a class named Prong that creates a one-way pipe for sending
a notification to the Ruby thread to wake it up. The pipe is created
using the pipe() API, then fcntl() to make the descriptor non-blocking.
> The symptom sounds like what would happen if a TCP socket write were done
> and TCP buffered the byte. Setting no-delay on the socket would resolve
> that.
I tried adding O_NDELAY to the flags for the fds and it made no
difference:
flags = ::fcntl(yourHandle, F_GETFL);
if((::fcntl(yourHandle, F_SETFL, flags | O_NONBLOCK | O_NDELAY))== -1)
throw MessagingException("Unable to make your handle non-blocking.");
--
Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc.
Delivering value year after year.
Red Hat ranks #1 in value among software vendors.
http://www.redhat.com/promo/vendor/
RE: Strange situation with qpid::sys::Thread and fds
Posted by Steve Huston <sh...@riverace.com>.
What kind of fd is this? Linux?
The symptom sounds like what would happen if a TCP socket write were done
and TCP buffered the byte. Setting no-delay on the socket would resolve
that.
-Steve
> -----Original Message-----
> From: Darryl L. Pierce [mailto:dpierce@redhat.com]
> Sent: Monday, May 07, 2012 5:17 PM
> To: dev@qpid.apache.org
> Subject: Strange situation with qpid::sys::Thread and fds
>
> In reworking the Ruby 1.8 non-block I/O model, I'm banging my head
against
> a nasty blocker.
>
> In the new model I have two main types: a Runner and a SynchioCommand.
> All of the existing code that was written has been refactored so that
each of
> the commands (Receiver::Get/Fetch, Sender::Send, etc) are inherit of
> SynchioCommand.
>
> The Runner class implements qpid::sys::Runnable and has a queue that
> contains instances of SynchioCommand.
>
> When a command is created and the start() method invoked, it adds itself
to
> the Runner's queue. Runner periodically checks the queue and when it
sees
> a new command, it pops it off the front of the queue, tells it to create
the file
> descriptor that it uses to communicate with the ruby extension.
>
> All of this works fine. I've debugged and verified that the fd is
created and
> properly returned to the Ruby extension, and that the SynchioCommand
> runs. The problem is that, after the command finishes running and it
pushes
> a byte into the fd (changing it's status to readable), the Ruby
extension never
> sees this. So the blocking I/O command finishes, but then Ruby never
wakes
> up.
>
> Any ideas on what would be causing the fd's readable state to not
change?
> I've tried several different ways to find the problem and nothing has
> uncovered the problem...
>
> --
> Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc.
> Delivering value year after year.
> Red Hat ranks #1 in value among software vendors.
> http://www.redhat.com/promo/vendor/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org