You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nuttx.apache.org by Flavio Castro Alves Filho <fl...@gmail.com> on 2021/06/26 13:11:27 UTC
Socket programming: send() function blocked
Hello,
I am using the MQTT-C library (https://github.com/LiamBindle/MQTT-C)
in my NuttX project and I am facing a quite interesting issue with a
specific server.
For a specific broker, which is a commercial one, after the connection
is established, when the MQTT Connect message is sent, the firmware
gets stuck.
Debugging the code, I can see that the send() function is blocked.
When I set the socket, I setup to work in non-blocking mode:
===
/*
* Opening socket
*/
int sockfd = socket(AF_INET, SOCK_STREAM | SOCK_NONBLOCK, IPPROTO_IP);
if (sockfd == -1) {
syslog(LOG_ERR, "MqttClient: error socket() %d
(%s)\n",errno,strerror(errno));
return -1;
}
struct timeval tv;
tv.tv_sec = 3;
tv.tv_usec = 0;
setsockopt(sockfd, SOL_SOCKET, SO_RCVTIMEO,
(FAR const void *)&tv, sizeof(struct timeval));
setsockopt(sockfd, SOL_SOCKET, SO_SNDTIMEO,
(FAR const void *)&tv, sizeof(struct timeval));
/*
* Connecting
*/
int rv = connect(sockfd, server_address, server_address_len);
syslog(LOG_DEBUG,"connect() = %d\n",rv);
if(rv == -1) {
syslog(LOG_ERR, "MqttClient: error connect() %d
(%s)\n",errno,strerror(errno));
close(sockfd);
sockfd = -1;
return -1;
}
/*
* make non-blocking
*/
if (sockfd != -1) {
fcntl(sockfd, F_SETFL, fcntl(sockfd, F_GETFL) | O_NONBLOCK);
}
===
I couldn't find the reason why the send() function gets stuck.
And, worst, it happens when I try to communicate with a specific broker.
Can anyone give me advice about why it is happening and what I am doing wrong?
Best regards,
Flavio
--
Flavio de Castro Alves Filho
flavio.alves@gmail.com
Twitter: http://twitter.com/#!/fraviofii
LinkedIn profile: www.linkedin.com/in/flaviocastroalves
Re: Socket programming: send() function blocked
Posted by Flavio Castro Alves Filho <fl...@gmail.com>.
Hello Johnny,
Thank you for the explanation. That's an important remark so I can
continue the evaluation here.
I was thinking that the issue was related to the MQTT library. But, in
fact, there is something wrong with the TCP connection.
I will post here as soon as I have more news.
Best regards,
Flavio
Em dom., 27 de jun. de 2021 às 13:56, Johnny Billquist
<bq...@softjar.se> escreveu:
>
> Something is seriously wrong in your first run.
> Maybe a tcpdump would expose something, but basically, tcp_max_wrb_size
> should never be 0, but in your first run it appears to be.
>
> As that is the buffer size to use, you end up with a tcp connection with
> a buffer size of 0, so of course all writes (including send) will return
> with success, but with 0 bytes written, since there wasn't space for
> anything more.
>
> tcp_max_wrb_size should be based on mss, which suggests that mss would
> be zero. But that is rather surprising. Since the mss is exchanged
> between the hosts at tcp connection establishment, it would be very
> interesting to report what mss each side is suggesting.
>
> It might be that one side isn't reporting any mss at all, and then the
> tcp implementation in Nuttx behaves badly. I don't really have time to
> dive into this more right now, but this should give you some pointers
> and ideas to where to look further.
>
> Johnny
>
> On 2021-06-27 16:34, Flavio Castro Alves Filho wrote:
> > I'm sorry ... I got the same behavior, but the send function now
> > started to work non-blocking.
> >
> > Here is my situation:
> >
> > * I have 2 mqtt brokers (tcp servers)
> > - one commercial (Tago.io)
> > - one personal (Mosquitto)
> >
> > After a successful connection in both cases, the next step is to send
> > the mqtt connect message.
> >
> > Below shows the execution trace for the commercial broker.
> >
> > [972934023.720000] [ INFO] psock_vfcntl: sockfd=0x10007a70 cmd=9
> > [972934023.730000] [ INFO] psock_tcp_send (0x1000a9ac):
> > [972934023.730000] [ INFO] 0000 10 42 00 04 4d 51 54 54 04 c2 01 90
> > 00 07 54 65 .B..MQTT......Te
> > [972934023.740000] [ INFO] 0010 73 74 65 47 57 00 07 70 68 69 74 61
> > 67 6f 00 24 steGW..phitago.$
> > [972934023.750000] [ INFO] 0020 30 66 30 62 36 37 33 64 2d 33 64 35
> > 66 2d 34 38 0f0b673d-3d5f-48
> > [972934023.760000] [ INFO] 0030 38 62 2d 62 38 62 36 2d 62 36 35 36
> > 35 66 37 39 8b-b8b6-b6565f79
> > [972934023.770000] [ INFO] 0040 66 32 66 64
> > f2fd
> > [972934023.780000] [ INFO] tcp_max_wrb_size: tcp_max_wrb_size = 0 for
> > conn 0x20006e78
> > [972934023.780000] [ INFO] psock_tcp_send: new wrb 0x200073fc (non blocking)
> > [972934023.790000] [ INFO] iob_copyin_internal: iob=0x20004238 len=0 offset=0
> >
> > The send() result is 0. And keeps always in this way.
> >
> > And below we have the traces for the personal broker transmission.
> >
> > [972932455.670000] [ INFO] psock_vfcntl: sockfd=0x10007a70 cmd=9
> > [972932455.670000] [ INFO] psock_tcp_send (0x1000a9ac):
> > [972932455.680000] [ INFO] 0000 10 42 00 04 4d 51 54 54 04 c2 01 90
> > 00 07 54 65 .B..MQTT......Te
> > [972932455.690000] [ INFO] 0010 73 74 65 47 57 00 07 70 68 69 74 61
> > 67 6f 00 24 steGW..phitago.$
> > [972932455.700000] [ INFO] 0020 30 66 30 62 36 37 33 64 2d 33 64 35
> > 66 2d 34 38 0f0b673d-3d5f-48
> > [972932455.710000] [ INFO] 0030 38 62 2d 62 38 62 36 2d 62 36 35 36
> > 35 66 37 39 8b-b8b6-b6565f79
> > [972932455.710000] [ INFO] 0040 66 32 66 64
> > f2fd
> > [972932455.720000] [ INFO] tcp_max_wrb_size: tcp_max_wrb_size = 2144
> > for conn 0x20006e78
> > [972932455.730000] [ INFO] psock_tcp_send: new wrb 0x200073fc (non blocking)
> > [972932455.740000] [ INFO] iob_copyin_internal: iob=0x20004238 len=68 offset=0
> > [972932455.750000] [ INFO] iob_copyin_internal: iob=0x20004238
> > avail=0 len=68 next=0
> >
> > In this case, send return a value bigger than 0.
> >
> > Could you please give me any advice to understand what's going on? Or
> > what can I check?
> >
> > The connect() response is 0 in both cases.
> >
> > Using PC-based tools, such as MQTT.Fx, I can connect to the host.
> >
> > Best regards,
> >
> > Flavio
> >
> > Em sáb., 26 de jun. de 2021 às 13:57, Gregory Nutt
> > <sp...@gmail.com> escreveu:
> >>
> >>
> >>> Is there any chance that the broker didn't like the mqtt connect
> >>> message and shutdown the connection and leaving my device alone ...
> >>> all during send call?
> >> If the peer disconnects, the socket should be marked disconnected and
> >> the send() should fail with an error.
> >
> >
> >
>
> --
> Johnny Billquist || "I'm on a bus
> || on a psychedelic trip
> email: bqt@softjar.se || Reading murder books
> pdp is alive! || tryin' to stay hip" - B. Idol
--
Flavio de Castro Alves Filho
flavio.alves@gmail.com
Twitter: http://twitter.com/#!/fraviofii
LinkedIn profile: www.linkedin.com/in/flaviocastroalves
Re: Socket programming: send() function blocked
Posted by Johnny Billquist <bq...@softjar.se>.
Something is seriously wrong in your first run.
Maybe a tcpdump would expose something, but basically, tcp_max_wrb_size
should never be 0, but in your first run it appears to be.
As that is the buffer size to use, you end up with a tcp connection with
a buffer size of 0, so of course all writes (including send) will return
with success, but with 0 bytes written, since there wasn't space for
anything more.
tcp_max_wrb_size should be based on mss, which suggests that mss would
be zero. But that is rather surprising. Since the mss is exchanged
between the hosts at tcp connection establishment, it would be very
interesting to report what mss each side is suggesting.
It might be that one side isn't reporting any mss at all, and then the
tcp implementation in Nuttx behaves badly. I don't really have time to
dive into this more right now, but this should give you some pointers
and ideas to where to look further.
Johnny
On 2021-06-27 16:34, Flavio Castro Alves Filho wrote:
> I'm sorry ... I got the same behavior, but the send function now
> started to work non-blocking.
>
> Here is my situation:
>
> * I have 2 mqtt brokers (tcp servers)
> - one commercial (Tago.io)
> - one personal (Mosquitto)
>
> After a successful connection in both cases, the next step is to send
> the mqtt connect message.
>
> Below shows the execution trace for the commercial broker.
>
> [972934023.720000] [ INFO] psock_vfcntl: sockfd=0x10007a70 cmd=9
> [972934023.730000] [ INFO] psock_tcp_send (0x1000a9ac):
> [972934023.730000] [ INFO] 0000 10 42 00 04 4d 51 54 54 04 c2 01 90
> 00 07 54 65 .B..MQTT......Te
> [972934023.740000] [ INFO] 0010 73 74 65 47 57 00 07 70 68 69 74 61
> 67 6f 00 24 steGW..phitago.$
> [972934023.750000] [ INFO] 0020 30 66 30 62 36 37 33 64 2d 33 64 35
> 66 2d 34 38 0f0b673d-3d5f-48
> [972934023.760000] [ INFO] 0030 38 62 2d 62 38 62 36 2d 62 36 35 36
> 35 66 37 39 8b-b8b6-b6565f79
> [972934023.770000] [ INFO] 0040 66 32 66 64
> f2fd
> [972934023.780000] [ INFO] tcp_max_wrb_size: tcp_max_wrb_size = 0 for
> conn 0x20006e78
> [972934023.780000] [ INFO] psock_tcp_send: new wrb 0x200073fc (non blocking)
> [972934023.790000] [ INFO] iob_copyin_internal: iob=0x20004238 len=0 offset=0
>
> The send() result is 0. And keeps always in this way.
>
> And below we have the traces for the personal broker transmission.
>
> [972932455.670000] [ INFO] psock_vfcntl: sockfd=0x10007a70 cmd=9
> [972932455.670000] [ INFO] psock_tcp_send (0x1000a9ac):
> [972932455.680000] [ INFO] 0000 10 42 00 04 4d 51 54 54 04 c2 01 90
> 00 07 54 65 .B..MQTT......Te
> [972932455.690000] [ INFO] 0010 73 74 65 47 57 00 07 70 68 69 74 61
> 67 6f 00 24 steGW..phitago.$
> [972932455.700000] [ INFO] 0020 30 66 30 62 36 37 33 64 2d 33 64 35
> 66 2d 34 38 0f0b673d-3d5f-48
> [972932455.710000] [ INFO] 0030 38 62 2d 62 38 62 36 2d 62 36 35 36
> 35 66 37 39 8b-b8b6-b6565f79
> [972932455.710000] [ INFO] 0040 66 32 66 64
> f2fd
> [972932455.720000] [ INFO] tcp_max_wrb_size: tcp_max_wrb_size = 2144
> for conn 0x20006e78
> [972932455.730000] [ INFO] psock_tcp_send: new wrb 0x200073fc (non blocking)
> [972932455.740000] [ INFO] iob_copyin_internal: iob=0x20004238 len=68 offset=0
> [972932455.750000] [ INFO] iob_copyin_internal: iob=0x20004238
> avail=0 len=68 next=0
>
> In this case, send return a value bigger than 0.
>
> Could you please give me any advice to understand what's going on? Or
> what can I check?
>
> The connect() response is 0 in both cases.
>
> Using PC-based tools, such as MQTT.Fx, I can connect to the host.
>
> Best regards,
>
> Flavio
>
> Em sáb., 26 de jun. de 2021 às 13:57, Gregory Nutt
> <sp...@gmail.com> escreveu:
>>
>>
>>> Is there any chance that the broker didn't like the mqtt connect
>>> message and shutdown the connection and leaving my device alone ...
>>> all during send call?
>> If the peer disconnects, the socket should be marked disconnected and
>> the send() should fail with an error.
>
>
>
--
Johnny Billquist || "I'm on a bus
|| on a psychedelic trip
email: bqt@softjar.se || Reading murder books
pdp is alive! || tryin' to stay hip" - B. Idol
Re: Socket programming: send() function blocked
Posted by Flavio Castro Alves Filho <fl...@gmail.com>.
I'm sorry ... I got the same behavior, but the send function now
started to work non-blocking.
Here is my situation:
* I have 2 mqtt brokers (tcp servers)
- one commercial (Tago.io)
- one personal (Mosquitto)
After a successful connection in both cases, the next step is to send
the mqtt connect message.
Below shows the execution trace for the commercial broker.
[972934023.720000] [ INFO] psock_vfcntl: sockfd=0x10007a70 cmd=9
[972934023.730000] [ INFO] psock_tcp_send (0x1000a9ac):
[972934023.730000] [ INFO] 0000 10 42 00 04 4d 51 54 54 04 c2 01 90
00 07 54 65 .B..MQTT......Te
[972934023.740000] [ INFO] 0010 73 74 65 47 57 00 07 70 68 69 74 61
67 6f 00 24 steGW..phitago.$
[972934023.750000] [ INFO] 0020 30 66 30 62 36 37 33 64 2d 33 64 35
66 2d 34 38 0f0b673d-3d5f-48
[972934023.760000] [ INFO] 0030 38 62 2d 62 38 62 36 2d 62 36 35 36
35 66 37 39 8b-b8b6-b6565f79
[972934023.770000] [ INFO] 0040 66 32 66 64
f2fd
[972934023.780000] [ INFO] tcp_max_wrb_size: tcp_max_wrb_size = 0 for
conn 0x20006e78
[972934023.780000] [ INFO] psock_tcp_send: new wrb 0x200073fc (non blocking)
[972934023.790000] [ INFO] iob_copyin_internal: iob=0x20004238 len=0 offset=0
The send() result is 0. And keeps always in this way.
And below we have the traces for the personal broker transmission.
[972932455.670000] [ INFO] psock_vfcntl: sockfd=0x10007a70 cmd=9
[972932455.670000] [ INFO] psock_tcp_send (0x1000a9ac):
[972932455.680000] [ INFO] 0000 10 42 00 04 4d 51 54 54 04 c2 01 90
00 07 54 65 .B..MQTT......Te
[972932455.690000] [ INFO] 0010 73 74 65 47 57 00 07 70 68 69 74 61
67 6f 00 24 steGW..phitago.$
[972932455.700000] [ INFO] 0020 30 66 30 62 36 37 33 64 2d 33 64 35
66 2d 34 38 0f0b673d-3d5f-48
[972932455.710000] [ INFO] 0030 38 62 2d 62 38 62 36 2d 62 36 35 36
35 66 37 39 8b-b8b6-b6565f79
[972932455.710000] [ INFO] 0040 66 32 66 64
f2fd
[972932455.720000] [ INFO] tcp_max_wrb_size: tcp_max_wrb_size = 2144
for conn 0x20006e78
[972932455.730000] [ INFO] psock_tcp_send: new wrb 0x200073fc (non blocking)
[972932455.740000] [ INFO] iob_copyin_internal: iob=0x20004238 len=68 offset=0
[972932455.750000] [ INFO] iob_copyin_internal: iob=0x20004238
avail=0 len=68 next=0
In this case, send return a value bigger than 0.
Could you please give me any advice to understand what's going on? Or
what can I check?
The connect() response is 0 in both cases.
Using PC-based tools, such as MQTT.Fx, I can connect to the host.
Best regards,
Flavio
Em sáb., 26 de jun. de 2021 às 13:57, Gregory Nutt
<sp...@gmail.com> escreveu:
>
>
> > Is there any chance that the broker didn't like the mqtt connect
> > message and shutdown the connection and leaving my device alone ...
> > all during send call?
> If the peer disconnects, the socket should be marked disconnected and
> the send() should fail with an error.
--
Flavio de Castro Alves Filho
flavio.alves@gmail.com
Twitter: http://twitter.com/#!/fraviofii
LinkedIn profile: www.linkedin.com/in/flaviocastroalves
Re: Socket programming: send() function blocked
Posted by Gregory Nutt <sp...@gmail.com>.
> Is there any chance that the broker didn't like the mqtt connect
> message and shutdown the connection and leaving my device alone ...
> all during send call?
If the peer disconnects, the socket should be marked disconnected and
the send() should fail with an error.
Re: Socket programming: send() function blocked
Posted by Flavio Castro Alves Filho <fl...@gmail.com>.
Is there any chance that the broker didn't like the mqtt connect
message and shutdown the connection and leaving my device alone ...
all during send call?
Em sáb., 26 de jun. de 2021 às 11:42, Flavio Castro Alves Filho
<fl...@gmail.com> escreveu:
>
> Hello Greg,
>
> No. It was not set :-(
>
> I turned on the configuration, but the problem persisted. :-(
>
> Best regards,
>
> Flavio
>
> Em sáb., 26 de jun. de 2021 às 10:41, Gregory Nutt
> <sp...@gmail.com> escreveu:
> >
> > Do you have CONFIG_NET_TCP_WRITE_BUFFERS=y selected? It is required for
> > TCP non-blocking send.
> >
>
>
> --
> Flavio de Castro Alves Filho
>
> flavio.alves@gmail.com
> Twitter: http://twitter.com/#!/fraviofii
> LinkedIn profile: www.linkedin.com/in/flaviocastroalves
--
Flavio de Castro Alves Filho
flavio.alves@gmail.com
Twitter: http://twitter.com/#!/fraviofii
LinkedIn profile: www.linkedin.com/in/flaviocastroalves
Re: Socket programming: send() function blocked
Posted by Flavio Castro Alves Filho <fl...@gmail.com>.
Hello Greg,
No. It was not set :-(
I turned on the configuration, but the problem persisted. :-(
Best regards,
Flavio
Em sáb., 26 de jun. de 2021 às 10:41, Gregory Nutt
<sp...@gmail.com> escreveu:
>
> Do you have CONFIG_NET_TCP_WRITE_BUFFERS=y selected? It is required for
> TCP non-blocking send.
>
--
Flavio de Castro Alves Filho
flavio.alves@gmail.com
Twitter: http://twitter.com/#!/fraviofii
LinkedIn profile: www.linkedin.com/in/flaviocastroalves
Re: Socket programming: send() function blocked
Posted by Gregory Nutt <sp...@gmail.com>.
Do you have CONFIG_NET_TCP_WRITE_BUFFERS=y selected? It is required for
TCP non-blocking send.