You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nuttx.apache.org by Fotis Panagiotopoulos <f....@gmail.com> on 2023/06/04 14:23:17 UTC
Re: sendfile() problems - Destination unreachable
I am trying this new PMTUD functionality, but it seems it doesn't work.
I have configured:
CONFIG_NET_ICMP_PMTU_ENTRIES = 10
CONFIG_NET_ICMP_PMTU_TIMEOUT = 10
Again I see lots of segments being sent (all with size 1400), and all of
them are responded with the same ICMP reply.
A couple of retransmissions are attempted, and then the connection is reset.
I try again the same procedure (now that the system had the chance to
discover the maximum PMTU), but it still fails.
Again all segments have a size of 1400, instead of less.
I am using sendfile() in case this matters.
On Tue, May 30, 2023 at 12:28 AM Fotis Panagiotopoulos <f....@gmail.com>
wrote:
> I can see that there are two different problems with MTU.
> They are completely independent from each other, so let's start with the
> simple case first.
>
> I am testing on an STM32F427, using Ethernet.
>
> As previously noted, the following code will cause the running task to
> hang.
>
> netlib_set_mtu(CONFIG_NETIF_DEV_NAME, 1500);
>
> int sd = socket(AF_INET, SOCK_DGRAM, 0);
>
> struct sockaddr_in server;
> server.sin_family = AF_INET;
> server.sin_port = 1000;
> server.sin_addr.s_addr = inet_addr("192.168.1.235");
>
> uint8_t * data = malloc(2048);
> memset(data, 0xAA, 2048);
>
> sendto(sd, data, 2048, 0, (struct sockaddr*)&server, sizeof(server));
>
> close(sd);
>
> As you can see, the MTU is set to 1500, and then I try to send a UDP
> datagram with a larger size (2048).
> Indeed `devif_send()` fails, and the aforementioned semaphore is never
> posted.
>
> (This is without buffering in UTP, in case this is important).
>
> This draft PR, provides a solution to the issue.
> https://github.com/apache/nuttx/pull/9423
>
> If this is correct, I will also check buffered UDP, and other uses of
> devif_send().
>
> Alternatively, devif_send() may be changed to actually return an error
> code (instead of returning void), so improved error handling can take place.
>
>
> On Mon, May 29, 2023 at 1:41 PM Fotis Panagiotopoulos <f....@gmail.com>
> wrote:
>
>> The failure scenario is a bit more complicated...
>>
>> Give me some time and I will provide a correct and reproducible example,
>> with a clear explanation.
>>
>> On Mon, May 29, 2023, 13:27 Fotis Panagiotopoulos <f....@gmail.com>
>> wrote:
>>
>>> > sendfile should return an error in this case, but senfile should only
>>> be
>>> > used with TCP, not UDP, since sendfile doesn't have any logic to ack or
>>> > retry..
>>>
>>> Sorry if this wasn't clear. This last test was with plain old `send()`...
>>>
>>> I opened a UDP socket, and used `send()` to transmit a buffer larger
>>> than the MTU.
>>> Instead of getting an error, the application hangs indefinitely.
>>> `devif_send()` is called periodically, but of course it always fails.
>>>
>>>
>>>
>>> On Mon, May 29, 2023 at 1:13 PM Xiang Xiao <xi...@gmail.com>
>>> wrote:
>>>
>>>> On Mon, May 29, 2023 at 5:02 PM Fotis Panagiotopoulos <
>>>> f.j.panag@gmail.com>
>>>> wrote:
>>>>
>>>> > > You need to enable IP fragmentation in this case, which is also
>>>> added
>>>> > > recently and disabled by default:
>>>> > > https://github.com/apache/nuttx/pull/8059
>>>> > <https://github.com/apache/nuttx/pull/8059>
>>>> > > Otherwise, any packet bigger than MTU will be dropped silently.
>>>> >
>>>> > Yes, this is the expected behavior.
>>>> > But, instead of dropping the packet, the system hangs because the
>>>> semaphore
>>>> > is never posted.
>>>> > It just tries endlessly to call devif_send() which always fails.
>>>> >
>>>> >
>>>> sendfile should return an error in this case, but senfile should only be
>>>> used with TCP, not UDP, since sendfile doesn't have any logic to ack or
>>>> retry..
>>>>
>>>>
>>>> >
>>>> >
>>>> > On Mon, May 29, 2023 at 11:42 AM Xiang Xiao <
>>>> xiaoxiang781216@gmail.com>
>>>> > wrote:
>>>> >
>>>> > > On Sun, May 28, 2023 at 11:55 PM Fotis Panagiotopoulos <
>>>> > > f.j.panag@gmail.com>
>>>> > > wrote:
>>>> > >
>>>> > > > While experimenting with MTU, and checking the stability of my
>>>> system,
>>>> > I
>>>> > > > noticed the following.
>>>> > > >
>>>> > > > I try to send a UDP datagram that is larger than the configured
>>>> MTU.
>>>> > > > In this case, the offending thread seems to hang indefinitely (or
>>>> at
>>>> > > least
>>>> > > > waiting for a very long timeout?)
>>>> > > >
>>>> > >
>>>> > > You need to enable IP fragmentation in this case, which is also
>>>> added
>>>> > > recently and disabled by default:
>>>> > > https://github.com/apache/nuttx/pull/8059
>>>> > > Otherwise, any packet bigger than MTU will be dropped silently.
>>>> > >
>>>> > >
>>>> > > > The problem seems to be this line:
>>>> > > >
>>>> > > >
>>>> > >
>>>> >
>>>> https://github.com/apache/nuttx/blob/master/net/udp/udp_sendto_unbuffered.c#L197
>>>> > > > `devif_send()` fails because the datagram is too large, but
>>>> > > > `pstate->st_sem` is never posted (the code returns immediately).
>>>> > > >
>>>> > > > This leaves the sending task to be blocked here:
>>>> > > >
>>>> > > >
>>>> > >
>>>> >
>>>> https://github.com/apache/nuttx/blob/master/net/udp/udp_sendto_unbuffered.c#L469
>>>> > > >
>>>> > > > Shouldn't this failure also post the semaphore?
>>>> > > > And let the code proceed returning an error in `send()`?
>>>> > > >
>>>> > > >
>>>> > > > On Sun, May 28, 2023 at 5:26 PM Fotis Panagiotopoulos <
>>>> > > f.j.panag@gmail.com
>>>> > > > >
>>>> > > > wrote:
>>>> > > >
>>>> > > > >
>>>> > > > > On Sat, May 27, 2023 at 5:35 PM Xiang Xiao <
>>>> > xiaoxiang781216@gmail.com>
>>>> > > > > wrote:
>>>> > > > >
>>>> > > > >> On Sat, May 27, 2023 at 8:19 PM Fotis Panagiotopoulos <
>>>> > > > >> f.j.panag@gmail.com>
>>>> > > > >> wrote:
>>>> > > > >>
>>>> > > > >> > Hello,
>>>> > > > >> >
>>>> > > > >> > I encounter some problems using sendfile().
>>>> > > > >> >
>>>> > > > >> > I am using sendfile to... send a file to a remote server,
>>>> with my
>>>> > > own
>>>> > > > >> > implementation of an FTP client.
>>>> > > > >> > sendfile() indeed starts to transmit chunks of the file, but
>>>> as I
>>>> > > see
>>>> > > > in
>>>> > > > >> > Wireshark, I get an ICMP response "Destination unreachable
>>>> > > > >> (Fragmentation
>>>> > > > >> > needed)".
>>>> > > > >> > I have verified that the Ethrenet MTU is correctly set to
>>>> 1500.
>>>> > > > >> >
>>>> > > > >> > I tried lowering the MTU a lot (1000 bytes), and the problem
>>>> is
>>>> > > > solved.
>>>> > > > >> > Communication succeeds.
>>>> > > > >> >
>>>> > > > >> > This raises some questions, and indicates some potential
>>>> bugs:
>>>> > > > >> >
>>>> > > > >> > 1. Why is there a problem with MTU in the first place?
>>>> Shouldn't
>>>> > MTU
>>>> > > > be
>>>> > > > >> > negotiated? (Is this functionality available in NuttX?)
>>>> > > > >> >
>>>> > > > >>
>>>> > > > >> MTU isn't negotiated but a physical attribute of your
>>>> > > transport(netdev).
>>>> > > > >> On
>>>> > > > >> the other hand, PMTU could be discovered from ICMP.
>>>> > > > >>
>>>> > > > >
>>>> > > > > I am not very familiar with MTU negotiation, so it seems that it
>>>> > > doesn't
>>>> > > > > happen in the network layer that I thought...
>>>> > > > >
>>>> > > > >
>>>> > > > >>
>>>> > > > >>
>>>> > > > >> > 2. Why is the ICMP response not handled? It seems that
>>>> sendfile()
>>>> > > just
>>>> > > > >> > ignores it and continues to send chunks, nevertheless.
>>>> > > > >> >
>>>> > > > >>
>>>> > > > >> It is handled by the recent addition here:
>>>> > > > >> https://github.com/apachey/nuttx/pull/9254
>>>> > > > >> <https://github.com/apache/nuttx/pull/9254>
>>>> > > > >> but this feature is disabled by default, you have to enable it
>>>> > > > manually..
>>>> > > > >>
>>>> > > > >
>>>> > > > > I will definitely take a look at this. Thank you.
>>>> > > > >
>>>> > > > >
>>>> > > > >>
>>>> > > > >>
>>>> > > > >
>>>> > > > >> > 3. Why sendfile() sends TCP segments without receiving any
>>>> ACKs
>>>> > > back?
>>>> > > > >> > AFAIK, depending on the configuration, TCP allows at most two
>>>> > > pending
>>>> > > > >> > segments on the wire. But I see dozens of them, till sendfile
>>>> > > finally
>>>> > > > >> > fails.
>>>> > > > >> >
>>>> > > > >> >
>>>> > > > >> Why only two segments? TCP can send packages until the slide
>>>> window
>>>> > is
>>>> > > > >> full.
>>>> > > > >>
>>>> > > > >> Disregard this. I was confused with delayed ACKs. Which is a
>>>> > > receiver's
>>>> > > > > functionality, not a sender's...
>>>> > > > >
>>>> > > > >
>>>> > > > >>
>>>> > > > >> > This last point is also verified in my MQTT client.
>>>> > > > >> > I have seen NuttX TCP allowing sending lots of TCP segments
>>>> > without
>>>> > > > >> ACKing
>>>> > > > >> > the previous data.
>>>> > > > >> >
>>>> > > > >> > So, is there any insight on the above?
>>>> > > > >> > Is my configuration wrong, or is there anything wrong with
>>>> TCP?
>>>> > > > >> >
>>>> > > > >> > Thank you.
>>>> > > > >> >
>>>> > > > >>
>>>> > > > >
>>>> > > >
>>>> > >
>>>> >
>>>>
>>>
Re: sendfile() problems - Destination unreachable
Posted by Xiang Xiao <xi...@gmail.com>.
On Sun, Jun 4, 2023 at 10:40 PM Fotis Panagiotopoulos <f....@gmail.com>
wrote:
> As said, this is sendfile(). I do not have control on the size of the
> chunks sent. sendfile is also using TCP.
>
> So, sendfile cannot take advantage of PMTUD?
>
From thttps://
github.com/apache/nuttx/blob/master/net/tcp/tcp_sendfile.c#L259-L262:
sendfile never sends one package larger than MSS which should smaller than
MTU in most case.
> On Sun, Jun 4, 2023 at 5:33 PM Xiang Xiao <xi...@gmail.com>
> wrote:
>
> > On Sun, Jun 4, 2023 at 10:23 PM Fotis Panagiotopoulos <
> f.j.panag@gmail.com
> > >
> > wrote:
> >
> > > I am trying this new PMTUD functionality, but it seems it doesn't work.
> > >
> > > I have configured:
> > > CONFIG_NET_ICMP_PMTU_ENTRIES = 10
> > > CONFIG_NET_ICMP_PMTU_TIMEOUT = 10
> > >
> > > Again I see lots of segments being sent (all with size 1400), and all
> of
> > > them are responded with the same ICMP reply.
> > > A couple of retransmissions are attempted, and then the connection is
> > > reset.
> > >
> >
> > The option can discover the minimal MTU from the source to the
> destination.
> > You need to use PMTU to split your data into small packets(<= PMTU) by
> > yourself to improve the efficiency.
> >
> >
> > >
> > > I try again the same procedure (now that the system had the chance to
> > > discover the maximum PMTU), but it still fails.
> > > Again all segments have a size of 1400, instead of less.
> > >
> >
> > If you don't split your package to fit MTU by yourself, you have
> > to enable NET_IPFRAG. But I would suggest that.you switch UDP to TCP
> > because the protocol you implement on top of UDP is likely very
> > inefficient.
> >
> >
> > >
> > > I am using sendfile() in case this matters.
> > >
> > > On Tue, May 30, 2023 at 12:28 AM Fotis Panagiotopoulos <
> > > f.j.panag@gmail.com>
> > > wrote:
> > >
> > > > I can see that there are two different problems with MTU.
> > > > They are completely independent from each other, so let's start with
> > the
> > > > simple case first.
> > > >
> > > > I am testing on an STM32F427, using Ethernet.
> > > >
> > > > As previously noted, the following code will cause the running task
> to
> > > > hang.
> > > >
> > > > netlib_set_mtu(CONFIG_NETIF_DEV_NAME, 1500);
> > > >
> > > > int sd = socket(AF_INET, SOCK_DGRAM, 0);
> > > >
> > > > struct sockaddr_in server;
> > > > server.sin_family = AF_INET;
> > > > server.sin_port = 1000;
> > > > server.sin_addr.s_addr = inet_addr("192.168.1.235");
> > > >
> > > > uint8_t * data = malloc(2048);
> > > > memset(data, 0xAA, 2048);
> > > >
> > > > sendto(sd, data, 2048, 0, (struct sockaddr*)&server, sizeof(server));
> > > >
> > > > close(sd);
> > > >
> > > > As you can see, the MTU is set to 1500, and then I try to send a UDP
> > > > datagram with a larger size (2048).
> > > > Indeed `devif_send()` fails, and the aforementioned semaphore is
> never
> > > > posted.
> > > >
> > > > (This is without buffering in UTP, in case this is important).
> > > >
> > > > This draft PR, provides a solution to the issue.
> > > > https://github.com/apache/nuttx/pull/9423
> > > >
> > > > If this is correct, I will also check buffered UDP, and other uses of
> > > > devif_send().
> > > >
> > > > Alternatively, devif_send() may be changed to actually return an
> error
> > > > code (instead of returning void), so improved error handling can take
> > > place.
> > > >
> > > >
> > > > On Mon, May 29, 2023 at 1:41 PM Fotis Panagiotopoulos <
> > > f.j.panag@gmail.com>
> > > > wrote:
> > > >
> > > >> The failure scenario is a bit more complicated...
> > > >>
> > > >> Give me some time and I will provide a correct and reproducible
> > example,
> > > >> with a clear explanation.
> > > >>
> > > >> On Mon, May 29, 2023, 13:27 Fotis Panagiotopoulos <
> > f.j.panag@gmail.com>
> > > >> wrote:
> > > >>
> > > >>> > sendfile should return an error in this case, but senfile should
> > only
> > > >>> be
> > > >>> > used with TCP, not UDP, since sendfile doesn't have any logic to
> > ack
> > > or
> > > >>> > retry..
> > > >>>
> > > >>> Sorry if this wasn't clear. This last test was with plain old
> > > `send()`...
> > > >>>
> > > >>> I opened a UDP socket, and used `send()` to transmit a buffer
> larger
> > > >>> than the MTU.
> > > >>> Instead of getting an error, the application hangs indefinitely.
> > > >>> `devif_send()` is called periodically, but of course it always
> fails.
> > > >>>
> > > >>>
> > > >>>
> > > >>> On Mon, May 29, 2023 at 1:13 PM Xiang Xiao <
> > xiaoxiang781216@gmail.com>
> > > >>> wrote:
> > > >>>
> > > >>>> On Mon, May 29, 2023 at 5:02 PM Fotis Panagiotopoulos <
> > > >>>> f.j.panag@gmail.com>
> > > >>>> wrote:
> > > >>>>
> > > >>>> > > You need to enable IP fragmentation in this case, which is
> also
> > > >>>> added
> > > >>>> > > recently and disabled by default:
> > > >>>> > > https://github.com/apache/nuttx/pull/8059
> > > >>>> > <https://github.com/apache/nuttx/pull/8059>
> > > >>>> > > Otherwise, any packet bigger than MTU will be dropped
> silently.
> > > >>>> >
> > > >>>> > Yes, this is the expected behavior.
> > > >>>> > But, instead of dropping the packet, the system hangs because
> the
> > > >>>> semaphore
> > > >>>> > is never posted.
> > > >>>> > It just tries endlessly to call devif_send() which always fails.
> > > >>>> >
> > > >>>> >
> > > >>>> sendfile should return an error in this case, but senfile should
> > only
> > > be
> > > >>>> used with TCP, not UDP, since sendfile doesn't have any logic to
> ack
> > > or
> > > >>>> retry..
> > > >>>>
> > > >>>>
> > > >>>> >
> > > >>>> >
> > > >>>> > On Mon, May 29, 2023 at 11:42 AM Xiang Xiao <
> > > >>>> xiaoxiang781216@gmail.com>
> > > >>>> > wrote:
> > > >>>> >
> > > >>>> > > On Sun, May 28, 2023 at 11:55 PM Fotis Panagiotopoulos <
> > > >>>> > > f.j.panag@gmail.com>
> > > >>>> > > wrote:
> > > >>>> > >
> > > >>>> > > > While experimenting with MTU, and checking the stability of
> my
> > > >>>> system,
> > > >>>> > I
> > > >>>> > > > noticed the following.
> > > >>>> > > >
> > > >>>> > > > I try to send a UDP datagram that is larger than the
> > configured
> > > >>>> MTU.
> > > >>>> > > > In this case, the offending thread seems to hang
> indefinitely
> > > (or
> > > >>>> at
> > > >>>> > > least
> > > >>>> > > > waiting for a very long timeout?)
> > > >>>> > > >
> > > >>>> > >
> > > >>>> > > You need to enable IP fragmentation in this case, which is
> also
> > > >>>> added
> > > >>>> > > recently and disabled by default:
> > > >>>> > > https://github.com/apache/nuttx/pull/8059
> > > >>>> > > Otherwise, any packet bigger than MTU will be dropped
> silently.
> > > >>>> > >
> > > >>>> > >
> > > >>>> > > > The problem seems to be this line:
> > > >>>> > > >
> > > >>>> > > >
> > > >>>> > >
> > > >>>> >
> > > >>>>
> > >
> >
> https://github.com/apache/nuttx/blob/master/net/udp/udp_sendto_unbuffered.c#L197
> > > >>>> > > > `devif_send()` fails because the datagram is too large, but
> > > >>>> > > > `pstate->st_sem` is never posted (the code returns
> > immediately).
> > > >>>> > > >
> > > >>>> > > > This leaves the sending task to be blocked here:
> > > >>>> > > >
> > > >>>> > > >
> > > >>>> > >
> > > >>>> >
> > > >>>>
> > >
> >
> https://github.com/apache/nuttx/blob/master/net/udp/udp_sendto_unbuffered.c#L469
> > > >>>> > > >
> > > >>>> > > > Shouldn't this failure also post the semaphore?
> > > >>>> > > > And let the code proceed returning an error in `send()`?
> > > >>>> > > >
> > > >>>> > > >
> > > >>>> > > > On Sun, May 28, 2023 at 5:26 PM Fotis Panagiotopoulos <
> > > >>>> > > f.j.panag@gmail.com
> > > >>>> > > > >
> > > >>>> > > > wrote:
> > > >>>> > > >
> > > >>>> > > > >
> > > >>>> > > > > On Sat, May 27, 2023 at 5:35 PM Xiang Xiao <
> > > >>>> > xiaoxiang781216@gmail.com>
> > > >>>> > > > > wrote:
> > > >>>> > > > >
> > > >>>> > > > >> On Sat, May 27, 2023 at 8:19 PM Fotis Panagiotopoulos <
> > > >>>> > > > >> f.j.panag@gmail.com>
> > > >>>> > > > >> wrote:
> > > >>>> > > > >>
> > > >>>> > > > >> > Hello,
> > > >>>> > > > >> >
> > > >>>> > > > >> > I encounter some problems using sendfile().
> > > >>>> > > > >> >
> > > >>>> > > > >> > I am using sendfile to... send a file to a remote
> server,
> > > >>>> with my
> > > >>>> > > own
> > > >>>> > > > >> > implementation of an FTP client.
> > > >>>> > > > >> > sendfile() indeed starts to transmit chunks of the
> file,
> > > but
> > > >>>> as I
> > > >>>> > > see
> > > >>>> > > > in
> > > >>>> > > > >> > Wireshark, I get an ICMP response "Destination
> > unreachable
> > > >>>> > > > >> (Fragmentation
> > > >>>> > > > >> > needed)".
> > > >>>> > > > >> > I have verified that the Ethrenet MTU is correctly set
> to
> > > >>>> 1500.
> > > >>>> > > > >> >
> > > >>>> > > > >> > I tried lowering the MTU a lot (1000 bytes), and the
> > > problem
> > > >>>> is
> > > >>>> > > > solved.
> > > >>>> > > > >> > Communication succeeds.
> > > >>>> > > > >> >
> > > >>>> > > > >> > This raises some questions, and indicates some
> potential
> > > >>>> bugs:
> > > >>>> > > > >> >
> > > >>>> > > > >> > 1. Why is there a problem with MTU in the first place?
> > > >>>> Shouldn't
> > > >>>> > MTU
> > > >>>> > > > be
> > > >>>> > > > >> > negotiated? (Is this functionality available in NuttX?)
> > > >>>> > > > >> >
> > > >>>> > > > >>
> > > >>>> > > > >> MTU isn't negotiated but a physical attribute of your
> > > >>>> > > transport(netdev).
> > > >>>> > > > >> On
> > > >>>> > > > >> the other hand, PMTU could be discovered from ICMP.
> > > >>>> > > > >>
> > > >>>> > > > >
> > > >>>> > > > > I am not very familiar with MTU negotiation, so it seems
> > that
> > > it
> > > >>>> > > doesn't
> > > >>>> > > > > happen in the network layer that I thought...
> > > >>>> > > > >
> > > >>>> > > > >
> > > >>>> > > > >>
> > > >>>> > > > >>
> > > >>>> > > > >> > 2. Why is the ICMP response not handled? It seems that
> > > >>>> sendfile()
> > > >>>> > > just
> > > >>>> > > > >> > ignores it and continues to send chunks, nevertheless.
> > > >>>> > > > >> >
> > > >>>> > > > >>
> > > >>>> > > > >> It is handled by the recent addition here:
> > > >>>> > > > >> https://github.com/apachey/nuttx/pull/9254
> > > >>>> > > > >> <https://github.com/apache/nuttx/pull/9254>
> > > >>>> > > > >> but this feature is disabled by default, you have to
> enable
> > > it
> > > >>>> > > > manually..
> > > >>>> > > > >>
> > > >>>> > > > >
> > > >>>> > > > > I will definitely take a look at this. Thank you.
> > > >>>> > > > >
> > > >>>> > > > >
> > > >>>> > > > >>
> > > >>>> > > > >>
> > > >>>> > > > >
> > > >>>> > > > >> > 3. Why sendfile() sends TCP segments without receiving
> > any
> > > >>>> ACKs
> > > >>>> > > back?
> > > >>>> > > > >> > AFAIK, depending on the configuration, TCP allows at
> most
> > > two
> > > >>>> > > pending
> > > >>>> > > > >> > segments on the wire. But I see dozens of them, till
> > > sendfile
> > > >>>> > > finally
> > > >>>> > > > >> > fails.
> > > >>>> > > > >> >
> > > >>>> > > > >> >
> > > >>>> > > > >> Why only two segments? TCP can send packages until the
> > slide
> > > >>>> window
> > > >>>> > is
> > > >>>> > > > >> full.
> > > >>>> > > > >>
> > > >>>> > > > >> Disregard this. I was confused with delayed ACKs. Which
> is
> > a
> > > >>>> > > receiver's
> > > >>>> > > > > functionality, not a sender's...
> > > >>>> > > > >
> > > >>>> > > > >
> > > >>>> > > > >>
> > > >>>> > > > >> > This last point is also verified in my MQTT client.
> > > >>>> > > > >> > I have seen NuttX TCP allowing sending lots of TCP
> > segments
> > > >>>> > without
> > > >>>> > > > >> ACKing
> > > >>>> > > > >> > the previous data.
> > > >>>> > > > >> >
> > > >>>> > > > >> > So, is there any insight on the above?
> > > >>>> > > > >> > Is my configuration wrong, or is there anything wrong
> > with
> > > >>>> TCP?
> > > >>>> > > > >> >
> > > >>>> > > > >> > Thank you.
> > > >>>> > > > >> >
> > > >>>> > > > >>
> > > >>>> > > > >
> > > >>>> > > >
> > > >>>> > >
> > > >>>> >
> > > >>>>
> > > >>>
> > >
> >
>
Re: sendfile() problems - Destination unreachable
Posted by Fotis Panagiotopoulos <f....@gmail.com>.
As I see, sendfile uses MSS instead of MTU.
Wouldn't it be better to scale also MSS along with MTU when PMTUD is
enabled?
On Sun, Jun 4, 2023 at 5:40 PM Fotis Panagiotopoulos <f....@gmail.com>
wrote:
> As said, this is sendfile(). I do not have control on the size of the
> chunks sent. sendfile is also using TCP.
>
> So, sendfile cannot take advantage of PMTUD?
>
> On Sun, Jun 4, 2023 at 5:33 PM Xiang Xiao <xi...@gmail.com>
> wrote:
>
>> On Sun, Jun 4, 2023 at 10:23 PM Fotis Panagiotopoulos <
>> f.j.panag@gmail.com>
>> wrote:
>>
>> > I am trying this new PMTUD functionality, but it seems it doesn't work.
>> >
>> > I have configured:
>> > CONFIG_NET_ICMP_PMTU_ENTRIES = 10
>> > CONFIG_NET_ICMP_PMTU_TIMEOUT = 10
>> >
>> > Again I see lots of segments being sent (all with size 1400), and all of
>> > them are responded with the same ICMP reply.
>> > A couple of retransmissions are attempted, and then the connection is
>> > reset.
>> >
>>
>> The option can discover the minimal MTU from the source to the
>> destination.
>> You need to use PMTU to split your data into small packets(<= PMTU) by
>> yourself to improve the efficiency.
>>
>>
>> >
>> > I try again the same procedure (now that the system had the chance to
>> > discover the maximum PMTU), but it still fails.
>> > Again all segments have a size of 1400, instead of less.
>> >
>>
>> If you don't split your package to fit MTU by yourself, you have
>> to enable NET_IPFRAG. But I would suggest that.you switch UDP to TCP
>> because the protocol you implement on top of UDP is likely very
>> inefficient.
>>
>>
>> >
>> > I am using sendfile() in case this matters.
>> >
>> > On Tue, May 30, 2023 at 12:28 AM Fotis Panagiotopoulos <
>> > f.j.panag@gmail.com>
>> > wrote:
>> >
>> > > I can see that there are two different problems with MTU.
>> > > They are completely independent from each other, so let's start with
>> the
>> > > simple case first.
>> > >
>> > > I am testing on an STM32F427, using Ethernet.
>> > >
>> > > As previously noted, the following code will cause the running task to
>> > > hang.
>> > >
>> > > netlib_set_mtu(CONFIG_NETIF_DEV_NAME, 1500);
>> > >
>> > > int sd = socket(AF_INET, SOCK_DGRAM, 0);
>> > >
>> > > struct sockaddr_in server;
>> > > server.sin_family = AF_INET;
>> > > server.sin_port = 1000;
>> > > server.sin_addr.s_addr = inet_addr("192.168.1.235");
>> > >
>> > > uint8_t * data = malloc(2048);
>> > > memset(data, 0xAA, 2048);
>> > >
>> > > sendto(sd, data, 2048, 0, (struct sockaddr*)&server, sizeof(server));
>> > >
>> > > close(sd);
>> > >
>> > > As you can see, the MTU is set to 1500, and then I try to send a UDP
>> > > datagram with a larger size (2048).
>> > > Indeed `devif_send()` fails, and the aforementioned semaphore is never
>> > > posted.
>> > >
>> > > (This is without buffering in UTP, in case this is important).
>> > >
>> > > This draft PR, provides a solution to the issue.
>> > > https://github.com/apache/nuttx/pull/9423
>> > >
>> > > If this is correct, I will also check buffered UDP, and other uses of
>> > > devif_send().
>> > >
>> > > Alternatively, devif_send() may be changed to actually return an error
>> > > code (instead of returning void), so improved error handling can take
>> > place.
>> > >
>> > >
>> > > On Mon, May 29, 2023 at 1:41 PM Fotis Panagiotopoulos <
>> > f.j.panag@gmail.com>
>> > > wrote:
>> > >
>> > >> The failure scenario is a bit more complicated...
>> > >>
>> > >> Give me some time and I will provide a correct and reproducible
>> example,
>> > >> with a clear explanation.
>> > >>
>> > >> On Mon, May 29, 2023, 13:27 Fotis Panagiotopoulos <
>> f.j.panag@gmail.com>
>> > >> wrote:
>> > >>
>> > >>> > sendfile should return an error in this case, but senfile should
>> only
>> > >>> be
>> > >>> > used with TCP, not UDP, since sendfile doesn't have any logic to
>> ack
>> > or
>> > >>> > retry..
>> > >>>
>> > >>> Sorry if this wasn't clear. This last test was with plain old
>> > `send()`...
>> > >>>
>> > >>> I opened a UDP socket, and used `send()` to transmit a buffer larger
>> > >>> than the MTU.
>> > >>> Instead of getting an error, the application hangs indefinitely.
>> > >>> `devif_send()` is called periodically, but of course it always
>> fails.
>> > >>>
>> > >>>
>> > >>>
>> > >>> On Mon, May 29, 2023 at 1:13 PM Xiang Xiao <
>> xiaoxiang781216@gmail.com>
>> > >>> wrote:
>> > >>>
>> > >>>> On Mon, May 29, 2023 at 5:02 PM Fotis Panagiotopoulos <
>> > >>>> f.j.panag@gmail.com>
>> > >>>> wrote:
>> > >>>>
>> > >>>> > > You need to enable IP fragmentation in this case, which is also
>> > >>>> added
>> > >>>> > > recently and disabled by default:
>> > >>>> > > https://github.com/apache/nuttx/pull/8059
>> > >>>> > <https://github.com/apache/nuttx/pull/8059>
>> > >>>> > > Otherwise, any packet bigger than MTU will be dropped silently.
>> > >>>> >
>> > >>>> > Yes, this is the expected behavior.
>> > >>>> > But, instead of dropping the packet, the system hangs because the
>> > >>>> semaphore
>> > >>>> > is never posted.
>> > >>>> > It just tries endlessly to call devif_send() which always fails.
>> > >>>> >
>> > >>>> >
>> > >>>> sendfile should return an error in this case, but senfile should
>> only
>> > be
>> > >>>> used with TCP, not UDP, since sendfile doesn't have any logic to
>> ack
>> > or
>> > >>>> retry..
>> > >>>>
>> > >>>>
>> > >>>> >
>> > >>>> >
>> > >>>> > On Mon, May 29, 2023 at 11:42 AM Xiang Xiao <
>> > >>>> xiaoxiang781216@gmail.com>
>> > >>>> > wrote:
>> > >>>> >
>> > >>>> > > On Sun, May 28, 2023 at 11:55 PM Fotis Panagiotopoulos <
>> > >>>> > > f.j.panag@gmail.com>
>> > >>>> > > wrote:
>> > >>>> > >
>> > >>>> > > > While experimenting with MTU, and checking the stability of
>> my
>> > >>>> system,
>> > >>>> > I
>> > >>>> > > > noticed the following.
>> > >>>> > > >
>> > >>>> > > > I try to send a UDP datagram that is larger than the
>> configured
>> > >>>> MTU.
>> > >>>> > > > In this case, the offending thread seems to hang indefinitely
>> > (or
>> > >>>> at
>> > >>>> > > least
>> > >>>> > > > waiting for a very long timeout?)
>> > >>>> > > >
>> > >>>> > >
>> > >>>> > > You need to enable IP fragmentation in this case, which is also
>> > >>>> added
>> > >>>> > > recently and disabled by default:
>> > >>>> > > https://github.com/apache/nuttx/pull/8059
>> > >>>> > > Otherwise, any packet bigger than MTU will be dropped silently.
>> > >>>> > >
>> > >>>> > >
>> > >>>> > > > The problem seems to be this line:
>> > >>>> > > >
>> > >>>> > > >
>> > >>>> > >
>> > >>>> >
>> > >>>>
>> >
>> https://github.com/apache/nuttx/blob/master/net/udp/udp_sendto_unbuffered.c#L197
>> > >>>> > > > `devif_send()` fails because the datagram is too large, but
>> > >>>> > > > `pstate->st_sem` is never posted (the code returns
>> immediately).
>> > >>>> > > >
>> > >>>> > > > This leaves the sending task to be blocked here:
>> > >>>> > > >
>> > >>>> > > >
>> > >>>> > >
>> > >>>> >
>> > >>>>
>> >
>> https://github.com/apache/nuttx/blob/master/net/udp/udp_sendto_unbuffered.c#L469
>> > >>>> > > >
>> > >>>> > > > Shouldn't this failure also post the semaphore?
>> > >>>> > > > And let the code proceed returning an error in `send()`?
>> > >>>> > > >
>> > >>>> > > >
>> > >>>> > > > On Sun, May 28, 2023 at 5:26 PM Fotis Panagiotopoulos <
>> > >>>> > > f.j.panag@gmail.com
>> > >>>> > > > >
>> > >>>> > > > wrote:
>> > >>>> > > >
>> > >>>> > > > >
>> > >>>> > > > > On Sat, May 27, 2023 at 5:35 PM Xiang Xiao <
>> > >>>> > xiaoxiang781216@gmail.com>
>> > >>>> > > > > wrote:
>> > >>>> > > > >
>> > >>>> > > > >> On Sat, May 27, 2023 at 8:19 PM Fotis Panagiotopoulos <
>> > >>>> > > > >> f.j.panag@gmail.com>
>> > >>>> > > > >> wrote:
>> > >>>> > > > >>
>> > >>>> > > > >> > Hello,
>> > >>>> > > > >> >
>> > >>>> > > > >> > I encounter some problems using sendfile().
>> > >>>> > > > >> >
>> > >>>> > > > >> > I am using sendfile to... send a file to a remote
>> server,
>> > >>>> with my
>> > >>>> > > own
>> > >>>> > > > >> > implementation of an FTP client.
>> > >>>> > > > >> > sendfile() indeed starts to transmit chunks of the file,
>> > but
>> > >>>> as I
>> > >>>> > > see
>> > >>>> > > > in
>> > >>>> > > > >> > Wireshark, I get an ICMP response "Destination
>> unreachable
>> > >>>> > > > >> (Fragmentation
>> > >>>> > > > >> > needed)".
>> > >>>> > > > >> > I have verified that the Ethrenet MTU is correctly set
>> to
>> > >>>> 1500.
>> > >>>> > > > >> >
>> > >>>> > > > >> > I tried lowering the MTU a lot (1000 bytes), and the
>> > problem
>> > >>>> is
>> > >>>> > > > solved.
>> > >>>> > > > >> > Communication succeeds.
>> > >>>> > > > >> >
>> > >>>> > > > >> > This raises some questions, and indicates some potential
>> > >>>> bugs:
>> > >>>> > > > >> >
>> > >>>> > > > >> > 1. Why is there a problem with MTU in the first place?
>> > >>>> Shouldn't
>> > >>>> > MTU
>> > >>>> > > > be
>> > >>>> > > > >> > negotiated? (Is this functionality available in NuttX?)
>> > >>>> > > > >> >
>> > >>>> > > > >>
>> > >>>> > > > >> MTU isn't negotiated but a physical attribute of your
>> > >>>> > > transport(netdev).
>> > >>>> > > > >> On
>> > >>>> > > > >> the other hand, PMTU could be discovered from ICMP.
>> > >>>> > > > >>
>> > >>>> > > > >
>> > >>>> > > > > I am not very familiar with MTU negotiation, so it seems
>> that
>> > it
>> > >>>> > > doesn't
>> > >>>> > > > > happen in the network layer that I thought...
>> > >>>> > > > >
>> > >>>> > > > >
>> > >>>> > > > >>
>> > >>>> > > > >>
>> > >>>> > > > >> > 2. Why is the ICMP response not handled? It seems that
>> > >>>> sendfile()
>> > >>>> > > just
>> > >>>> > > > >> > ignores it and continues to send chunks, nevertheless.
>> > >>>> > > > >> >
>> > >>>> > > > >>
>> > >>>> > > > >> It is handled by the recent addition here:
>> > >>>> > > > >> https://github.com/apachey/nuttx/pull/9254
>> > >>>> > > > >> <https://github.com/apache/nuttx/pull/9254>
>> > >>>> > > > >> but this feature is disabled by default, you have to
>> enable
>> > it
>> > >>>> > > > manually..
>> > >>>> > > > >>
>> > >>>> > > > >
>> > >>>> > > > > I will definitely take a look at this. Thank you.
>> > >>>> > > > >
>> > >>>> > > > >
>> > >>>> > > > >>
>> > >>>> > > > >>
>> > >>>> > > > >
>> > >>>> > > > >> > 3. Why sendfile() sends TCP segments without receiving
>> any
>> > >>>> ACKs
>> > >>>> > > back?
>> > >>>> > > > >> > AFAIK, depending on the configuration, TCP allows at
>> most
>> > two
>> > >>>> > > pending
>> > >>>> > > > >> > segments on the wire. But I see dozens of them, till
>> > sendfile
>> > >>>> > > finally
>> > >>>> > > > >> > fails.
>> > >>>> > > > >> >
>> > >>>> > > > >> >
>> > >>>> > > > >> Why only two segments? TCP can send packages until the
>> slide
>> > >>>> window
>> > >>>> > is
>> > >>>> > > > >> full.
>> > >>>> > > > >>
>> > >>>> > > > >> Disregard this. I was confused with delayed ACKs. Which
>> is a
>> > >>>> > > receiver's
>> > >>>> > > > > functionality, not a sender's...
>> > >>>> > > > >
>> > >>>> > > > >
>> > >>>> > > > >>
>> > >>>> > > > >> > This last point is also verified in my MQTT client.
>> > >>>> > > > >> > I have seen NuttX TCP allowing sending lots of TCP
>> segments
>> > >>>> > without
>> > >>>> > > > >> ACKing
>> > >>>> > > > >> > the previous data.
>> > >>>> > > > >> >
>> > >>>> > > > >> > So, is there any insight on the above?
>> > >>>> > > > >> > Is my configuration wrong, or is there anything wrong
>> with
>> > >>>> TCP?
>> > >>>> > > > >> >
>> > >>>> > > > >> > Thank you.
>> > >>>> > > > >> >
>> > >>>> > > > >>
>> > >>>> > > > >
>> > >>>> > > >
>> > >>>> > >
>> > >>>> >
>> > >>>>
>> > >>>
>> >
>>
>
Re: sendfile() problems - Destination unreachable
Posted by Fotis Panagiotopoulos <f....@gmail.com>.
As said, this is sendfile(). I do not have control on the size of the
chunks sent. sendfile is also using TCP.
So, sendfile cannot take advantage of PMTUD?
On Sun, Jun 4, 2023 at 5:33 PM Xiang Xiao <xi...@gmail.com> wrote:
> On Sun, Jun 4, 2023 at 10:23 PM Fotis Panagiotopoulos <f.j.panag@gmail.com
> >
> wrote:
>
> > I am trying this new PMTUD functionality, but it seems it doesn't work.
> >
> > I have configured:
> > CONFIG_NET_ICMP_PMTU_ENTRIES = 10
> > CONFIG_NET_ICMP_PMTU_TIMEOUT = 10
> >
> > Again I see lots of segments being sent (all with size 1400), and all of
> > them are responded with the same ICMP reply.
> > A couple of retransmissions are attempted, and then the connection is
> > reset.
> >
>
> The option can discover the minimal MTU from the source to the destination.
> You need to use PMTU to split your data into small packets(<= PMTU) by
> yourself to improve the efficiency.
>
>
> >
> > I try again the same procedure (now that the system had the chance to
> > discover the maximum PMTU), but it still fails.
> > Again all segments have a size of 1400, instead of less.
> >
>
> If you don't split your package to fit MTU by yourself, you have
> to enable NET_IPFRAG. But I would suggest that.you switch UDP to TCP
> because the protocol you implement on top of UDP is likely very
> inefficient.
>
>
> >
> > I am using sendfile() in case this matters.
> >
> > On Tue, May 30, 2023 at 12:28 AM Fotis Panagiotopoulos <
> > f.j.panag@gmail.com>
> > wrote:
> >
> > > I can see that there are two different problems with MTU.
> > > They are completely independent from each other, so let's start with
> the
> > > simple case first.
> > >
> > > I am testing on an STM32F427, using Ethernet.
> > >
> > > As previously noted, the following code will cause the running task to
> > > hang.
> > >
> > > netlib_set_mtu(CONFIG_NETIF_DEV_NAME, 1500);
> > >
> > > int sd = socket(AF_INET, SOCK_DGRAM, 0);
> > >
> > > struct sockaddr_in server;
> > > server.sin_family = AF_INET;
> > > server.sin_port = 1000;
> > > server.sin_addr.s_addr = inet_addr("192.168.1.235");
> > >
> > > uint8_t * data = malloc(2048);
> > > memset(data, 0xAA, 2048);
> > >
> > > sendto(sd, data, 2048, 0, (struct sockaddr*)&server, sizeof(server));
> > >
> > > close(sd);
> > >
> > > As you can see, the MTU is set to 1500, and then I try to send a UDP
> > > datagram with a larger size (2048).
> > > Indeed `devif_send()` fails, and the aforementioned semaphore is never
> > > posted.
> > >
> > > (This is without buffering in UTP, in case this is important).
> > >
> > > This draft PR, provides a solution to the issue.
> > > https://github.com/apache/nuttx/pull/9423
> > >
> > > If this is correct, I will also check buffered UDP, and other uses of
> > > devif_send().
> > >
> > > Alternatively, devif_send() may be changed to actually return an error
> > > code (instead of returning void), so improved error handling can take
> > place.
> > >
> > >
> > > On Mon, May 29, 2023 at 1:41 PM Fotis Panagiotopoulos <
> > f.j.panag@gmail.com>
> > > wrote:
> > >
> > >> The failure scenario is a bit more complicated...
> > >>
> > >> Give me some time and I will provide a correct and reproducible
> example,
> > >> with a clear explanation.
> > >>
> > >> On Mon, May 29, 2023, 13:27 Fotis Panagiotopoulos <
> f.j.panag@gmail.com>
> > >> wrote:
> > >>
> > >>> > sendfile should return an error in this case, but senfile should
> only
> > >>> be
> > >>> > used with TCP, not UDP, since sendfile doesn't have any logic to
> ack
> > or
> > >>> > retry..
> > >>>
> > >>> Sorry if this wasn't clear. This last test was with plain old
> > `send()`...
> > >>>
> > >>> I opened a UDP socket, and used `send()` to transmit a buffer larger
> > >>> than the MTU.
> > >>> Instead of getting an error, the application hangs indefinitely.
> > >>> `devif_send()` is called periodically, but of course it always fails.
> > >>>
> > >>>
> > >>>
> > >>> On Mon, May 29, 2023 at 1:13 PM Xiang Xiao <
> xiaoxiang781216@gmail.com>
> > >>> wrote:
> > >>>
> > >>>> On Mon, May 29, 2023 at 5:02 PM Fotis Panagiotopoulos <
> > >>>> f.j.panag@gmail.com>
> > >>>> wrote:
> > >>>>
> > >>>> > > You need to enable IP fragmentation in this case, which is also
> > >>>> added
> > >>>> > > recently and disabled by default:
> > >>>> > > https://github.com/apache/nuttx/pull/8059
> > >>>> > <https://github.com/apache/nuttx/pull/8059>
> > >>>> > > Otherwise, any packet bigger than MTU will be dropped silently.
> > >>>> >
> > >>>> > Yes, this is the expected behavior.
> > >>>> > But, instead of dropping the packet, the system hangs because the
> > >>>> semaphore
> > >>>> > is never posted.
> > >>>> > It just tries endlessly to call devif_send() which always fails.
> > >>>> >
> > >>>> >
> > >>>> sendfile should return an error in this case, but senfile should
> only
> > be
> > >>>> used with TCP, not UDP, since sendfile doesn't have any logic to ack
> > or
> > >>>> retry..
> > >>>>
> > >>>>
> > >>>> >
> > >>>> >
> > >>>> > On Mon, May 29, 2023 at 11:42 AM Xiang Xiao <
> > >>>> xiaoxiang781216@gmail.com>
> > >>>> > wrote:
> > >>>> >
> > >>>> > > On Sun, May 28, 2023 at 11:55 PM Fotis Panagiotopoulos <
> > >>>> > > f.j.panag@gmail.com>
> > >>>> > > wrote:
> > >>>> > >
> > >>>> > > > While experimenting with MTU, and checking the stability of my
> > >>>> system,
> > >>>> > I
> > >>>> > > > noticed the following.
> > >>>> > > >
> > >>>> > > > I try to send a UDP datagram that is larger than the
> configured
> > >>>> MTU.
> > >>>> > > > In this case, the offending thread seems to hang indefinitely
> > (or
> > >>>> at
> > >>>> > > least
> > >>>> > > > waiting for a very long timeout?)
> > >>>> > > >
> > >>>> > >
> > >>>> > > You need to enable IP fragmentation in this case, which is also
> > >>>> added
> > >>>> > > recently and disabled by default:
> > >>>> > > https://github.com/apache/nuttx/pull/8059
> > >>>> > > Otherwise, any packet bigger than MTU will be dropped silently.
> > >>>> > >
> > >>>> > >
> > >>>> > > > The problem seems to be this line:
> > >>>> > > >
> > >>>> > > >
> > >>>> > >
> > >>>> >
> > >>>>
> >
> https://github.com/apache/nuttx/blob/master/net/udp/udp_sendto_unbuffered.c#L197
> > >>>> > > > `devif_send()` fails because the datagram is too large, but
> > >>>> > > > `pstate->st_sem` is never posted (the code returns
> immediately).
> > >>>> > > >
> > >>>> > > > This leaves the sending task to be blocked here:
> > >>>> > > >
> > >>>> > > >
> > >>>> > >
> > >>>> >
> > >>>>
> >
> https://github.com/apache/nuttx/blob/master/net/udp/udp_sendto_unbuffered.c#L469
> > >>>> > > >
> > >>>> > > > Shouldn't this failure also post the semaphore?
> > >>>> > > > And let the code proceed returning an error in `send()`?
> > >>>> > > >
> > >>>> > > >
> > >>>> > > > On Sun, May 28, 2023 at 5:26 PM Fotis Panagiotopoulos <
> > >>>> > > f.j.panag@gmail.com
> > >>>> > > > >
> > >>>> > > > wrote:
> > >>>> > > >
> > >>>> > > > >
> > >>>> > > > > On Sat, May 27, 2023 at 5:35 PM Xiang Xiao <
> > >>>> > xiaoxiang781216@gmail.com>
> > >>>> > > > > wrote:
> > >>>> > > > >
> > >>>> > > > >> On Sat, May 27, 2023 at 8:19 PM Fotis Panagiotopoulos <
> > >>>> > > > >> f.j.panag@gmail.com>
> > >>>> > > > >> wrote:
> > >>>> > > > >>
> > >>>> > > > >> > Hello,
> > >>>> > > > >> >
> > >>>> > > > >> > I encounter some problems using sendfile().
> > >>>> > > > >> >
> > >>>> > > > >> > I am using sendfile to... send a file to a remote server,
> > >>>> with my
> > >>>> > > own
> > >>>> > > > >> > implementation of an FTP client.
> > >>>> > > > >> > sendfile() indeed starts to transmit chunks of the file,
> > but
> > >>>> as I
> > >>>> > > see
> > >>>> > > > in
> > >>>> > > > >> > Wireshark, I get an ICMP response "Destination
> unreachable
> > >>>> > > > >> (Fragmentation
> > >>>> > > > >> > needed)".
> > >>>> > > > >> > I have verified that the Ethrenet MTU is correctly set to
> > >>>> 1500.
> > >>>> > > > >> >
> > >>>> > > > >> > I tried lowering the MTU a lot (1000 bytes), and the
> > problem
> > >>>> is
> > >>>> > > > solved.
> > >>>> > > > >> > Communication succeeds.
> > >>>> > > > >> >
> > >>>> > > > >> > This raises some questions, and indicates some potential
> > >>>> bugs:
> > >>>> > > > >> >
> > >>>> > > > >> > 1. Why is there a problem with MTU in the first place?
> > >>>> Shouldn't
> > >>>> > MTU
> > >>>> > > > be
> > >>>> > > > >> > negotiated? (Is this functionality available in NuttX?)
> > >>>> > > > >> >
> > >>>> > > > >>
> > >>>> > > > >> MTU isn't negotiated but a physical attribute of your
> > >>>> > > transport(netdev).
> > >>>> > > > >> On
> > >>>> > > > >> the other hand, PMTU could be discovered from ICMP.
> > >>>> > > > >>
> > >>>> > > > >
> > >>>> > > > > I am not very familiar with MTU negotiation, so it seems
> that
> > it
> > >>>> > > doesn't
> > >>>> > > > > happen in the network layer that I thought...
> > >>>> > > > >
> > >>>> > > > >
> > >>>> > > > >>
> > >>>> > > > >>
> > >>>> > > > >> > 2. Why is the ICMP response not handled? It seems that
> > >>>> sendfile()
> > >>>> > > just
> > >>>> > > > >> > ignores it and continues to send chunks, nevertheless.
> > >>>> > > > >> >
> > >>>> > > > >>
> > >>>> > > > >> It is handled by the recent addition here:
> > >>>> > > > >> https://github.com/apachey/nuttx/pull/9254
> > >>>> > > > >> <https://github.com/apache/nuttx/pull/9254>
> > >>>> > > > >> but this feature is disabled by default, you have to enable
> > it
> > >>>> > > > manually..
> > >>>> > > > >>
> > >>>> > > > >
> > >>>> > > > > I will definitely take a look at this. Thank you.
> > >>>> > > > >
> > >>>> > > > >
> > >>>> > > > >>
> > >>>> > > > >>
> > >>>> > > > >
> > >>>> > > > >> > 3. Why sendfile() sends TCP segments without receiving
> any
> > >>>> ACKs
> > >>>> > > back?
> > >>>> > > > >> > AFAIK, depending on the configuration, TCP allows at most
> > two
> > >>>> > > pending
> > >>>> > > > >> > segments on the wire. But I see dozens of them, till
> > sendfile
> > >>>> > > finally
> > >>>> > > > >> > fails.
> > >>>> > > > >> >
> > >>>> > > > >> >
> > >>>> > > > >> Why only two segments? TCP can send packages until the
> slide
> > >>>> window
> > >>>> > is
> > >>>> > > > >> full.
> > >>>> > > > >>
> > >>>> > > > >> Disregard this. I was confused with delayed ACKs. Which is
> a
> > >>>> > > receiver's
> > >>>> > > > > functionality, not a sender's...
> > >>>> > > > >
> > >>>> > > > >
> > >>>> > > > >>
> > >>>> > > > >> > This last point is also verified in my MQTT client.
> > >>>> > > > >> > I have seen NuttX TCP allowing sending lots of TCP
> segments
> > >>>> > without
> > >>>> > > > >> ACKing
> > >>>> > > > >> > the previous data.
> > >>>> > > > >> >
> > >>>> > > > >> > So, is there any insight on the above?
> > >>>> > > > >> > Is my configuration wrong, or is there anything wrong
> with
> > >>>> TCP?
> > >>>> > > > >> >
> > >>>> > > > >> > Thank you.
> > >>>> > > > >> >
> > >>>> > > > >>
> > >>>> > > > >
> > >>>> > > >
> > >>>> > >
> > >>>> >
> > >>>>
> > >>>
> >
>
Re: sendfile() problems - Destination unreachable
Posted by Xiang Xiao <xi...@gmail.com>.
On Sun, Jun 4, 2023 at 10:23 PM Fotis Panagiotopoulos <f....@gmail.com>
wrote:
> I am trying this new PMTUD functionality, but it seems it doesn't work.
>
> I have configured:
> CONFIG_NET_ICMP_PMTU_ENTRIES = 10
> CONFIG_NET_ICMP_PMTU_TIMEOUT = 10
>
> Again I see lots of segments being sent (all with size 1400), and all of
> them are responded with the same ICMP reply.
> A couple of retransmissions are attempted, and then the connection is
> reset.
>
The option can discover the minimal MTU from the source to the destination.
You need to use PMTU to split your data into small packets(<= PMTU) by
yourself to improve the efficiency.
>
> I try again the same procedure (now that the system had the chance to
> discover the maximum PMTU), but it still fails.
> Again all segments have a size of 1400, instead of less.
>
If you don't split your package to fit MTU by yourself, you have
to enable NET_IPFRAG. But I would suggest that.you switch UDP to TCP
because the protocol you implement on top of UDP is likely very inefficient.
>
> I am using sendfile() in case this matters.
>
> On Tue, May 30, 2023 at 12:28 AM Fotis Panagiotopoulos <
> f.j.panag@gmail.com>
> wrote:
>
> > I can see that there are two different problems with MTU.
> > They are completely independent from each other, so let's start with the
> > simple case first.
> >
> > I am testing on an STM32F427, using Ethernet.
> >
> > As previously noted, the following code will cause the running task to
> > hang.
> >
> > netlib_set_mtu(CONFIG_NETIF_DEV_NAME, 1500);
> >
> > int sd = socket(AF_INET, SOCK_DGRAM, 0);
> >
> > struct sockaddr_in server;
> > server.sin_family = AF_INET;
> > server.sin_port = 1000;
> > server.sin_addr.s_addr = inet_addr("192.168.1.235");
> >
> > uint8_t * data = malloc(2048);
> > memset(data, 0xAA, 2048);
> >
> > sendto(sd, data, 2048, 0, (struct sockaddr*)&server, sizeof(server));
> >
> > close(sd);
> >
> > As you can see, the MTU is set to 1500, and then I try to send a UDP
> > datagram with a larger size (2048).
> > Indeed `devif_send()` fails, and the aforementioned semaphore is never
> > posted.
> >
> > (This is without buffering in UTP, in case this is important).
> >
> > This draft PR, provides a solution to the issue.
> > https://github.com/apache/nuttx/pull/9423
> >
> > If this is correct, I will also check buffered UDP, and other uses of
> > devif_send().
> >
> > Alternatively, devif_send() may be changed to actually return an error
> > code (instead of returning void), so improved error handling can take
> place.
> >
> >
> > On Mon, May 29, 2023 at 1:41 PM Fotis Panagiotopoulos <
> f.j.panag@gmail.com>
> > wrote:
> >
> >> The failure scenario is a bit more complicated...
> >>
> >> Give me some time and I will provide a correct and reproducible example,
> >> with a clear explanation.
> >>
> >> On Mon, May 29, 2023, 13:27 Fotis Panagiotopoulos <f....@gmail.com>
> >> wrote:
> >>
> >>> > sendfile should return an error in this case, but senfile should only
> >>> be
> >>> > used with TCP, not UDP, since sendfile doesn't have any logic to ack
> or
> >>> > retry..
> >>>
> >>> Sorry if this wasn't clear. This last test was with plain old
> `send()`...
> >>>
> >>> I opened a UDP socket, and used `send()` to transmit a buffer larger
> >>> than the MTU.
> >>> Instead of getting an error, the application hangs indefinitely.
> >>> `devif_send()` is called periodically, but of course it always fails.
> >>>
> >>>
> >>>
> >>> On Mon, May 29, 2023 at 1:13 PM Xiang Xiao <xi...@gmail.com>
> >>> wrote:
> >>>
> >>>> On Mon, May 29, 2023 at 5:02 PM Fotis Panagiotopoulos <
> >>>> f.j.panag@gmail.com>
> >>>> wrote:
> >>>>
> >>>> > > You need to enable IP fragmentation in this case, which is also
> >>>> added
> >>>> > > recently and disabled by default:
> >>>> > > https://github.com/apache/nuttx/pull/8059
> >>>> > <https://github.com/apache/nuttx/pull/8059>
> >>>> > > Otherwise, any packet bigger than MTU will be dropped silently.
> >>>> >
> >>>> > Yes, this is the expected behavior.
> >>>> > But, instead of dropping the packet, the system hangs because the
> >>>> semaphore
> >>>> > is never posted.
> >>>> > It just tries endlessly to call devif_send() which always fails.
> >>>> >
> >>>> >
> >>>> sendfile should return an error in this case, but senfile should only
> be
> >>>> used with TCP, not UDP, since sendfile doesn't have any logic to ack
> or
> >>>> retry..
> >>>>
> >>>>
> >>>> >
> >>>> >
> >>>> > On Mon, May 29, 2023 at 11:42 AM Xiang Xiao <
> >>>> xiaoxiang781216@gmail.com>
> >>>> > wrote:
> >>>> >
> >>>> > > On Sun, May 28, 2023 at 11:55 PM Fotis Panagiotopoulos <
> >>>> > > f.j.panag@gmail.com>
> >>>> > > wrote:
> >>>> > >
> >>>> > > > While experimenting with MTU, and checking the stability of my
> >>>> system,
> >>>> > I
> >>>> > > > noticed the following.
> >>>> > > >
> >>>> > > > I try to send a UDP datagram that is larger than the configured
> >>>> MTU.
> >>>> > > > In this case, the offending thread seems to hang indefinitely
> (or
> >>>> at
> >>>> > > least
> >>>> > > > waiting for a very long timeout?)
> >>>> > > >
> >>>> > >
> >>>> > > You need to enable IP fragmentation in this case, which is also
> >>>> added
> >>>> > > recently and disabled by default:
> >>>> > > https://github.com/apache/nuttx/pull/8059
> >>>> > > Otherwise, any packet bigger than MTU will be dropped silently.
> >>>> > >
> >>>> > >
> >>>> > > > The problem seems to be this line:
> >>>> > > >
> >>>> > > >
> >>>> > >
> >>>> >
> >>>>
> https://github.com/apache/nuttx/blob/master/net/udp/udp_sendto_unbuffered.c#L197
> >>>> > > > `devif_send()` fails because the datagram is too large, but
> >>>> > > > `pstate->st_sem` is never posted (the code returns immediately).
> >>>> > > >
> >>>> > > > This leaves the sending task to be blocked here:
> >>>> > > >
> >>>> > > >
> >>>> > >
> >>>> >
> >>>>
> https://github.com/apache/nuttx/blob/master/net/udp/udp_sendto_unbuffered.c#L469
> >>>> > > >
> >>>> > > > Shouldn't this failure also post the semaphore?
> >>>> > > > And let the code proceed returning an error in `send()`?
> >>>> > > >
> >>>> > > >
> >>>> > > > On Sun, May 28, 2023 at 5:26 PM Fotis Panagiotopoulos <
> >>>> > > f.j.panag@gmail.com
> >>>> > > > >
> >>>> > > > wrote:
> >>>> > > >
> >>>> > > > >
> >>>> > > > > On Sat, May 27, 2023 at 5:35 PM Xiang Xiao <
> >>>> > xiaoxiang781216@gmail.com>
> >>>> > > > > wrote:
> >>>> > > > >
> >>>> > > > >> On Sat, May 27, 2023 at 8:19 PM Fotis Panagiotopoulos <
> >>>> > > > >> f.j.panag@gmail.com>
> >>>> > > > >> wrote:
> >>>> > > > >>
> >>>> > > > >> > Hello,
> >>>> > > > >> >
> >>>> > > > >> > I encounter some problems using sendfile().
> >>>> > > > >> >
> >>>> > > > >> > I am using sendfile to... send a file to a remote server,
> >>>> with my
> >>>> > > own
> >>>> > > > >> > implementation of an FTP client.
> >>>> > > > >> > sendfile() indeed starts to transmit chunks of the file,
> but
> >>>> as I
> >>>> > > see
> >>>> > > > in
> >>>> > > > >> > Wireshark, I get an ICMP response "Destination unreachable
> >>>> > > > >> (Fragmentation
> >>>> > > > >> > needed)".
> >>>> > > > >> > I have verified that the Ethrenet MTU is correctly set to
> >>>> 1500.
> >>>> > > > >> >
> >>>> > > > >> > I tried lowering the MTU a lot (1000 bytes), and the
> problem
> >>>> is
> >>>> > > > solved.
> >>>> > > > >> > Communication succeeds.
> >>>> > > > >> >
> >>>> > > > >> > This raises some questions, and indicates some potential
> >>>> bugs:
> >>>> > > > >> >
> >>>> > > > >> > 1. Why is there a problem with MTU in the first place?
> >>>> Shouldn't
> >>>> > MTU
> >>>> > > > be
> >>>> > > > >> > negotiated? (Is this functionality available in NuttX?)
> >>>> > > > >> >
> >>>> > > > >>
> >>>> > > > >> MTU isn't negotiated but a physical attribute of your
> >>>> > > transport(netdev).
> >>>> > > > >> On
> >>>> > > > >> the other hand, PMTU could be discovered from ICMP.
> >>>> > > > >>
> >>>> > > > >
> >>>> > > > > I am not very familiar with MTU negotiation, so it seems that
> it
> >>>> > > doesn't
> >>>> > > > > happen in the network layer that I thought...
> >>>> > > > >
> >>>> > > > >
> >>>> > > > >>
> >>>> > > > >>
> >>>> > > > >> > 2. Why is the ICMP response not handled? It seems that
> >>>> sendfile()
> >>>> > > just
> >>>> > > > >> > ignores it and continues to send chunks, nevertheless.
> >>>> > > > >> >
> >>>> > > > >>
> >>>> > > > >> It is handled by the recent addition here:
> >>>> > > > >> https://github.com/apachey/nuttx/pull/9254
> >>>> > > > >> <https://github.com/apache/nuttx/pull/9254>
> >>>> > > > >> but this feature is disabled by default, you have to enable
> it
> >>>> > > > manually..
> >>>> > > > >>
> >>>> > > > >
> >>>> > > > > I will definitely take a look at this. Thank you.
> >>>> > > > >
> >>>> > > > >
> >>>> > > > >>
> >>>> > > > >>
> >>>> > > > >
> >>>> > > > >> > 3. Why sendfile() sends TCP segments without receiving any
> >>>> ACKs
> >>>> > > back?
> >>>> > > > >> > AFAIK, depending on the configuration, TCP allows at most
> two
> >>>> > > pending
> >>>> > > > >> > segments on the wire. But I see dozens of them, till
> sendfile
> >>>> > > finally
> >>>> > > > >> > fails.
> >>>> > > > >> >
> >>>> > > > >> >
> >>>> > > > >> Why only two segments? TCP can send packages until the slide
> >>>> window
> >>>> > is
> >>>> > > > >> full.
> >>>> > > > >>
> >>>> > > > >> Disregard this. I was confused with delayed ACKs. Which is a
> >>>> > > receiver's
> >>>> > > > > functionality, not a sender's...
> >>>> > > > >
> >>>> > > > >
> >>>> > > > >>
> >>>> > > > >> > This last point is also verified in my MQTT client.
> >>>> > > > >> > I have seen NuttX TCP allowing sending lots of TCP segments
> >>>> > without
> >>>> > > > >> ACKing
> >>>> > > > >> > the previous data.
> >>>> > > > >> >
> >>>> > > > >> > So, is there any insight on the above?
> >>>> > > > >> > Is my configuration wrong, or is there anything wrong with
> >>>> TCP?
> >>>> > > > >> >
> >>>> > > > >> > Thank you.
> >>>> > > > >> >
> >>>> > > > >>
> >>>> > > > >
> >>>> > > >
> >>>> > >
> >>>> >
> >>>>
> >>>
>