You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@nuttx.apache.org by GitBox <gi...@apache.org> on 2021/10/12 04:32:06 UTC
[GitHub] [incubator-nuttx] a-lunev opened a new pull request #4659: net/tcp(unbuffered): retransmit only one the earliest not acknowledged segment
a-lunev opened a new pull request #4659:
URL: https://github.com/apache/incubator-nuttx/pull/4659
## Summary
As it turned out, the existing TCP unbuffered send implementation does a full rewind from the most recent sent segment back to the earliest one, thus many TCP segments are re-sent every time when TCP retrasmission timeout occurs.
According to RFC 6298 (5.4) only one the earliest not acknowledged segment should be retransmitted instead.
This PR implements TCP retrasmission according to RFC 6298 (5.4) if CONFIG_NET_TCP_SPLIT is disabled (this is by default).
If CONFIG_NET_TCP_SPLIT is enabled, TCP retrasmission works as before (full rewind). So far it is not clear how to adapt CONFIG_NET_TCP_SPLIT algorithm to conform to RFC 6298 (5.4).
## Impact
TCP
## Testing
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] yamt commented on a change in pull request #4659: net/tcp(unbuffered): retransmit only one the earliest not acknowledged segment
Posted by GitBox <gi...@apache.org>.
yamt commented on a change in pull request #4659:
URL: https://github.com/apache/incubator-nuttx/pull/4659#discussion_r727654868
##########
File path: net/tcp/tcp_send_unbuffered.c
##########
@@ -243,18 +243,54 @@ static uint16_t tcpsend_eventhandler(FAR struct net_driver_s *dev,
else if ((flags & TCP_REXMIT) != 0)
{
+#if defined(CONFIG_NET_TCP_SPLIT)
/* Yes.. in this case, reset the number of bytes that have been sent
* to the number of bytes that have been ACKed.
*/
pstate->snd_sent = pstate->snd_acked;
Review comment:
ok. it seems i was misreading the description. thank you for explanation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] xiaoxiang781216 commented on pull request #4659: net/tcp(unbuffered): retransmit only one the earliest not acknowledged segment
Posted by GitBox <gi...@apache.org>.
xiaoxiang781216 commented on pull request #4659:
URL: https://github.com/apache/incubator-nuttx/pull/4659#issuecomment-965617057
So, let's merge this PR first.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] yamt commented on a change in pull request #4659: net/tcp(unbuffered): retransmit only one the earliest not acknowledged segment
Posted by GitBox <gi...@apache.org>.
yamt commented on a change in pull request #4659:
URL: https://github.com/apache/incubator-nuttx/pull/4659#discussion_r727610211
##########
File path: net/tcp/tcp_send_unbuffered.c
##########
@@ -243,18 +243,54 @@ static uint16_t tcpsend_eventhandler(FAR struct net_driver_s *dev,
else if ((flags & TCP_REXMIT) != 0)
{
+#if defined(CONFIG_NET_TCP_SPLIT)
/* Yes.. in this case, reset the number of bytes that have been sent
* to the number of bytes that have been ACKed.
*/
pstate->snd_sent = pstate->snd_acked;
Review comment:
doesn't this mean not to resend bytes already acked?
if this is not working for some reasons, it's better to fix it instead of introducing a separate code path.
##########
File path: net/tcp/tcp.h
##########
@@ -172,6 +172,9 @@ struct tcp_conn_s
uint8_t rcvseq[4]; /* The sequence number that we expect to
* receive next */
uint8_t sndseq[4]; /* The sequence number that was last sent by us */
+#if !defined(CONFIG_NET_TCP_SPLIT) && !defined(CONFIG_NET_TCP_WRITE_BUFFERS)
+ uint32_t rexmit_seq; /* The sequence number to be retrasmitted */
Review comment:
i feel a bit uneasy about yet another communication channel to tcp_send.c.
isn't it a responsibility of tcp_send_xxx and other event handlers to set sndseq appropriately?
##########
File path: net/tcp/tcp_send_unbuffered.c
##########
@@ -243,18 +243,54 @@ static uint16_t tcpsend_eventhandler(FAR struct net_driver_s *dev,
else if ((flags & TCP_REXMIT) != 0)
{
+#if defined(CONFIG_NET_TCP_SPLIT)
/* Yes.. in this case, reset the number of bytes that have been sent
* to the number of bytes that have been ACKed.
*/
pstate->snd_sent = pstate->snd_acked;
Review comment:
i meant, doesn't snd_acked here indicate "one the earliest not acknowledged segment"?
i'm trying to understand why it actually "does a full rewind".
##########
File path: net/tcp/tcp_send_unbuffered.c
##########
@@ -243,18 +243,54 @@ static uint16_t tcpsend_eventhandler(FAR struct net_driver_s *dev,
else if ((flags & TCP_REXMIT) != 0)
{
+#if defined(CONFIG_NET_TCP_SPLIT)
/* Yes.. in this case, reset the number of bytes that have been sent
* to the number of bytes that have been ACKed.
*/
pstate->snd_sent = pstate->snd_acked;
Review comment:
ok. it seems i was misreading the description. thank you for explanation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] yamt commented on pull request #4659: net/tcp(unbuffered): retransmit only one the earliest not acknowledged segment
Posted by GitBox <gi...@apache.org>.
yamt commented on pull request #4659:
URL: https://github.com/apache/incubator-nuttx/pull/4659#issuecomment-941801270
btw, it has been on my todo list to remove CONFIG_NET_TCP_SPLIT.
if it helps your work, please let me know. it isn't urgent and i don't want to interfere you by submitting conflicting patches.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] a-lunev edited a comment on pull request #4659: net/tcp(unbuffered): retransmit only one the earliest not acknowledged segment
Posted by GitBox <gi...@apache.org>.
a-lunev edited a comment on pull request #4659:
URL: https://github.com/apache/incubator-nuttx/pull/4659#issuecomment-941886030
> * i suspect this is something should be fixed with cwnd, which nuttx doesn't have right now.
>
> * doesn't the "buffered" version behave in a similar way? it basically moves segments on unacked_q back to write_q.
>
>
> how do you think?
I think it would be reasonable to implement congestion window control as an option. Also I see that Fast Retransmit algorithm is not supported in unbuffered tcp mode of NuttX that is possibly even more important.
Concerning "buffered" version I can not say at the moment as I have not yet analyzed the source code in that part.
So far I'm using "unbuffered" mode as in my case it surprisingly provides twice as high throughput as "buffered" one. Though, I expected that "buffered" mode should have better performance compared to "unbuffered" one.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] a-lunev commented on a change in pull request #4659: net/tcp(unbuffered): retransmit only one the earliest not acknowledged segment
Posted by GitBox <gi...@apache.org>.
a-lunev commented on a change in pull request #4659:
URL: https://github.com/apache/incubator-nuttx/pull/4659#discussion_r727656529
##########
File path: net/tcp/tcp_send_unbuffered.c
##########
@@ -243,18 +243,54 @@ static uint16_t tcpsend_eventhandler(FAR struct net_driver_s *dev,
else if ((flags & TCP_REXMIT) != 0)
{
+#if defined(CONFIG_NET_TCP_SPLIT)
/* Yes.. in this case, reset the number of bytes that have been sent
* to the number of bytes that have been ACKed.
*/
pstate->snd_sent = pstate->snd_acked;
Review comment:
Just now I have updated the description:
from "... to the earliest one"
to "... to the earliest not acknowledged one".
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] a-lunev commented on pull request #4659: net/tcp(unbuffered): retransmit only one the earliest not acknowledged segment
Posted by GitBox <gi...@apache.org>.
a-lunev commented on pull request #4659:
URL: https://github.com/apache/incubator-nuttx/pull/4659#issuecomment-941830913
> > > btw, it has been on my todo list to remove CONFIG_NET_TCP_SPLIT. if it helps your work, please let me know. it isn't urgent and i don't want to interfere you by submitting conflicting patches.
> >
> >
> > Oh, really? I do not personally need CONFIG_NET_TCP_SPLIT algorithm. What about others? Is there any previous discussion about its deletion? I'm not sure, if there is a reason to preserve it? If I understand correctly, CONFIG_NET_TCP_SPLIT was developed for MCUs with tiny memories that do not have enough memory for a sufficient TCP retransmission queue depth.
>
> it's just my personal todo. not community's. while i can understand the motivation, it doesn't work well even with a traditional tcp stack.
Can we poll NuttX developers about CONFIG_NET_TCP_SPLIT deletion?
As I said, I personally do not need it. However, I'm afraid of if someone still needs CONFIG_NET_TCP_SPLIT?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] a-lunev commented on a change in pull request #4659: net/tcp(unbuffered): retransmit only one the earliest not acknowledged segment
Posted by GitBox <gi...@apache.org>.
a-lunev commented on a change in pull request #4659:
URL: https://github.com/apache/incubator-nuttx/pull/4659#discussion_r727651822
##########
File path: net/tcp/tcp_send_unbuffered.c
##########
@@ -243,18 +243,54 @@ static uint16_t tcpsend_eventhandler(FAR struct net_driver_s *dev,
else if ((flags & TCP_REXMIT) != 0)
{
+#if defined(CONFIG_NET_TCP_SPLIT)
/* Yes.. in this case, reset the number of bytes that have been sent
* to the number of bytes that have been ACKed.
*/
pstate->snd_sent = pstate->snd_acked;
Review comment:
I understand that part of code as follows:
Here is an example:
```
pstate->snd_sent = 0, pstate->snd_acked = 0: SND.seq = 100 (it's ISN), len = 10 -> receiver successfully gets the segment by ACKing with RCV.ack = 110.
pstate->snd_sent = 10, pstate->snd_acked = 10: SND.seq = 110, len = 10 -> the segment is lost somewhere in the network
pstate->snd_sent = 20, pstate->snd_acked = 10: SND.seq = 120, len = 10 -> receiver gets the segment, however still ACKing with RCV.ack = 110.
pstate->snd_sent = 30, pstate->snd_acked = 10: SND.seq = 130, len = 10 -> receiver gets the segment, however still ACKing with RCV.ack = 110.
pstate->snd_sent = 40, pstate->snd_acked = 10: SND.seq = 140, len = 10 -> receiver gets the segment, however still ACKing with RCV.ack = 110.
```
Due to timeout NuttX TCP sender sets pstate->snd_sent variable back to 10 (and SND.seq is set back to 110) as follows:
```
pstate->snd_sent = 10, pstate->snd_acked = 10: SND.seq = 110, len = 10 -> receiver successfully gets the segment by ACKing with RCV.ack = 120.
pstate->snd_sent = 20, pstate->snd_acked = 20: SND.seq = 120, len = 10 -> receiver successfully gets the segment by ACKing with RCV.ack = 130.
pstate->snd_sent = 30, pstate->snd_acked = 30: SND.seq = 130, len = 10 -> receiver successfully gets the segment by ACKing with RCV.ack = 140.
pstate->snd_sent = 40, pstate->snd_acked = 40: SND.seq = 140, len = 10 -> receiver successfully gets the segment by ACKing with RCV.ack = 150.
```
Thus in this example it re-sends all 4 segments instead of only one segment with SND.seq = 110.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] yamt commented on a change in pull request #4659: net/tcp(unbuffered): retransmit only one the earliest not acknowledged segment
Posted by GitBox <gi...@apache.org>.
yamt commented on a change in pull request #4659:
URL: https://github.com/apache/incubator-nuttx/pull/4659#discussion_r727633309
##########
File path: net/tcp/tcp_send_unbuffered.c
##########
@@ -243,18 +243,54 @@ static uint16_t tcpsend_eventhandler(FAR struct net_driver_s *dev,
else if ((flags & TCP_REXMIT) != 0)
{
+#if defined(CONFIG_NET_TCP_SPLIT)
/* Yes.. in this case, reset the number of bytes that have been sent
* to the number of bytes that have been ACKed.
*/
pstate->snd_sent = pstate->snd_acked;
Review comment:
i meant, doesn't snd_acked here indicate "one the earliest not acknowledged segment"?
i'm trying to understand why it actually "does a full rewind".
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] a-lunev commented on pull request #4659: net/tcp(unbuffered): retransmit only one the earliest not acknowledged segment
Posted by GitBox <gi...@apache.org>.
a-lunev commented on pull request #4659:
URL: https://github.com/apache/incubator-nuttx/pull/4659#issuecomment-965026412
> separate? maybe. but they can be implemented with the common machinary.
Yes, the congestion window control code is of course expected to be tightly integrated into the existing NuttX code. I mean this is an extra algorithm that it would be reasonable to enable via menuconfig as an option. And this new potential code (congestion window control) could be activated by means of conditional compilation (#ifdef #endif) like e.g. CONFIG_NET_TCP_DELAYED_ACK, CONFIG_NET_TCP_WINDOW_SCALE and CONFIG_NET_TCP_FAST_RETRANSMIT_WATERMARK options, that just extend the basic NuttX TCP/IP functionality (optionally).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] yamt commented on pull request #4659: net/tcp(unbuffered): retransmit only one the earliest not acknowledged segment
Posted by GitBox <gi...@apache.org>.
yamt commented on pull request #4659:
URL: https://github.com/apache/incubator-nuttx/pull/4659#issuecomment-964639312
i still think it's better to do it with cwnd-like structure. let me think a bit.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] yamt commented on pull request #4659: net/tcp(unbuffered): retransmit only one the earliest not acknowledged segment
Posted by GitBox <gi...@apache.org>.
yamt commented on pull request #4659:
URL: https://github.com/apache/incubator-nuttx/pull/4659#issuecomment-964673541
> > i still think it's better to do it with cwnd-like structure. let me think a bit.
>
> By "cwnd" do you mean congestion window control algorithm or something else?
yes, congestion window.
clamping the window to one segment for retransmit is what this patch does, right?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] a-lunev commented on pull request #4659: net/tcp(unbuffered): retransmit only one the earliest not acknowledged segment
Posted by GitBox <gi...@apache.org>.
a-lunev commented on pull request #4659:
URL: https://github.com/apache/incubator-nuttx/pull/4659#issuecomment-964671301
> i still think it's better to do it with cwnd-like structure. let me think a bit.
By "cwnd" do you mean congestion window control algorithm or something else?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] a-lunev commented on a change in pull request #4659: net/tcp(unbuffered): retransmit only one the earliest not acknowledged segment
Posted by GitBox <gi...@apache.org>.
a-lunev commented on a change in pull request #4659:
URL: https://github.com/apache/incubator-nuttx/pull/4659#discussion_r727627443
##########
File path: net/tcp/tcp_send_unbuffered.c
##########
@@ -243,18 +243,54 @@ static uint16_t tcpsend_eventhandler(FAR struct net_driver_s *dev,
else if ((flags & TCP_REXMIT) != 0)
{
+#if defined(CONFIG_NET_TCP_SPLIT)
/* Yes.. in this case, reset the number of bytes that have been sent
* to the number of bytes that have been ACKed.
*/
pstate->snd_sent = pstate->snd_acked;
Review comment:
Yes, it does. I wrote about this in the PR summary. It contradicts to RFC 6298 (5.4) and other TCP related RFCs. Possibly it was done intentionally as a workaround (as I understand, CONFIG_NET_TCP_SPLIT algorithm is some sort of a trick as it is described in the source code comments).
If CONFIG_NET_TCP_SPLIT was planned to be removed, the removing would simplify this part of the code.
##########
File path: net/tcp/tcp_send_unbuffered.c
##########
@@ -243,18 +243,54 @@ static uint16_t tcpsend_eventhandler(FAR struct net_driver_s *dev,
else if ((flags & TCP_REXMIT) != 0)
{
+#if defined(CONFIG_NET_TCP_SPLIT)
/* Yes.. in this case, reset the number of bytes that have been sent
* to the number of bytes that have been ACKed.
*/
pstate->snd_sent = pstate->snd_acked;
Review comment:
I understand that part of code as follows:
Here is an example:
```
pstate->snd_sent = 0, pstate->snd_acked = 0: SND.seq = 100 (it's ISN), len = 10 -> receiver successfully gets the segment by ACKing with RCV.ack = 110.
pstate->snd_sent = 10, pstate->snd_acked = 10: SND.seq = 110, len = 10 -> the segment is lost somewhere in the network
pstate->snd_sent = 20, pstate->snd_acked = 10: SND.seq = 120, len = 10 -> receiver gets the segment, however still ACKing with RCV.ack = 110.
pstate->snd_sent = 30, pstate->snd_acked = 10: SND.seq = 130, len = 10 -> receiver gets the segment, however still ACKing with RCV.ack = 110.
pstate->snd_sent = 40, pstate->snd_acked = 10: SND.seq = 140, len = 10 -> receiver gets the segment, however still ACKing with RCV.ack = 110.
```
Due to timeout NuttX TCP sender sets pstate->snd_sent variable back to 10 as follows:
```
pstate->snd_sent = 10, pstate->snd_acked = 10: SND.seq = 110, len = 10 -> receiver successfully gets the segment by ACKing with RCV.ack = 120.
pstate->snd_sent = 20, pstate->snd_acked = 20: SND.seq = 120, len = 10 -> receiver successfully gets the segment by ACKing with RCV.ack = 130.
pstate->snd_sent = 30, pstate->snd_acked = 30: SND.seq = 130, len = 10 -> receiver successfully gets the segment by ACKing with RCV.ack = 140.
pstate->snd_sent = 40, pstate->snd_acked = 40: SND.seq = 140, len = 10 -> receiver successfully gets the segment by ACKing with RCV.ack = 150.
```
Thus in this example it re-sends all 4 segments instead of only one segment with SND.seq = 110.
##########
File path: net/tcp/tcp_send_unbuffered.c
##########
@@ -243,18 +243,54 @@ static uint16_t tcpsend_eventhandler(FAR struct net_driver_s *dev,
else if ((flags & TCP_REXMIT) != 0)
{
+#if defined(CONFIG_NET_TCP_SPLIT)
/* Yes.. in this case, reset the number of bytes that have been sent
* to the number of bytes that have been ACKed.
*/
pstate->snd_sent = pstate->snd_acked;
Review comment:
I understand that part of code as follows:
Here is an example:
```
pstate->snd_sent = 0, pstate->snd_acked = 0: SND.seq = 100 (it's ISN), len = 10 -> receiver successfully gets the segment by ACKing with RCV.ack = 110.
pstate->snd_sent = 10, pstate->snd_acked = 10: SND.seq = 110, len = 10 -> the segment is lost somewhere in the network
pstate->snd_sent = 20, pstate->snd_acked = 10: SND.seq = 120, len = 10 -> receiver gets the segment, however still ACKing with RCV.ack = 110.
pstate->snd_sent = 30, pstate->snd_acked = 10: SND.seq = 130, len = 10 -> receiver gets the segment, however still ACKing with RCV.ack = 110.
pstate->snd_sent = 40, pstate->snd_acked = 10: SND.seq = 140, len = 10 -> receiver gets the segment, however still ACKing with RCV.ack = 110.
```
Due to timeout NuttX TCP sender sets pstate->snd_sent variable back to 10 (and SND.seq is set back to 110) as follows:
```
pstate->snd_sent = 10, pstate->snd_acked = 10: SND.seq = 110, len = 10 -> receiver successfully gets the segment by ACKing with RCV.ack = 120.
pstate->snd_sent = 20, pstate->snd_acked = 20: SND.seq = 120, len = 10 -> receiver successfully gets the segment by ACKing with RCV.ack = 130.
pstate->snd_sent = 30, pstate->snd_acked = 30: SND.seq = 130, len = 10 -> receiver successfully gets the segment by ACKing with RCV.ack = 140.
pstate->snd_sent = 40, pstate->snd_acked = 40: SND.seq = 140, len = 10 -> receiver successfully gets the segment by ACKing with RCV.ack = 150.
```
Thus in this example it re-sends all 4 segments instead of only one segment with SND.seq = 110.
##########
File path: net/tcp/tcp_send_unbuffered.c
##########
@@ -243,18 +243,54 @@ static uint16_t tcpsend_eventhandler(FAR struct net_driver_s *dev,
else if ((flags & TCP_REXMIT) != 0)
{
+#if defined(CONFIG_NET_TCP_SPLIT)
/* Yes.. in this case, reset the number of bytes that have been sent
* to the number of bytes that have been ACKed.
*/
pstate->snd_sent = pstate->snd_acked;
Review comment:
Just now I have updated the description:
from "... to the earliest one"
to "... to the earliest not acknowledged one".
##########
File path: net/tcp/tcp.h
##########
@@ -172,6 +172,9 @@ struct tcp_conn_s
uint8_t rcvseq[4]; /* The sequence number that we expect to
* receive next */
uint8_t sndseq[4]; /* The sequence number that was last sent by us */
+#if !defined(CONFIG_NET_TCP_SPLIT) && !defined(CONFIG_NET_TCP_WRITE_BUFFERS)
+ uint32_t rexmit_seq; /* The sequence number to be retrasmitted */
Review comment:
Previously I read your description in #4293 ("in some cases, we send tcp segments with sequence numbers back and forth.
especially when sending an empty segment. (eg. pure ack, window updates)").
Thus I have added conn->rexmit_seq field that is used only for retrasmitting a single TCP segment w/o a need to modify conn->sndseq each time and recovering it back.
Thus I consider using conn->rexmit_seq as more reliable and stable way rather than using conn->sndseq for retransmissions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] a-lunev edited a comment on pull request #4659: net/tcp(unbuffered): retransmit only one the earliest not acknowledged segment
Posted by GitBox <gi...@apache.org>.
a-lunev edited a comment on pull request #4659:
URL: https://github.com/apache/incubator-nuttx/pull/4659#issuecomment-964625263
I force-pushed to restart ci because "Build Documentation / build-html" previously failed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] yamt commented on pull request #4659: net/tcp(unbuffered): retransmit only one the earliest not acknowledged segment
Posted by GitBox <gi...@apache.org>.
yamt commented on pull request #4659:
URL: https://github.com/apache/incubator-nuttx/pull/4659#issuecomment-964816647
> > > > i still think it's better to do it with cwnd-like structure. let me think a bit.
> > >
> > >
> > > By "cwnd" do you mean congestion window control algorithm or something else?
> >
> >
> > yes, congestion window. clamping the window to one segment for retransmit is what this patch does, right?
>
> You are saying about RFC 5681 (slow start of sending data etc.). My patch is not related to the congestion window control. My patch instead just fixes a basic requirement (RFC 6298 (5.4)) how a TCP segment should be retransmitted e.g. in case of ACK timeout. If it is detected that a segment is lost, only that particular segment should be retransmitted. However, currently NuttX TCP/IP stack (at least "unbuffered send" mode) retransmits all segments between NACK and SEQ provoking network traffic collapse. The congestion window control is a separate advanced algorithm that I think could be implemented in NuttX TCP/IP stack as an option.
separate? maybe. but they can be implemented with the common machinary.
i will think a bit more, but because of holidays etc, i suspect it will take at least a few weeks.
i'm going to approve this for now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] a-lunev commented on pull request #4659: net/tcp(unbuffered): retransmit only one the earliest not acknowledged segment
Posted by GitBox <gi...@apache.org>.
a-lunev commented on pull request #4659:
URL: https://github.com/apache/incubator-nuttx/pull/4659#issuecomment-941886030
> * i suspect this is something should be fixed with cwnd, which nuttx doesn't have right now.
>
> * doesn't the "buffered" version behave in a similar way? it basically moves segments on unacked_q back to write_q.
>
>
> how do you think?
I think it would be reasonable to implement congestion window control as an option. Also I see that Fast Retransmit algorithm is not supported in unbuffered tcp mode of NuttX that is possibly even more important.
Concerning "buffered" version I can not say at the moment as I have not yet analyzed the source code in that part.
So far I'm using "unbuffered" mode as in my case it surprisingly provides twice as higher throughput compared to "buffered" mode. Though, I expected that "buffered" mode should have better performance compared to "unbuffered" one.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] a-lunev commented on a change in pull request #4659: net/tcp(unbuffered): retransmit only one the earliest not acknowledged segment
Posted by GitBox <gi...@apache.org>.
a-lunev commented on a change in pull request #4659:
URL: https://github.com/apache/incubator-nuttx/pull/4659#discussion_r727651822
##########
File path: net/tcp/tcp_send_unbuffered.c
##########
@@ -243,18 +243,54 @@ static uint16_t tcpsend_eventhandler(FAR struct net_driver_s *dev,
else if ((flags & TCP_REXMIT) != 0)
{
+#if defined(CONFIG_NET_TCP_SPLIT)
/* Yes.. in this case, reset the number of bytes that have been sent
* to the number of bytes that have been ACKed.
*/
pstate->snd_sent = pstate->snd_acked;
Review comment:
I understand that part of code as follows:
Here is an example:
```
pstate->snd_sent = 0, pstate->snd_acked = 0: SND.seq = 100 (it's ISN), len = 10 -> receiver successfully gets the segment by ACKing with RCV.ack = 110.
pstate->snd_sent = 10, pstate->snd_acked = 10: SND.seq = 110, len = 10 -> the segment is lost somewhere in the network
pstate->snd_sent = 20, pstate->snd_acked = 10: SND.seq = 120, len = 10 -> receiver gets the segment, however still ACKing with RCV.ack = 110.
pstate->snd_sent = 30, pstate->snd_acked = 10: SND.seq = 130, len = 10 -> receiver gets the segment, however still ACKing with RCV.ack = 110.
pstate->snd_sent = 40, pstate->snd_acked = 10: SND.seq = 140, len = 10 -> receiver gets the segment, however still ACKing with RCV.ack = 110.
```
Due to timeout NuttX TCP sender sets pstate->snd_sent variable back to 10 as follows:
```
pstate->snd_sent = 10, pstate->snd_acked = 10: SND.seq = 110, len = 10 -> receiver successfully gets the segment by ACKing with RCV.ack = 120.
pstate->snd_sent = 20, pstate->snd_acked = 20: SND.seq = 120, len = 10 -> receiver successfully gets the segment by ACKing with RCV.ack = 130.
pstate->snd_sent = 30, pstate->snd_acked = 30: SND.seq = 130, len = 10 -> receiver successfully gets the segment by ACKing with RCV.ack = 140.
pstate->snd_sent = 40, pstate->snd_acked = 40: SND.seq = 140, len = 10 -> receiver successfully gets the segment by ACKing with RCV.ack = 150.
```
Thus in this example it re-sends all 4 segments instead of only one segment with SND.seq = 110.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] a-lunev commented on pull request #4659: net/tcp(unbuffered): retransmit only one the earliest not acknowledged segment
Posted by GitBox <gi...@apache.org>.
a-lunev commented on pull request #4659:
URL: https://github.com/apache/incubator-nuttx/pull/4659#issuecomment-941826145
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] yamt commented on pull request #4659: net/tcp(unbuffered): retransmit only one the earliest not acknowledged segment
Posted by GitBox <gi...@apache.org>.
yamt commented on pull request #4659:
URL: https://github.com/apache/incubator-nuttx/pull/4659#issuecomment-941868525
* i suspect this is something should be fixed with cwnd, which nuttx doesn't have right now.
* doesn't the "buffered" version behave in a similar way? it basically moves segments on unacked_q back to write_q.
how do you think?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] a-lunev commented on a change in pull request #4659: net/tcp(unbuffered): retransmit only one the earliest not acknowledged segment
Posted by GitBox <gi...@apache.org>.
a-lunev commented on a change in pull request #4659:
URL: https://github.com/apache/incubator-nuttx/pull/4659#discussion_r727627443
##########
File path: net/tcp/tcp_send_unbuffered.c
##########
@@ -243,18 +243,54 @@ static uint16_t tcpsend_eventhandler(FAR struct net_driver_s *dev,
else if ((flags & TCP_REXMIT) != 0)
{
+#if defined(CONFIG_NET_TCP_SPLIT)
/* Yes.. in this case, reset the number of bytes that have been sent
* to the number of bytes that have been ACKed.
*/
pstate->snd_sent = pstate->snd_acked;
Review comment:
Yes, it does. I wrote about this in the PR summary. It contradicts to RFC 6298 (5.4) and other TCP related RFCs. Possibly it was done intentionally as a workaround (as I understand, CONFIG_NET_TCP_SPLIT algorithm is some sort of a trick as it is described in the source code comments).
If CONFIG_NET_TCP_SPLIT was planned to be removed, the removing would simplify this part of the code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] yamt commented on pull request #4659: net/tcp(unbuffered): retransmit only one the earliest not acknowledged segment
Posted by GitBox <gi...@apache.org>.
yamt commented on pull request #4659:
URL: https://github.com/apache/incubator-nuttx/pull/4659#issuecomment-941801270
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] a-lunev commented on pull request #4659: net/tcp(unbuffered): retransmit only one the earliest not acknowledged segment
Posted by GitBox <gi...@apache.org>.
a-lunev commented on pull request #4659:
URL: https://github.com/apache/incubator-nuttx/pull/4659#issuecomment-965029814
My patch (PR #4659) does not add a new functionality. It just fixes the existing basic retransmission part in NuttX TCP/IP stack ("unbuffered send" mode).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] xiaoxiang781216 merged pull request #4659: net/tcp(unbuffered): retransmit only one the earliest not acknowledged segment
Posted by GitBox <gi...@apache.org>.
xiaoxiang781216 merged pull request #4659:
URL: https://github.com/apache/incubator-nuttx/pull/4659
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] a-lunev commented on pull request #4659: net/tcp(unbuffered): retransmit only one the earliest not acknowledged segment
Posted by GitBox <gi...@apache.org>.
a-lunev commented on pull request #4659:
URL: https://github.com/apache/incubator-nuttx/pull/4659#issuecomment-964685522
> > > i still think it's better to do it with cwnd-like structure. let me think a bit.
> >
> >
> > By "cwnd" do you mean congestion window control algorithm or something else?
>
> yes, congestion window. clamping the window to one segment for retransmit is what this patch does, right?
You are saying about RFC 5681 (slow start of sending data etc.). My patch is not related to the congestion window control. My patch instead just fixes a basic requirement (RFC 6298 (5.4)) how a TCP segment should be retransmitted e.g. in case of ACK timeout. If it is detected that a segment is lost, only that particular segment should be retransmitted. However, currently NuttX TCP/IP stack (at least "unbuffered send" mode) retransmits all segments between NACK and SEQ provoking network traffic collapse.
The congestion window control is a separate advanced algorithm that I think could be implemented in NuttX TCP/IP stack as an option.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] a-lunev commented on a change in pull request #4659: net/tcp(unbuffered): retransmit only one the earliest not acknowledged segment
Posted by GitBox <gi...@apache.org>.
a-lunev commented on a change in pull request #4659:
URL: https://github.com/apache/incubator-nuttx/pull/4659#discussion_r727667843
##########
File path: net/tcp/tcp.h
##########
@@ -172,6 +172,9 @@ struct tcp_conn_s
uint8_t rcvseq[4]; /* The sequence number that we expect to
* receive next */
uint8_t sndseq[4]; /* The sequence number that was last sent by us */
+#if !defined(CONFIG_NET_TCP_SPLIT) && !defined(CONFIG_NET_TCP_WRITE_BUFFERS)
+ uint32_t rexmit_seq; /* The sequence number to be retrasmitted */
Review comment:
Previously I read your description in #4293 ("in some cases, we send tcp segments with sequence numbers back and forth.
especially when sending an empty segment. (eg. pure ack, window updates)").
Thus I have added conn->rexmit_seq field that is used only for retrasmitting a single TCP segment w/o a need to modify conn->sndseq each time and recovering it back.
Thus I consider using conn->rexmit_seq as more reliable and stable way rather than using conn->sndseq for retransmissions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] yamt commented on pull request #4659: net/tcp(unbuffered): retransmit only one the earliest not acknowledged segment
Posted by GitBox <gi...@apache.org>.
yamt commented on pull request #4659:
URL: https://github.com/apache/incubator-nuttx/pull/4659#issuecomment-941850277
> > > > btw, it has been on my todo list to remove CONFIG_NET_TCP_SPLIT. if it helps your work, please let me know. it isn't urgent and i don't want to interfere you by submitting conflicting patches.
> > >
> > >
> > > Oh, really? I do not personally need CONFIG_NET_TCP_SPLIT algorithm. What about others? Is there any previous discussion about its deletion? I'm not sure, if there is a reason to preserve it? If I understand correctly, CONFIG_NET_TCP_SPLIT was developed for MCUs with tiny memories that do not have enough memory for a sufficient TCP retransmission queue depth.
> >
> >
> > it's just my personal todo. not community's. while i can understand the motivation, it doesn't work well even with a traditional tcp stack.
>
> Can we poll NuttX developers about CONFIG_NET_TCP_SPLIT deletion? As I said, I personally do not need it. However, I'm afraid of if someone still needs CONFIG_NET_TCP_SPLIT?
it's a valid concern. i posted it on the ML.
https://www.mail-archive.com/dev@nuttx.apache.org/msg07045.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] yamt commented on pull request #4659: net/tcp(unbuffered): retransmit only one the earliest not acknowledged segment
Posted by GitBox <gi...@apache.org>.
yamt commented on pull request #4659:
URL: https://github.com/apache/incubator-nuttx/pull/4659#issuecomment-941828271
> > btw, it has been on my todo list to remove CONFIG_NET_TCP_SPLIT. if it helps your work, please let me know. it isn't urgent and i don't want to interfere you by submitting conflicting patches.
>
> Oh, really? I do not personally need CONFIG_NET_TCP_SPLIT algorithm. What about others? Is there any previous discussion about its deletion? I'm not sure, if there is a reason to preserve it? If I understand correctly, CONFIG_NET_TCP_SPLIT was developed for MCUs with tiny memories that do not have enough memory for a sufficient TCP retransmission queue depth.
it's just my personal todo. not community's.
while i can understand the motivation, it doesn't work well even with a traditional tcp stack.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] a-lunev edited a comment on pull request #4659: net/tcp(unbuffered): retransmit only one the earliest not acknowledged segment
Posted by GitBox <gi...@apache.org>.
a-lunev edited a comment on pull request #4659:
URL: https://github.com/apache/incubator-nuttx/pull/4659#issuecomment-941886030
> * i suspect this is something should be fixed with cwnd, which nuttx doesn't have right now.
>
> * doesn't the "buffered" version behave in a similar way? it basically moves segments on unacked_q back to write_q.
>
>
> how do you think?
I think it would be reasonable to implement congestion window control as an option. Also I see that Fast Retransmit algorithm is not supported in unbuffered tcp mode of NuttX that is possibly even more important.
Concerning "buffered" version I can not say at the moment as I have not yet analyzed the source code in that part.
So far I'm using "unbuffered" mode as in my case it surprisingly provides twice as high throughput as "buffered" one. Though, I expected that "buffered" mode should have better performance compared to "unbuffered" one.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] a-lunev commented on pull request #4659: net/tcp(unbuffered): retransmit only one the earliest not acknowledged segment
Posted by GitBox <gi...@apache.org>.
a-lunev commented on pull request #4659:
URL: https://github.com/apache/incubator-nuttx/pull/4659#issuecomment-962528759
> > > > > btw, it has been on my todo list to remove CONFIG_NET_TCP_SPLIT. if it helps your work, please let me know. it isn't urgent and i don't want to interfere you by submitting conflicting patches.
> > > >
> > > >
> > > > Oh, really? I do not personally need CONFIG_NET_TCP_SPLIT algorithm. What about others? Is there any previous discussion about its deletion? I'm not sure, if there is a reason to preserve it? If I understand correctly, CONFIG_NET_TCP_SPLIT was developed for MCUs with tiny memories that do not have enough memory for a sufficient TCP retransmission queue depth.
> > >
> > >
> > > it's just my personal todo. not community's. while i can understand the motivation, it doesn't work well even with a traditional tcp stack.
> >
> >
> > Can we poll NuttX developers about CONFIG_NET_TCP_SPLIT deletion? As I said, I personally do not need it. However, I'm afraid of if someone still needs CONFIG_NET_TCP_SPLIT?
>
> it's a valid concern. i posted it on the ML. https://www.mail-archive.com/dev@nuttx.apache.org/msg07045.html
@yamt as PR #4660 is already merged into the master branch, I have adapted my patch to the removed NET_TCP_SPLIT option.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] yamt commented on a change in pull request #4659: net/tcp(unbuffered): retransmit only one the earliest not acknowledged segment
Posted by GitBox <gi...@apache.org>.
yamt commented on a change in pull request #4659:
URL: https://github.com/apache/incubator-nuttx/pull/4659#discussion_r727610211
##########
File path: net/tcp/tcp_send_unbuffered.c
##########
@@ -243,18 +243,54 @@ static uint16_t tcpsend_eventhandler(FAR struct net_driver_s *dev,
else if ((flags & TCP_REXMIT) != 0)
{
+#if defined(CONFIG_NET_TCP_SPLIT)
/* Yes.. in this case, reset the number of bytes that have been sent
* to the number of bytes that have been ACKed.
*/
pstate->snd_sent = pstate->snd_acked;
Review comment:
doesn't this mean not to resend bytes already acked?
if this is not working for some reasons, it's better to fix it instead of introducing a separate code path.
##########
File path: net/tcp/tcp.h
##########
@@ -172,6 +172,9 @@ struct tcp_conn_s
uint8_t rcvseq[4]; /* The sequence number that we expect to
* receive next */
uint8_t sndseq[4]; /* The sequence number that was last sent by us */
+#if !defined(CONFIG_NET_TCP_SPLIT) && !defined(CONFIG_NET_TCP_WRITE_BUFFERS)
+ uint32_t rexmit_seq; /* The sequence number to be retrasmitted */
Review comment:
i feel a bit uneasy about yet another communication channel to tcp_send.c.
isn't it a responsibility of tcp_send_xxx and other event handlers to set sndseq appropriately?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] a-lunev commented on pull request #4659: net/tcp(unbuffered): retransmit only one the earliest not acknowledged segment
Posted by GitBox <gi...@apache.org>.
a-lunev commented on pull request #4659:
URL: https://github.com/apache/incubator-nuttx/pull/4659#issuecomment-941860848
> it's a valid concern. i posted it on the ML. https://www.mail-archive.com/dev@nuttx.apache.org/msg07045.html
Thank you.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] a-lunev commented on pull request #4659: net/tcp(unbuffered): retransmit only one the earliest not acknowledged segment
Posted by GitBox <gi...@apache.org>.
a-lunev commented on pull request #4659:
URL: https://github.com/apache/incubator-nuttx/pull/4659#issuecomment-941826145
> btw, it has been on my todo list to remove CONFIG_NET_TCP_SPLIT. if it helps your work, please let me know. it isn't urgent and i don't want to interfere you by submitting conflicting patches.
Oh, really? I do not personally need CONFIG_NET_TCP_SPLIT algorithm. What about others? Is there any previous discussion about its deletion? I'm not sure, if there is a reason to preserve it? If I understand correctly, CONFIG_NET_TCP_SPLIT was developed for MCUs with tiny memories that do not have enough memory for a sufficient TCP retransmission queue depth.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] a-lunev commented on pull request #4659: net/tcp(unbuffered): retransmit only one the earliest not acknowledged segment
Posted by GitBox <gi...@apache.org>.
a-lunev commented on pull request #4659:
URL: https://github.com/apache/incubator-nuttx/pull/4659#issuecomment-964625263
I forced-push to restart ci because "Build Documentation / build-html" previously failed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org