You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@nuttx.apache.org by GitBox <gi...@apache.org> on 2022/08/30 09:28:28 UTC

[GitHub] [incubator-nuttx] fjpanag opened a new issue, #6960: Behaviour of CONFIG_NET_ALLOC_CONNS

fjpanag opened a new issue, #6960:
URL: https://github.com/apache/incubator-nuttx/issues/6960

A while back I studied the implementation of `CONFIG_NET_ALLOC_CONNS`, and I saw some things that seemed strange to me.

I would like to discuss my thoughts, being motivated by #6956.

I see that when `CONFIG_NET_ALLOC_CONNS` is enabled, arrays like `g_cbprealloc[]` are not defined. Rather, the connection structures are dynamically allocated, as expected.

However, I see that:
* When new connection structure is needed, a **batch** of them is allocated. Not only one. For example in `tcp_conn.c` line 559.
* When the connection is closed, the structure is returned in the "free" list instead of it being deallocated.

I believe that this is not correct.
When you only need one single new connection, the memory allocated may be much more than actually needed.
If the system experiences a load-spike, creating many new connections, it will never be able to return to "normal". Memory used for the connections is lost forever, multiple connections may not be needed ever again.

In one of our cases this is more prominent.
During system boot many connections are opened towards various services. When everything is ready, the system keeps only one connection open for the rest of its life-time.

---

I propose the following changes:
* Allocate only one structure when needed.
* Ensure that when finished, the connection is actually de-allocated properly.
* Maybe keep a small (and configurable) pool of structs always allocated (either from the heap or BSS), to reduce heap fragmentation, and enhance performance. This will be useful for example, in cases of multiple successive calls to `netlib_*()` functions.

--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [nuttx] fjpanag commented on issue #6960: Behaviour of CONFIG_NET_ALLOC_CONNS

Posted by GitBox <gi...@apache.org>.

fjpanag commented on issue #6960:
URL: https://github.com/apache/nuttx/issues/6960#issuecomment-1346306045

   As work is being done on this, I am taking note of the connections that need improvement:
   
   - [ ] Bluetooth (`g_bluetooth_connections`)
   - [ ] Devif (`g_cbprealloc`)
   - [ ] ICMP (`g_icmp_connections`)
   - [ ] ICMPv6 (`g_icmpv6_connections`)
   - [ ] IEEE802154 (`g_ieee802154_connections`)
   - [ ] Netlink (`g_netlink_connections`)
   - [ ] Pkt (`g_pkt_connections`)
   - [ ] TCP (`g_tcp_connections`)
   - [ ] UDP (`g_udp_connections`)
   - [ ] Usrsock (`g_usrsock_connections`)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [nuttx] xiaoxiang781216 closed issue #6960: Behaviour of CONFIG_NET_ALLOC_CONNS

Posted by "xiaoxiang781216 (via GitHub)" <gi...@apache.org>.

xiaoxiang781216 closed issue #6960: Behaviour of CONFIG_NET_ALLOC_CONNS
URL: https://github.com/apache/nuttx/issues/6960


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-nuttx] fjpanag commented on issue #6960: Behaviour of CONFIG_NET_ALLOC_CONNS

Posted by GitBox <gi...@apache.org>.

fjpanag commented on issue #6960:
URL: https://github.com/apache/incubator-nuttx/issues/6960#issuecomment-1231500115

   > If you like allocate/free every time when the pool is exhausted and don't care about the fragmentation. We can simply do this special action by checking `CONFIG_NET_ALLOC_CONNS == 1`
   
   No it is not the same. In this case, when you finish using the connection it is returned to the free list, it is not deallocated with a proper call to `free()`.
   
   Thus your system ends up using memory equal to its maximum number of connections it had seen, regardless of the current needs.
   
   Consider for example a case were 20 connections are needed for 5 mins and then only 1 for the rest of the operation. In this case, 20 connections will always be reserved from the heap, forever.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-nuttx] xiaoxiang781216 commented on issue #6960: Behaviour of CONFIG_NET_ALLOC_CONNS

Posted by GitBox <gi...@apache.org>.

xiaoxiang781216 commented on issue #6960:
URL: https://github.com/apache/incubator-nuttx/issues/6960#issuecomment-1231489177

   If you like allocate/free every time when the pool is exhausted and don't care about the fragmentation. We can simply do this special action by checking `CONFIG_NET_ALLOC_CONNS == 1`


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-nuttx] fjpanag commented on issue #6960: Behaviour of CONFIG_NET_ALLOC_CONNS

Posted by GitBox <gi...@apache.org>.

fjpanag commented on issue #6960:
URL: https://github.com/apache/incubator-nuttx/issues/6960#issuecomment-1231415692

   Personally I think I would prefer us to ditch the `CONFIG_NET_ALLOC_CONNS` option completely.
   
   Always have a poll of statically allocated structs, for example using options like `CONFIG_NET_TCP_CONNS`, and when this pool is exhausted, proceed to heap allocations.
   
   A hybrid approach. Like `CONFIG_PREALLOC_TIMERS`.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-nuttx] xiaoxiang781216 commented on issue #6960: Behaviour of CONFIG_NET_ALLOC_CONNS

Posted by GitBox <gi...@apache.org>.

xiaoxiang781216 commented on issue #6960:
URL: https://github.com/apache/incubator-nuttx/issues/6960#issuecomment-1231539487

   I mean you can kmm_free directly in case of `ONFIG_NET_ALLOC_CONNS == 1` not to the pool.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org