You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@nuttx.apache.org by GitBox <gi...@apache.org> on 2021/02/09 13:36:32 UTC
[GitHub] [incubator-nuttx] xiaoxiang781216 opened a new issue #2834: Dynamically extend fd table as needed
xiaoxiang781216 opened a new issue #2834:
URL: https://github.com/apache/incubator-nuttx/issues/2834
Now, all tasks can have hold at most CONFIG_NFILE_DESCRIPTORS file handles. But the complexity for each task is totally different, so it's better to extend the file table size at runtime to match the different requirement.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] v01d commented on issue #2834: Dynamically extend fd table as needed
Posted by GitBox <gi...@apache.org>.
v01d commented on issue #2834:
URL: https://github.com/apache/incubator-nuttx/issues/2834#issuecomment-778209556
It would be great to support this as it is very common for me to start building app my project logic when it eventually fails to due opening to many FDs. It is the kind of setting that requires guesswork, which is not ideal.
Do you think it could affect performance? I imagine it would require traversing a list for every file operation. Or would you reallocate an array?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] xiaoxiang781216 edited a comment on issue #2834: Dynamically extend fd table as needed
Posted by GitBox <gi...@apache.org>.
xiaoxiang781216 edited a comment on issue #2834:
URL: https://github.com/apache/incubator-nuttx/issues/2834#issuecomment-776470675
Yes, we plan to address this issue first: https://github.com/apache/incubator-nuttx/issues/2833, the rough idea is that:
1. Keep the kernel psock API as before
2. Implement a new file_operation g_sock_fileops
3. socket api allocate handle from filelist and associate with g_sock_fileops
4. Common API(e.g. dup, close, ioctl, read, write) is handled by new g_sock_fileops
5. g_sock_fileops forward to the related psock API or reuse fs implementation directly
6. Remove socklist from task_group and NSOCKET_DESCRIPTORS option
so, we get an unified representation of file and socket.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] xiaoxiang781216 commented on issue #2834: Dynamically extend fd table as needed
Posted by GitBox <gi...@apache.org>.
xiaoxiang781216 commented on issue #2834:
URL: https://github.com/apache/incubator-nuttx/issues/2834#issuecomment-776470675
We plan to address this issue first: https://github.com/apache/incubator-nuttx/issues/2833, the rough idea is that:
1. Keep the kernel psock API as before
2. Implement a new file_operation g_sock_fileops
3. socket api allocate handle from filelist and associate with g_sock_fileops
4. Common API(e.g. dup, close, ioctl, read, write) is handled by new g_sock_fileops
5. g_sock_fileops forward common API to the related psock API
6. Remove socklist from task_group and NSOCKET_DESCRIPTORS option
so, we get an unified representation of file and socket.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] xiaoxiang781216 commented on issue #2834: Dynamically extend fd table as needed
Posted by GitBox <gi...@apache.org>.
xiaoxiang781216 commented on issue #2834:
URL: https://github.com/apache/incubator-nuttx/issues/2834#issuecomment-778239483
> It would be great to support this as it is very common for me to start building app my project logic when it eventually fails to due opening to many FDs. It is the kind of setting that requires guesswork, which is not ideal.
>
> Do you think it could affect performance? I imagine it would require traversing a list for every file operation. Or would you reallocate an array?
Yes, fd->file conversion become to O(n) operation if we switch to list, but the reallocation array isn't perfect too, because:
1. All file pointers will be invalid after reallocation, we has to hold the lock in the entire fs call.
2. It's very hard to find the big continuous free memory to hold the fd array
so we propose a two layer array structure(like STL deque):
We can get both benefit from list and array:
1. Convert fd to file in O(1)
2. Allocated file struct never become invalid
3. Don't require the bigger and bigger continuous free memory
4. Avoid the housekeeping overhead of list(each file ~12 bytes)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] xiaoxiang781216 closed issue #2834: Dynamically extend fd table as needed
Posted by GitBox <gi...@apache.org>.
xiaoxiang781216 closed issue #2834:
URL: https://github.com/apache/incubator-nuttx/issues/2834
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] patacongo commented on issue #2834: Dynamically extend fd table as needed
Posted by GitBox <gi...@apache.org>.
patacongo commented on issue #2834:
URL: https://github.com/apache/incubator-nuttx/issues/2834#issuecomment-776312386
From top-level TODO list (under " UNIFIED DESCRIPTOR REPRESENTATION"):
Another example of how the current implementation limits
functionality: I recently started to implement of the FILEMAX
(using pctl() instead sysctl()). My objective was to be able
to control the number of available file descriptors on a task-
by-task basis. The complexity due to the partitioning of
descriptor space into a range for file descriptors and a range
for socket descriptors made this feature nearly impossible to
implement.
Are you considering implementation of the FILEMAX sysctl() call? That is one of the non-standard OS interfaces that at least has a precedent. The Linux call is global, I believe. prctl() would be a better location for the per-task control of the number of file descriptors.
What happens when a task is spawned? Does the FILEMAX size propagate too? I think that it has to be inherited or else you could not propagate the open file descriptors.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] xiaoxiang781216 edited a comment on issue #2834: Dynamically extend fd table as needed
Posted by GitBox <gi...@apache.org>.
xiaoxiang781216 edited a comment on issue #2834:
URL: https://github.com/apache/incubator-nuttx/issues/2834#issuecomment-776470675
Yes, we plan to address this issue first: https://github.com/apache/incubator-nuttx/issues/2833, the rough idea is that:
1. Keep the kernel psock API as before
2. Implement a new file_operation g_sock_fileops
3. socket api allocate handle from filelist and associate with g_sock_fileops
4. Common API(e.g. dup, close, ioctl, read, write) is handled by new g_sock_fileops
5. g_sock_fileops forward common API to the related psock API or reuse fs implementation directly
6. Remove socklist from task_group and NSOCKET_DESCRIPTORS option
so, we get an unified representation of file and socket.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org