You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@nuttx.apache.org by GitBox <gi...@apache.org> on 2020/05/06 16:37:36 UTC
[GitHub] [incubator-nuttx] patacongo opened a new issue #986: Userspace errno
patacongo opened a new issue #986:
URL: https://github.com/apache/incubator-nuttx/issues/986
## Background
The user errno value is currently kept in the TCB and can be accesses via an OS system call called `__errno()`.
# define errno *__errno()
The errno valud is kept in the thread's TCB in the `pterrno` field. The _`_errno() `function finds the TCB associated with the thread and returns a pointer to the errno storage location.
This works in the FLAT build mode in because it dereferences the address of the errno in TCB as both an RVALUE:
errcode == errno;
And as an LVALUE:
errno = errode;
That works, however, it is rather inefficient.
However, that will not work at all in the PROTECTED or KERNEL build modes because the memory holding the TCB is protected and cannot be directly accessed from the user mode application. Instead, it is accessed through accessor functions:
void set_errno(int errcode);
int get_errno(void);
Accessing the errno via these functions in PROTECTED or KERNEL mode is a huge performance hit, because the calles must go system call which is implemented as a software interrupt.
And in this case, the errno is defined to be:
# define errno get_errno()
This works fine as an RVALUE:
errcode == errno;
But will cause a compilation error if used as an LVALUE:
errno = errode;
## TLS
The proposed solution is to move the errno storage location out of the proctected TCB memory and into the unprotected application thread stack memory. Then it can be access with the appropriate TLS (Thread Local Storage) interfaces.
NOTE: The errno value must still be retained in the TCB because it is also used for other purpose, such as informing sem_wait() that it was awakened by a a signal.
## What has been done?
Historically, TLS works by aligning the stack base address then simplying AND'ing the current stack pointer to obtain the base address of the stack where the TLS data can be found (as struct tls_info_s). This works well in the KERNEL build mode where is the resides at highly aligned virtual address, but does not work well in FLAT and PROTECTED modes:
1. The alignment must be large. That is because it also determine the maximum size of the thread's stack. If the size of the thread's stack exceeds that maximum value then the AND operation will result in the wrong address.
2. If all of the addresses are highly aligned then (1) you need to have much more memory available for stack allocation. This is because (2) the large alignment cause bad memory fragmentation and degraded use of memory.
An alternative way to get the stack base address address is to call into the OS to get the unaligned stack base address. This involves a system call, but is more usable that other alernatives in the FLAT and PROTECTED modes.
We have implemented a new configuration: CONFIG_TLS_ALIGNED that selects the legacy aligned stack. If this is not defined, then the new unaligned stack TLS logic is used. We have also implemented aligned and unaligned TLS support for every architecture.
## What else do we have to do?
The next steps are:
1. Move the errno storage location out of the TCB and in TLS (into the struct tls_info_s).
2. Modify the errno access definitions and logic in sched/errno to use the TLS logic in user space. This probably belongs in libs/libc/tls or perhaps libs/libc/errno.
3. Enable TLS be default.
4. Remove the errno-related system calls.
## pthread-specific data
pthread-specific data is another mechanism for accessing thread-specific data. It consists of these POSIX standard interfaces:
int pthread_key_create(FAR pthread_key_t *key,
CODE void (*destructor)(FAR void *));
int pthread_setspecific(pthread_key_t key, FAR const void *value);
FAR void *pthread_getspecific(pthread_key_t key);
int pthread_key_delete(pthread_key_t key);
This is a natural extension of the TLS logic would be natural to move into the TLS data as well. This would amount to:
1. Moving the pthread-specific storage out of the TCB and into the TLS data.
2. Moving the pthread-specific data accessors out of sched/pthread and into libs/libc/pthread
3. Remove the pthread-specific data system calls.
## TLS interfaces
TLS is a non-standard, but more general interface. It differs from pthread-specific data only in that is semantics are general; the semantics of the pthread-specific data interfaces are focused on pthreads. But they really should shared the same common underly logic.
Currently, there are two TLS interfaces:
uintptr_t tls_get_element(int elem);
void tls_set_element(int elem, uintptr_t value);
I propose that we extend these interfaces so that they can coexit with the TLS interfaces. Linux (actual GDB) does not provide a good mechanism; it relies on a storage class that is specific to ELF binaries. That is not useful in an embedded system.
Instead, I think we should adapt from the Windows TLS interfaces which are directly anlagous to the POSIC pthread-specific data interfaces. Here is such an adaption (following NuttX coding standards. See https://en.wikipedia.org/wiki/Thread-local_storage#Windows_implementation):
int tls_alloc(void);
FAR void *tls_get_value(int tlsindex);
bool tls_set_value(int tlsindex, uinptr_t value);
bool tls_free(int tlsindex);
The Windows prototypes can be found from links on this page: https://docs.microsoft.com/en-us/windows/win32/api/processthreadsapi/nf-processthreadsapi-tlsalloc
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] xiaoxiang781216 edited a comment on issue #986: Userspace errno
Posted by GitBox <gi...@apache.org>.
xiaoxiang781216 edited a comment on issue #986:
URL: https://github.com/apache/incubator-nuttx/issues/986#issuecomment-625306304
> ## TLS interfaces
> TLS is a non-standard, but more general interface. It differs from pthread-specific data only in that is semantics are general; the semantics of the pthread-specific data interfaces are focused on pthreads. But they really should shared the same common underly logic.
But we already use pthread API across the code base which is only portable C API we can use for the thread creation. I don't think there is any problem that TLS desgin bias on pthreads. The tls_* exist just because we can't move all per phread data to the start of the stack because the old implmentation require the very huge stack alignment which isn't suitable for FLAT/PROTECTED build. Since we remove the alignment requirement from the new TLS implmentation, we can move all per thread data(include errno and pthread TLS) into the stack. Then I prefer that pthread TLS API is implemented directly on top of sched_get_stackinfo and remove all private tls_* API.
>
> Currently, there are two TLS interfaces:
>
> ```
> uintptr_t tls_get_element(int elem);
> void tls_set_element(int elem, uintptr_t value);
> ```
>
> I propose that we extend these TLS interfaces so that they can coexit with the pthread interfaces. Linux (actual GLIBC) does not provide a good mechanism; it relies on a storage class that is specific to ELF binaries. That is not useful in an embedded system where no ELF information is present.
>
> [This is a case of the bad decision to let specific tools environment drive a design. Good designs should be independent of tools].
>
> Instead, I think we should adapt from the Windows TLS interfaces which are directly anlagous to the POSIC pthread-specific data interfaces. Here is such an adaption (following NuttX coding standards. See https://en.wikipedia.org/wiki/Thread-local_storage#Windows_implementation):
>
> ```
> int tls_alloc(void);
> FAR void *tls_get_value(int tlsindex);
> bool tls_set_value(int tlsindex, uinptr_t tlsvalue);
> bool tls_free(int tlsindex);
> ```
>
> The Windows prototypes can be found from links on this page: https://docs.microsoft.com/en-us/windows/win32/api/processthreadsapi/nf-processthreadsapi-tlsalloc
>
Why we can't directly use pthread TLS API here? I don't see there is any benefit to define a new set of interface and forward pthread API to them, which just mean that:
1.Increase the code size
2.Increase the document effort
3.Make people confusion which API he/she should use
4.The mix usage will spread in the code base and inroduce the inconsistence
> ## Accessing the errno from kernel space.
> We should never access the errno from kernel space. This is especially dangerous from kernel space because we can't be certain which user address environment is in effect or which stack is being used (if we were to use separate kernel mode stacks in the future as Linux does ... for security reasons).
>
> Most OS interfaces have two parts:
>
> 1. A user callable function, say `osapi()` that sets the errno value is necessary (and may implement cancellation points) and
> 2. An internal OS version which might then be `nx_osapi()` which does not modify the errno value.
>
This design isn't perfect, most people don't understand the difference between osapi and nx_osapi. Even he/she know the difference but often forget in coding. I have fixed more than thousound of this type of bug(e.g. driver call osapi) and find that the same issue appear in the new code.
> Ideally, all of the user callable OS interfaces (the `osapi()`) should be moved to the location in libs/libc the system call (`sycall/`) should be redirected to `nx_osapi()`. A complication to doing this is that the OS interfaces which are also cancellation points call special internal interfaces, `enter_cancellation_point()` and `leave_cancellation_point()` that are not available from user space. So some addition partitioning would be need to accomplish this.
>
Ideally, we just need to:
1.Implement osapi which return error code instead modifing errno
Then the difference can be done by the tool automatically:
2.STUB_xxx can add cancel point for us
3.The PROXY_xxx can modify errno for us
For FLAT build, since we don't separate userspace and kernel space, some hack need to be taken:
1.Redefine osapi to nx_osapi when __KERNEL__ is defined
2.Rename osapi to nx_osapi by toolchain like simulator
Maybe we can find better method after more dissussing. But at least we definitely can find a way to do this thing automatically.
With this design, both kernel/userspace call API with the same name, but get the little bit different behaviour.
> ## Code References
> Things to look at:
>
> ```
> include/errno.h - Defines current errno access
> include/nuttx/tls.h - Defines the tls_info_s structure
> include/nuttx/sched.h - The errno and pthread-specific data
> sched/errno/* - The old errno access logic. Move to libs/libc/errno or tls
> libs/libc/tls - The new TLS and possible errno access logic
> sched/pthread - pthread_key*.c, pthread_*specific.c - Move to libs/libc/pthread
> ```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] xiaoxiang781216 edited a comment on issue #986: Userspace errno
Posted by GitBox <gi...@apache.org>.
xiaoxiang781216 edited a comment on issue #986:
URL: https://github.com/apache/incubator-nuttx/issues/986#issuecomment-625306304
> ## TLS interfaces
> TLS is a non-standard, but more general interface. It differs from pthread-specific data only in that is semantics are general; the semantics of the pthread-specific data interfaces are focused on pthreads. But they really should shared the same common underly logic.
But we already use pthread API across the code base which is only portable C API we can use for the thread creation. I don't think there is any problem that TLS desgin bias on pthreads. The tls_* exist just because we can't move phread TLS storage to the start of the stack because the old implmentation require the very huge stack alignment which isn't suitable for FLAT/PROTECTED build. Since we remove the alignment requirement from the new TLS implmentation, we can move all per thread data(include errno and pthread TLS) into the stack. I prefer that pthread TLS API is implemented directly on top of sched_get_stackinfo and remove all private tls_* API.
>
> Currently, there are two TLS interfaces:
>
> ```
> uintptr_t tls_get_element(int elem);
> void tls_set_element(int elem, uintptr_t value);
> ```
>
> I propose that we extend these TLS interfaces so that they can coexit with the pthread interfaces. Linux (actual GLIBC) does not provide a good mechanism; it relies on a storage class that is specific to ELF binaries. That is not useful in an embedded system where no ELF information is present.
>
> [This is a case of the bad decision to let specific tools environment drive a design. Good designs should be independent of tools].
>
> Instead, I think we should adapt from the Windows TLS interfaces which are directly anlagous to the POSIC pthread-specific data interfaces. Here is such an adaption (following NuttX coding standards. See https://en.wikipedia.org/wiki/Thread-local_storage#Windows_implementation):
>
> ```
> int tls_alloc(void);
> FAR void *tls_get_value(int tlsindex);
> bool tls_set_value(int tlsindex, uinptr_t tlsvalue);
> bool tls_free(int tlsindex);
> ```
>
> The Windows prototypes can be found from links on this page: https://docs.microsoft.com/en-us/windows/win32/api/processthreadsapi/nf-processthreadsapi-tlsalloc
>
Why we can't directly use pthread TLS API here? I don't see there is any benefit to define a new set of interface and forward pthread API to them, which just:
1.Increase the code size
2.Increase the document effort
3.Make people confusion which API he/she should use
4.The mix usage will spread in the code base and inroduce the inconsistence
> ## Accessing the errno from kernel space.
> We should never access the errno from kernel space. This is especially dangerous from kernel space because we can't be certain which user address environment is in effect or which stack is being used (if we were to use separate kernel mode stacks in the future as Linux does ... for security reasons).
>
> Most OS interfaces have two parts:
>
> 1. A user callable function, say `osapi()` that sets the errno value is necessary (and may implement cancellation points) and
> 2. An internal OS version which might then be `nx_osapi()` which does not modify the errno value.
>
This design isn't perfect, most people don't understand the difference between osapi and nx_osapi. Even he/she know the difference but often forget in coding. I have fixed more than thousound of this type of bug(e.g. driver call osapi) and find that the same issue appear in the new code.
> Ideally, all of the user callable OS interfaces (the `osapi()`) should be moved to the location in libs/libc the system call (`sycall/`) should be redirected to `nx_osapi()`. A complication to doing this is that the OS interfaces which are also cancellation points call special internal interfaces, `enter_cancellation_point()` and `leave_cancellation_point()` that are not available from user space. So some addition partitioning would be need to accomplish this.
>
Ideally, we just need to:
1.Implement osapi which return error code instead modifing errno
Then the difference can be done by the tool automatically:
2.STUB_xxx can add cancel point for us
3.The PROXY_xxx can modify errno for us
For FLAT build, since we don't separate userspace and kernel space, some hack need to be taken:
1.Redefine osapi to nx_osapi when __KERNEL__ is defined
2.Rename osapi to nx_osapi by toolchain like simulator
Maybe we can find better method after more dissussing. But at least we definitely can find a way to do this thing automatically.
> ## Code References
> Things to look at:
>
> ```
> include/errno.h - Defines current errno access
> include/nuttx/tls.h - Defines the tls_info_s structure
> include/nuttx/sched.h - The errno and pthread-specific data
> sched/errno/* - The old errno access logic. Move to libs/libc/errno or tls
> libs/libc/tls - The new TLS and possible errno access logic
> sched/pthread - pthread_key*.c, pthread_*specific.c - Move to libs/libc/pthread
> ```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] xiaoxiang781216 commented on issue #986: Userspace errno
Posted by GitBox <gi...@apache.org>.
xiaoxiang781216 commented on issue #986:
URL: https://github.com/apache/incubator-nuttx/issues/986#issuecomment-625612125
@patacongo Ok. I will look at the change in that branch.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] xiaoxiang781216 edited a comment on issue #986: Userspace errno
Posted by GitBox <gi...@apache.org>.
xiaoxiang781216 edited a comment on issue #986:
URL: https://github.com/apache/incubator-nuttx/issues/986#issuecomment-625306304
> ## TLS interfaces
> TLS is a non-standard, but more general interface. It differs from pthread-specific data only in that is semantics are general; the semantics of the pthread-specific data interfaces are focused on pthreads. But they really should shared the same common underly logic.
But we already use pthread API across the code base which is only portable C API we can use for the thread creation. I don't think there is any problem that TLS inteface focus on pthreads. The tls_* exist just because we can't move phread TLS storage to the start of the stack because the old implmentation require the very huge stack alignment which isn't suitable for FLAT/PROTECTED build. Since we remove the alignment requirement from the new TLS implmentation, we can move all per thread data(include errno and pthread TLS) into the stack. I prefer that pthread TLS API is implemented directly on top of sched_get_stackinfo and remove all private tls_* API.
>
> Currently, there are two TLS interfaces:
>
> ```
> uintptr_t tls_get_element(int elem);
> void tls_set_element(int elem, uintptr_t value);
> ```
>
> I propose that we extend these TLS interfaces so that they can coexit with the pthread interfaces. Linux (actual GLIBC) does not provide a good mechanism; it relies on a storage class that is specific to ELF binaries. That is not useful in an embedded system where no ELF information is present.
>
> [This is a case of the bad decision to let specific tools environment drive a design. Good designs should be independent of tools].
>
> Instead, I think we should adapt from the Windows TLS interfaces which are directly anlagous to the POSIC pthread-specific data interfaces. Here is such an adaption (following NuttX coding standards. See https://en.wikipedia.org/wiki/Thread-local_storage#Windows_implementation):
>
> ```
> int tls_alloc(void);
> FAR void *tls_get_value(int tlsindex);
> bool tls_set_value(int tlsindex, uinptr_t tlsvalue);
> bool tls_free(int tlsindex);
> ```
>
> The Windows prototypes can be found from links on this page: https://docs.microsoft.com/en-us/windows/win32/api/processthreadsapi/nf-processthreadsapi-tlsalloc
>
Why we can't directly use pthread TLS API here? I don't see there is any benefit to define a new set of interface and forward pthread API to them, which just:
1.Increase the code size
2.Increase the document effort
3.Make people confusion which API he/she should use
4.The mix usage will spread in the code base and inroduce the inconsistence
> ## Accessing the errno from kernel space.
> We should never access the errno from kernel space. This is especially dangerous from kernel space because we can't be certain which user address environment is in effect or which stack is being used (if we were to use separate kernel mode stacks in the future as Linux does ... for security reasons).
>
> Most OS interfaces have two parts:
>
> 1. A user callable function, say `osapi()` that sets the errno value is necessary (and may implement cancellation points) and
> 2. An internal OS version which might then be `nx_osapi()` which does not modify the errno value.
>
This design isn't perfect, most people don't understand the difference between osapi and nx_osapi. Even he/she know the difference but often forget in coding. I have fixed more than thousound of this type of bug(e.g. driver call osapi) and find that the same issue appear in the new code.
> Ideally, all of the user callable OS interfaces (the `osapi()`) should be moved to the location in libs/libc the system call (`sycall/`) should be redirected to `nx_osapi()`. A complication to doing this is that the OS interfaces which are also cancellation points call special internal interfaces, `enter_cancellation_point()` and `leave_cancellation_point()` that are not available from user space. So some addition partitioning would be need to accomplish this.
>
Ideally, we just need to:
1.Implement osapi which return error code instead modifing errno
Then the difference can be done by the tool automatically:
2.STUB_xxx can add cancel point for us
3.The PROXY_xxx can modify errno for us
For FLAT build, since we don't separate userspace and kernel space, some hack need to be taken:
1.Redefine osapi to nx_osapi when __KERNEL__ is defined
2.Rename osapi to nx_osapi by toolchain like simulator
Maybe we can find better method after more dissussing. But at least we definitely can find a way to do this thing automatically.
> ## Code References
> Things to look at:
>
> ```
> include/errno.h - Defines current errno access
> include/nuttx/tls.h - Defines the tls_info_s structure
> include/nuttx/sched.h - The errno and pthread-specific data
> sched/errno/* - The old errno access logic. Move to libs/libc/errno or tls
> libs/libc/tls - The new TLS and possible errno access logic
> sched/pthread - pthread_key*.c, pthread_*specific.c - Move to libs/libc/pthread
> ```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] xiaoxiang781216 commented on issue #986: Userspace errno
Posted by GitBox <gi...@apache.org>.
xiaoxiang781216 commented on issue #986:
URL: https://github.com/apache/incubator-nuttx/issues/986#issuecomment-625306304
> ## TLS interfaces
> TLS is a non-standard, but more general interface. It differs from pthread-specific data only in that is semantics are general; the semantics of the pthread-specific data interfaces are focused on pthreads. But they really should shared the same common underly logic.
But we already use pthread API across the code base which is only portable C API we can use for the thread creation. The tls_* exist just because we can't move phread TLS storage to the start of the stack because the old implmentation require the very huge stack alignment which isn't suitable for FLAT/PROTECTED build. Since we remove the alignment requirement from the new TLS implmentation, we can move all per thread data(include errno and pthread TLS) into the stack. I prefer that pthread TLS API is implemented directly on top of sched_get_stackinfo and remove all private tls_* API.
>
> Currently, there are two TLS interfaces:
>
> ```
> uintptr_t tls_get_element(int elem);
> void tls_set_element(int elem, uintptr_t value);
> ```
>
> I propose that we extend these TLS interfaces so that they can coexit with the pthread interfaces. Linux (actual GLIBC) does not provide a good mechanism; it relies on a storage class that is specific to ELF binaries. That is not useful in an embedded system where no ELF information is present.
>
> [This is a case of the bad decision to let specific tools environment drive a design. Good designs should be independent of tools].
>
> Instead, I think we should adapt from the Windows TLS interfaces which are directly anlagous to the POSIC pthread-specific data interfaces. Here is such an adaption (following NuttX coding standards. See https://en.wikipedia.org/wiki/Thread-local_storage#Windows_implementation):
>
> ```
> int tls_alloc(void);
> FAR void *tls_get_value(int tlsindex);
> bool tls_set_value(int tlsindex, uinptr_t tlsvalue);
> bool tls_free(int tlsindex);
> ```
>
> The Windows prototypes can be found from links on this page: https://docs.microsoft.com/en-us/windows/win32/api/processthreadsapi/nf-processthreadsapi-tlsalloc
>
Why we can't directly use pthread TLS API here? I don't see there is any benefit to define a new set of interface and forward pthread API to them, which just:
1.Increase the code size
2.Increase the document effort
3.Make people confusion which API he/she should use
4.The mix usage will spread in the code base and inroduce the inconsistence
> ## Accessing the errno from kernel space.
> We should never access the errno from kernel space. This is especially dangerous from kernel space because we can't be certain which user address environment is in effect or which stack is being used (if we were to use separate kernel mode stacks in the future as Linux does ... for security reasons).
>
> Most OS interfaces have two parts:
>
> 1. A user callable function, say `osapi()` that sets the errno value is necessary (and may implement cancellation points) and
> 2. An internal OS version which might then be `nx_osapi()` which does not modify the errno value.
>
This design isn't perfect, most people don't understand the difference between osapi and nx_osapi. Even he/she know the difference but often forget in coding. I have fixed more than thousound of this type of bug(e.g. driver call osapi) and find that the same issue appear in the new code.
> Ideally, all of the user callable OS interfaces (the `osapi()`) should be moved to the location in libs/libc the system call (`sycall/`) should be redirected to `nx_osapi()`. A complication to doing this is that the OS interfaces which are also cancellation points call special internal interfaces, `enter_cancellation_point()` and `leave_cancellation_point()` that are not available from user space. So some addition partitioning would be need to accomplish this.
>
Ideally, we just need to:
1.Implement osapi which return error code instead modifing errno
Then the difference can be done by the tool automatically:
2.STUB_xxx can add cancel point for us
3.The PROXY_xxx can modify errno for us
For FLAT build, since we don't separate userspace and kernel space, some hack need to be taken:
1.Redefine osapi to nx_osapi when __KERNEL__ is defined
2.Rename osapi to nx_osapi by toolchain like simulator
Maybe we can find better method after more dissussing. But at least we definitely can find a way to do this thing automatically.
> ## Code References
> Things to look at:
>
> ```
> include/errno.h - Defines current errno access
> include/nuttx/tls.h - Defines the tls_info_s structure
> include/nuttx/sched.h - The errno and pthread-specific data
> sched/errno/* - The old errno access logic. Move to libs/libc/errno or tls
> libs/libc/tls - The new TLS and possible errno access logic
> sched/pthread - pthread_key*.c, pthread_*specific.c - Move to libs/libc/pthread
> ```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] patacongo commented on issue #986: Userspace errno
Posted by GitBox <gi...@apache.org>.
patacongo commented on issue #986:
URL: https://github.com/apache/incubator-nuttx/issues/986#issuecomment-625278735
>
>
> > Linux (actual GDB) does not provide a good mechanism; it relies on a storage class that is specific to ELF binaries. That is not useful in an embedded system.
>
> This thing did bite me when I was trying to get Linux ELF running on Nuttx.
> That's a horrible implementation, relaying perfect cooperation of the toolchain and kernel, IMHO.
It is an easy trap to fall into in doing system software to exploit the tool environment that you work in as part of the implementation of the software. That is usually disastrous because it ties the implementation to the tools and that tool environment.
This is one of the reasons that in NuttX we forbid any tool-specific dependencies in the common OS code.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] patacongo commented on issue #986: Userspace errno
Posted by GitBox <gi...@apache.org>.
patacongo commented on issue #986:
URL: https://github.com/apache/incubator-nuttx/issues/986#issuecomment-625974429
Discussion moved to https://cwiki.apache.org/confluence/display/NUTTX/Userspace+errno+and+TLS
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] sonicyang commented on issue #986: Userspace errno
Posted by GitBox <gi...@apache.org>.
sonicyang commented on issue #986:
URL: https://github.com/apache/incubator-nuttx/issues/986#issuecomment-625205581
> Linux (actual GDB) does not provide a good mechanism; it relies on a storage class that is specific to ELF binaries. That is not useful in an embedded system.
This thing did bite me when I was trying to get Linux ELF running on Nuttx.
That's a horrible implementation, relaying perfect cooperation of the toolchain and kernel, IMHO.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] patacongo edited a comment on issue #986: Userspace errno
Posted by GitBox <gi...@apache.org>.
patacongo edited a comment on issue #986:
URL: https://github.com/apache/incubator-nuttx/issues/986#issuecomment-625278735
>
>
> > Linux (actual GDB) does not provide a good mechanism; it relies on a storage class that is specific to ELF binaries. That is not useful in an embedded system.
>
> This thing did bite me when I was trying to get Linux ELF running on Nuttx.
> That's a horrible implementation, relaying perfect cooperation of the toolchain and kernel, IMHO.
[Of course, I meant GLIBC not GDB]
It is an easy trap to fall into in doing system software to exploit the tool environment that you work in as part of the implementation of the software. That is usually disastrous because it ties the implementation to the tools and that tool environment.
This is one of the reasons that in NuttX we forbid any tool-specific dependencies in the common OS code.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] xiaoxiang781216 edited a comment on issue #986: Userspace errno
Posted by GitBox <gi...@apache.org>.
xiaoxiang781216 edited a comment on issue #986:
URL: https://github.com/apache/incubator-nuttx/issues/986#issuecomment-625306304
> ## TLS interfaces
> TLS is a non-standard, but more general interface. It differs from pthread-specific data only in that is semantics are general; the semantics of the pthread-specific data interfaces are focused on pthreads. But they really should shared the same common underly logic.
But we already use pthread API across the code base which is only portable C API we can use for the thread creation. I don't think there is any problem that TLS desgin bias on pthreads. The tls_* exist just because we can't move all per phread data to the start of the stack because the old implmentation require the very huge stack alignment which isn't suitable for FLAT/PROTECTED build. Since we remove the alignment requirement from the new TLS implmentation, we can move all per thread data(include errno and pthread TLS) into the stack. Then I prefer that pthread TLS API is implemented directly on top of sched_get_stackinfo and remove all private tls_* API.
>
> Currently, there are two TLS interfaces:
>
> ```
> uintptr_t tls_get_element(int elem);
> void tls_set_element(int elem, uintptr_t value);
> ```
>
> I propose that we extend these TLS interfaces so that they can coexit with the pthread interfaces. Linux (actual GLIBC) does not provide a good mechanism; it relies on a storage class that is specific to ELF binaries. That is not useful in an embedded system where no ELF information is present.
>
> [This is a case of the bad decision to let specific tools environment drive a design. Good designs should be independent of tools].
>
> Instead, I think we should adapt from the Windows TLS interfaces which are directly anlagous to the POSIC pthread-specific data interfaces. Here is such an adaption (following NuttX coding standards. See https://en.wikipedia.org/wiki/Thread-local_storage#Windows_implementation):
>
> ```
> int tls_alloc(void);
> FAR void *tls_get_value(int tlsindex);
> bool tls_set_value(int tlsindex, uinptr_t tlsvalue);
> bool tls_free(int tlsindex);
> ```
>
> The Windows prototypes can be found from links on this page: https://docs.microsoft.com/en-us/windows/win32/api/processthreadsapi/nf-processthreadsapi-tlsalloc
>
Why we can't directly use pthread TLS API here? I don't see there is any benefit to define a new set of interface and forward pthread API to them, which just:
1.Increase the code size
2.Increase the document effort
3.Make people confusion which API he/she should use
4.The mix usage will spread in the code base and inroduce the inconsistence
> ## Accessing the errno from kernel space.
> We should never access the errno from kernel space. This is especially dangerous from kernel space because we can't be certain which user address environment is in effect or which stack is being used (if we were to use separate kernel mode stacks in the future as Linux does ... for security reasons).
>
> Most OS interfaces have two parts:
>
> 1. A user callable function, say `osapi()` that sets the errno value is necessary (and may implement cancellation points) and
> 2. An internal OS version which might then be `nx_osapi()` which does not modify the errno value.
>
This design isn't perfect, most people don't understand the difference between osapi and nx_osapi. Even he/she know the difference but often forget in coding. I have fixed more than thousound of this type of bug(e.g. driver call osapi) and find that the same issue appear in the new code.
> Ideally, all of the user callable OS interfaces (the `osapi()`) should be moved to the location in libs/libc the system call (`sycall/`) should be redirected to `nx_osapi()`. A complication to doing this is that the OS interfaces which are also cancellation points call special internal interfaces, `enter_cancellation_point()` and `leave_cancellation_point()` that are not available from user space. So some addition partitioning would be need to accomplish this.
>
Ideally, we just need to:
1.Implement osapi which return error code instead modifing errno
Then the difference can be done by the tool automatically:
2.STUB_xxx can add cancel point for us
3.The PROXY_xxx can modify errno for us
For FLAT build, since we don't separate userspace and kernel space, some hack need to be taken:
1.Redefine osapi to nx_osapi when __KERNEL__ is defined
2.Rename osapi to nx_osapi by toolchain like simulator
Maybe we can find better method after more dissussing. But at least we definitely can find a way to do this thing automatically.
> ## Code References
> Things to look at:
>
> ```
> include/errno.h - Defines current errno access
> include/nuttx/tls.h - Defines the tls_info_s structure
> include/nuttx/sched.h - The errno and pthread-specific data
> sched/errno/* - The old errno access logic. Move to libs/libc/errno or tls
> libs/libc/tls - The new TLS and possible errno access logic
> sched/pthread - pthread_key*.c, pthread_*specific.c - Move to libs/libc/pthread
> ```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] patacongo commented on issue #986: Userspace errno
Posted by GitBox <gi...@apache.org>.
patacongo commented on issue #986:
URL: https://github.com/apache/incubator-nuttx/issues/986#issuecomment-625313012
@xiaoxiang781216 I have created an errno feature branch. Feel free to contribute to this branch via PRs.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org