You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@nuttx.apache.org by "MainframeReboot (via GitHub)" <gi...@apache.org> on 2024/05/15 20:52:57 UTC
[I] Kernel Build Apps [nuttx]
MainframeReboot opened a new issue, #12356:
URL: https://github.com/apache/nuttx/issues/12356
Hi,
I have NuttX starting up in kernel build but it fails to load the init ELF file and as such, nothing really happens:
I have read various documentation regarding building kernal apps which leads me down the road of performing a `make export` in the nuttx directory followed by a `make import` in the apps directory. This produces an apps/bin folder but the files contained in this folder are all over 4MB in size and the resulting `boot_romfsimg.h` file is over 40MB in size. Does this sound right? The resulting flat build outputs a binary that is <200kb in size so I am failing to understand why kernel build generates files this large.
Is there an elegant way to do this? Any documentation that explains the proper way to build and run the NSH library in kernal mode?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Kernel Build Apps [nuttx]
Posted by "patacongo (via GitHub)" <gi...@apache.org>.
patacongo commented on issue #12356:
URL: https://github.com/apache/nuttx/issues/12356#issuecomment-2115472462
> ... I am trying to understand if there's a way to compile the nsh library into the kernel or if I have to perform a `make export` and then a `make import` inside the `../apps` folder, followed by rebuilding NuttX with the generated romfs header file using the `mkromfsimg.sh` tool. ...
No, that is not possible. NSH users only user space OS interfaces. These cannot (or at least should) be used inside of the OS.
Also, each process lives inside a protected address space. If NSH were in the kernel, it could not interact with anything in user space and vice versa.
> If so, can you comment on why the size of the generated `boot_romfsimg.h` file is so large? Is there a way to minimize the size of this?
That is mostly because there are no shared libraries. As a consequence, a lot of C library code must be duplicated in each process. There is a lot of room for improvement for KERNEL build tools to make things cleaner. Wouldn't it be nice if there were a elf-nuttx- toolchain that could build efficient ELF modules as simply as GLIBC and GCC make build Linux processes?
You can reduce the amount of C library code in the module to reducing the size of the symbol table that draws the code into the link.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Kernel Build Apps [nuttx]
Posted by "patacongo (via GitHub)" <gi...@apache.org>.
patacongo commented on issue #12356:
URL: https://github.com/apache/nuttx/issues/12356#issuecomment-2138414347
> I sat down and RTFM to fully understand the differences between crt0, crt1, crti, crtbegin, crtn and crtend
I recall that crtbegin and crtend come from GCC, deriving from a file called crtstuff. I think these are essentially "libraries". I think they contain the configurable, Compiler specific parts for crt1 and crtn. crt1 and crtn are then provided by GLIBC for most architectures.
I looked into GLIBC and it does follow this clean breakdown for most architectures, but not all of them. I think we should not get too tied into either GCC or GLIBC
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Kernel Build Apps [nuttx]
Posted by "patacongo (via GitHub)" <gi...@apache.org>.
patacongo commented on issue #12356:
URL: https://github.com/apache/nuttx/issues/12356#issuecomment-2115441940
> Maybe @xiaoxiang781216 @patacongo @lupyuen other could help me to understand what you want to do.
To use the kernel builds, you need to have two blobs: (1) the kernel binary, and (2) the root file system. The root file system must contain an executable called "init" that is started when by the kernel during initialization. The root file system can be provided via external media such as SD card or in-memory as a ROMFS file system.
There are two separate builds. If you build a KERNEL build configuration, you get only the kernel blob. To build the ROMFS init file system you have to invent something else. There are some simple examples in the tree. There used to be a README.txt file describing how to do the ROMFS build, but it has been removed or, perhaps, moved to another location.
Yes, using the export files can be helpful.
Here is some discussion:
https://github.com/apache/nuttx/blob/nuttx-8.1/boards/arm/sama5/sama5d4-ek/README.txt#L4186
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Kernel Build Apps [nuttx]
Posted by "patacongo (via GitHub)" <gi...@apache.org>.
patacongo commented on issue #12356:
URL: https://github.com/apache/nuttx/issues/12356#issuecomment-2123545889
> I have noticed that the icicle knsh config has `CONFIG_ARCH_VMA_MAPPING` enabled as well as addresses set for `CONFIG_ARCH_SHM_VBASE` and `CONFIG_ARCH_KMAP_VBASE`, something I haven't seen in other knsh profiles.
Careful. There are several knsh or kostest configurations that are used in the PROTECTED build. That PROTECTED build is for MCUs that don't have MMUs, but rather MPUs and, hence, don't support virtual addressing. Make sure that you only look at configurations that have CONFIG_BUILD_KERNEL=y
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Kernel Build Apps [nuttx]
Posted by "MainframeReboot (via GitHub)" <gi...@apache.org>.
MainframeReboot commented on issue #12356:
URL: https://github.com/apache/nuttx/issues/12356#issuecomment-2125321267
Thank you all so much for your help. The recent suggestions pointed me down the right path and I can confirm that init now loads:
To get this to work, I took a look at the crt0.c source file mentioned by @patacongo as well as the linker scripts in the ../apps folder. I noticed that the entry point in the apps linker files does include crt0.o during linking but the entry point `__start` does not match the NuttX function in crt0.c. Within NuttX this function appears to have been changed to a single underscore which is why it could not be found.
I also took the advice from @lupyuen and looked at the thread local storage configuration. As mentioned in my previous reply this wasn't set at all, so I added `CONFIG_TLS_ALIGNED=y` as well as `CONFIG_TLS_LOG2_MAXSTACK=16` into my .config file. I also increased the stack size to 64kb by setting `CONFIG_POSIX_SPAWN_DEFAULT_STACKSIZE=65536` as per his instructions
@acassis, As I don't yet have a blog, I will document the other changes I had to make as the knsh defconfig for the mpfs icicle kit required additional modifications in order to work (not in chronological order):
- The default flow of the Icicle kit utilizes Hart-Software-Services (HSS) on the E54 to boot and monitor system applications. The HSS configures the PMP so to ensure the PMP was fully unlocked and not causing issues, I modified the `HSS_PMP_Init` function from the original to one that contains only two entries (credit goes to @pussuw for this suggestion):
``` C
csr_write(CSR_PMPADDR0, -1);
csr_write(CSR_PMPCFG0, (PMP_A_NAPOT | PMP_R | PMP_W | PMP_X));
```
- I noticed that `CONFIG_NUTTSBI` was set in the defconfig so payloads generated using the hss-payload-generator had the `skip-opensbi` flag set to true. More on the HSS file used below.
- Out of the box, `CONFIG_RAM_VSTART` is set to 0x00. This caused issued inside of `riscv_pgvaddr()` during system initialization as the computed addresses would subtract `CONFIG_RAM_START` from the passed in address and add `CONFIG_RAM_VSTART` which was 0x00. This resulted in a hard fault as addresses outside of memory were being indexed.
- Address environments were enabled but only `CONFIG_ARCH_DATA_VBASE` was set and the NPAGES was set to 0 for all. I modified this as follows:
```
CONFIG_ARCH_TEXT_VBASE=0xC0000000
CONFIG_ARCH_DATA_VBASE=0xC0400000
CONFIG_ARCH_HEAP_VBASE=0xC0800000
CONFIG_ARCH_TEXT_NPAGES=128
CONFIG_ARCH_DATA_NPAGES=128
CONFIG_ARCH_HEAP_NPAGES=128
```
- Enabled Thread Local Storage by appending the following to the config file:
```
CONFIG_TLS_ALIGNED=y
CONFIG_TLS_LOG2_MAXSTACK=16
CONFIG_TLS_NELEM=0
```
- Next, I had to build the apps by first building NuttX, exporting it, and then importing it into the ../apps directory so I could build the apps and generate a `boot_romfsimg.h` header file. I followed the steps from this article to accomplish this: https://cwiki.apache.org/confluence/display/NUTTX/APPNAME+vs.+PROGNAME
- I was getting strange warnings regarding `__start` not being found during linking which caused problems when running the apps so as per the suggestions from @patacongo, I took a look at `crt0.c` as well the make files in the /apps folder. Here I noticed that the /apps makefiles set the entry point as `__start` where as the actual function is named `_start` within `crt0.c`. I modified the apps linker script to remove the extra underscore which resolved the linker warnings.
- I noticed the output was very large so I followed the article linked by @lupyuen to switch from relocatable ELF files to fully linked ELF files: https://github.com/lupyuen/quickjs-nuttx#full-linking-for-nuttx-apps
- The files were still too large for my liking, so I used objcopy to strip the ELF files the `make import` generates in /apps/bin:
``` Bash
riscv-none-elf-objcopy --strip-unneeded init
riscv-none-elf-objcopy --strip-unneeded hello
riscv-none-elf-objcopy --strip-unneeded helloxx
riscv-none-elf-objcopy --strip-unneeded sh
```
- After stripping the ELF files, I generated a new `boot_romfsimg.h` header file that was much smaller and copied it over to `nuttx/boards/risc-v/mpfs/icicle/include` and rebuilt NuttX.
- Generated a payload using the `hss-payload-generator` using a modified yaml file to ensure the Kernel was running in Supervisor mode and that opensbi was skipped (this is different from the yaml that comes with NuttX and works for flat builds):
``` YAML
# HSS Payload Generator
# First, we can optionally set a name for our image, otherwise one will be created dynamically
set-name: 'PolarFire-SoC-HSS::nuttxbsp_kernel_payload'
# Next, we'll define the entry point addresses for each hart, as follows:
hart-entry-points: {u54_1: '0x80000000', u54_2: '0x80000000', u54_3: '0x80000000', u54_4: '0x80000000>
#
payloads:
nuttxbsp.bin: {exec-addr: '0x80000000', owner-hart: u54_1, priv-mode: prv_s, skip-opensbi: true}
```
After all of that, NuttX kernel build runs on the PolarFire Icicle kit. Although I am running into an issue getting `helloxx` to run. The app `hello` works but I get a segmentation fault when running `helloxx`:
I noticed `crt0.c` has preprocessor definitions for CXX so I will look into this as well other C++ related configuration options as I might be missing something on that front.
Thank you again @patacongo, @lupyuen, @acassis and @pussuw for your support on getting this to work.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Kernel Build Apps [nuttx]
Posted by "patacongo (via GitHub)" <gi...@apache.org>.
patacongo commented on issue #12356:
URL: https://github.com/apache/nuttx/issues/12356#issuecomment-2138401325
> @patacongo @MainframeReboot should this arch/risc-v/src/common/crt1.c be submitted to inclusion into nuttx mainline?
I would think so. On one hand it should be functionally equivalent, but this is the cannonically correct way to organize the logic and will also enable us to support the dynamic loader, ldso, and shared libraries in the future. @xiaoxiang781216 what do you think?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Kernel Build Apps [nuttx]
Posted by "MainframeReboot (via GitHub)" <gi...@apache.org>.
MainframeReboot commented on issue #12356:
URL: https://github.com/apache/nuttx/issues/12356#issuecomment-2139711102
> > I sat down and RTFM to fully understand the differences between crt0, crt1, crti, crtbegin, crtn and crtend
>
> I recall that crtbegin and crtend come from GCC, deriving from a file called crtstuff. I think these are essentially "libraries". I think they contain the configurable, Compiler specific parts for crt1 and crtn. crt1 and crtn are then provided by GLIBC for most architectures.
>
> I looked into GLIBC and it does follow this clean breakdown for most architectures, but not all of them. I think we should not get too tied into either GCC or GLIBC
Fair enough.
I do want to reiterate that the switch from all logic in crt0.c to a crt0.c and crt1.c approach didn't solve my issue alone. I had to modify how the apps were compiled and linked or else it either broke C applications, C++ applications or sometimes both. With that said, I understand this might only be relevant to my architecture and toolchain combination so pushing the modifications out to common makefiles is most likely not desirable. I'm interested to hear ideas on how this can be properly implemented to support all architectures.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Kernel Build Apps [nuttx]
Posted by "patacongo (via GitHub)" <gi...@apache.org>.
patacongo commented on issue #12356:
URL: https://github.com/apache/nuttx/issues/12356#issuecomment-2125864651
> However, that alone did not solve the issue. I had to remove the calls `exec_ctores()` and `atexit(exec_dtors)` from `crt0.c` in order to not crash on ELF load.
exec_ctors() is really simple, but it depends on a table of constructor addresses created by the build logic. These are defined by binfmt/libelf/gnu-elf.ld or similar:
```
/* Linker defined symbols to .ctors and .dtors */
extern initializer_t _sctors[];
extern initializer_t _ectors[];
extern initializer_t _sdtors[];
extern initializer_t _edtors[];
```
Check whatever linker script you use to build the elf modules. Check that those symbols exist in the module.
These define a table of constructor destructor address. Constructors start at _sctors and end at _ectors. Make sure that there are constructors in the table.
Issues with atexit() may be like those of #1263. I don't know the state of that.
> Regardless, I can live with NuttX kernel build that has no C++ static constructor support given everything else appears to be working flawlessly.
But you shouldn't have to!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Kernel Build Apps [nuttx]
Posted by "lupyuen (via GitHub)" <gi...@apache.org>.
lupyuen commented on issue #12356:
URL: https://github.com/apache/nuttx/issues/12356#issuecomment-2123705471
Hi @MainframeReboot: The RISC-V Exception looks interesting:
```yaml
EXCEPTION: Instruction page fault.
MCAUSE: 0x0C
EPC: 0x00
MTVAL: 0x00
Segmentation fault in PID 4: /bin/init
```
EPC says that the NSH Shell is trying to execute the code at Address 0 and failing? This is very odd. It's possible that the Stack Size is too small (8KB):
```yaml
binfmt_dumpmodule:
stacksize: 8192
```
Could you increase the Stack Size to 64KB? ("Default task_spawn Stack Size" and "Thread Local Storage")
https://lupyuen.github.io/articles/quickjs#nuttx-stack-is-full-of-quickjs
It's also possible that Full Linking of NSH Shell messed up the code addresses.
I wonder why NuttX doesn't show the full Crash Dump, it might show us the code that's trying to execute Address 0. Hmmm...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Kernel Build Apps [nuttx]
Posted by "MainframeReboot (via GitHub)" <gi...@apache.org>.
MainframeReboot commented on issue #12356:
URL: https://github.com/apache/nuttx/issues/12356#issuecomment-2116224512
Hi All,
Thank you for your replies and the direction, I've definitely learned a lot.
I have gone down the road of exporting NuttX, importing it into the `../apps` folder, making the apps and then exporting the romfs header. I then rebuild NuttX with this romfs header.
I can confirm that from the debug messages that the file system is being mounted, no issues here. However, I do seem to run into problems whenit attempts to load the program `/bin/init`. The issues appear to occur at `up_relocateadd` with the error `up_relocateadd: ERROR: PCREL_HI20 at c0000022 bad:ffffffff40001000` then followed up with `elf_relocateadd: ERROR: Section 2 reloc 0: Relocation failed: -22`.
I'm not quite sure what this means other than my address environments are incorrectly set?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Kernel Build Apps [nuttx]
Posted by "acassis (via GitHub)" <gi...@apache.org>.
acassis commented on issue #12356:
URL: https://github.com/apache/nuttx/issues/12356#issuecomment-2125710344
Congratulations @MainframeReboot !!! I'm glad to know you got it working.
Since you don't have a blog, I have a better suggestion:
Please submit it a guide "Running NuttX in kernel mode (with MMU support) on Microchip PolarFire Icicle board". So it could be included here: https://nuttx.apache.org/docs/latest/guides/index.html
Just create it inside nuttx/Documentation/guides/ so it will be available as reference for everybody using this board.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Kernel Build Apps [nuttx]
Posted by "MainframeReboot (via GitHub)" <gi...@apache.org>.
MainframeReboot commented on issue #12356:
URL: https://github.com/apache/nuttx/issues/12356#issuecomment-2115287106
> Hi @MainframeReboot please enable Debug Scheduler Info and Debug FileSystem Info, it will report more info showing why it is failing. Look at sched/init/nx_bringup.c line 379, probably it is failing here:
>
> ```
> ret = exec_spawn(CONFIG_INIT_FILEPATH, argv, NULL,
> CONFIG_INIT_SYMTAB, CONFIG_INIT_NEXPORTS, NULL, &attr);
> ```
Yes, this is the line it's failing at.
I understand this is due to the file system needing to be mounted but I am trying to understand if there's a way to compile the nsh library into the kernel or if I have to perform a `make export` and then a `make import` inside the `../apps` folder, followed by rebuilding NuttX with the generated romfs header file using the `mkromfsimg.sh` tool. If so, can you comment on why the size of the generated `boot_romfsimg.h` file is so large? Is there a way to minimize the size of this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Kernel Build Apps [nuttx]
Posted by "acassis (via GitHub)" <gi...@apache.org>.
acassis commented on issue #12356:
URL: https://github.com/apache/nuttx/issues/12356#issuecomment-2115355484
I don't know if I understood correctly what you want to do and what you are doing, but let to highlight some points: in kernel build mode NuttX is very similar to Linux, it means it will need to mount the userspace to execute the "init" application.
I think you only need to use "make export" when you are building external ELFs, to be loaded later on, when the system is already running. For example to create a ELF program to be loaded from SDCard or USB Thumb Drive.
Maybe @xiaoxiang781216 @patacongo @lupyuen other could help me to understand what you want to do.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Kernel Build Apps [nuttx]
Posted by "MainframeReboot (via GitHub)" <gi...@apache.org>.
MainframeReboot commented on issue #12356:
URL: https://github.com/apache/nuttx/issues/12356#issuecomment-2123800666
> Hi @MainframeReboot: The RISC-V Exception looks interesting:
>
> ```yaml
> EXCEPTION: Instruction page fault.
> MCAUSE: 0x0C
> EPC: 0x00
> MTVAL: 0x00
> Segmentation fault in PID 4: /bin/init
> ```
>
> EPC says that the NSH Shell is trying to execute the code at Address 0 and failing? This is very odd. It's possible that the Stack Size is too small (8KB):
>
> ```yaml
> binfmt_dumpmodule:
> stacksize: 8192
> ```
>
> Could you increase the Stack Size to 64KB? ("Default task_spawn Stack Size" and "Thread Local Storage")
>
> https://lupyuen.github.io/articles/quickjs#nuttx-stack-is-full-of-quickjs
>
> It's also possible that Full Linking of NSH Shell messed up the code addresses.
>
> I wonder why NuttX doesn't show the full Crash Dump, it might show us the code that's trying to execute Address 0. Hmmm...
Thanks for the suggestion, I will give this a try and let you know the result. One thing I am noticing right away is that `CONFIG_TLS_ALIGNED` is not enabled at all in this config so `CONFIG_TLS_LOG2_MAXSTACK` also does not exist. I will enable this and retry before attempting the other increases.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Kernel Build Apps [nuttx]
Posted by "acassis (via GitHub)" <gi...@apache.org>.
acassis commented on issue #12356:
URL: https://github.com/apache/nuttx/issues/12356#issuecomment-2116319053
Hi @MainframeReboot I'm happy to know that you are evolving with your testings! (Suggestion: if you have a blog, create a post documenting the path you are following, it could help more people in the future, even your future you).
This error message is coming from here:
```
if (!_valid_hi20_imm(imm_hi))
{
berr("ERROR: %s at %08" PRIxPTR " bad:%08lx\n",
_get_rname(relotype), addr, imm_hi << 12);
return -EINVAL;
}
```
I don't know much about RV64 arch, but I think you are putting the userspace init ELF at wrong position, maybe the function explanation rings a bell:
```
* Name: _valid_hi20_imm
*
* Description:
* Check that any XX_HI20 relocation has a valid upper 20-bit immediate.
* Note that this test is not necessary for RV32 targets, the problem is
* related to RV64 sign extension.
*
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Kernel Build Apps [nuttx]
Posted by "acassis (via GitHub)" <gi...@apache.org>.
acassis commented on issue #12356:
URL: https://github.com/apache/nuttx/issues/12356#issuecomment-2113602706
Hi @MainframeReboot please enable Debug Scheduler Info and Debug FileSystem Info, it will report more info showing why it is failing. Look at sched/init/nx_bringup.c line 379, probably it is failing here:
```
ret = exec_spawn(CONFIG_INIT_FILEPATH, argv, NULL,
CONFIG_INIT_SYMTAB, CONFIG_INIT_NEXPORTS, NULL, &attr);
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Kernel Build Apps [nuttx]
Posted by "pussuw (via GitHub)" <gi...@apache.org>.
pussuw commented on issue #12356:
URL: https://github.com/apache/nuttx/issues/12356#issuecomment-2125908203
I fixed #1263 a while back (years back), so that should not be an issue here. However CONFIG_BINFMT_CONSTRUCTORS can cause such issues.
I use C++ almost exclusively with BUILD_KERNEL, it works just fine. There is something odd about your build environment still, hope you can find it!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Kernel Build Apps [nuttx]
Posted by "acassis (via GitHub)" <gi...@apache.org>.
acassis commented on issue #12356:
URL: https://github.com/apache/nuttx/issues/12356#issuecomment-2123527949
Nice findings @MainframeReboot ! Seems like finally the code is loading, but an exception Instruction page fault (MCAUSE = 0x0c) is happening. Maybe @lupyuen has some tips to track this issue because I saw it in his PureScript dump parser.
I noted that you are doing the board initialization in the AppBringUp thread, did you try avoid it? Disabling CONFIG_BOARD_LATE_INITIALIZE ? I don't think it will solve the root cause, but at least can have some different effect.
Regarding the blog, I suggest you write down each logical decision and testing you are doing, this will help you later when you write the post. Remember: when you arrive at your destination you won't remember all the trees you saw along the way.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Kernel Build Apps [nuttx]
Posted by "MainframeReboot (via GitHub)" <gi...@apache.org>.
MainframeReboot commented on issue #12356:
URL: https://github.com/apache/nuttx/issues/12356#issuecomment-2127711351
> > However, that alone did not solve the issue. I had to remove the calls `exec_ctores()` and `atexit(exec_dtors)` from `crt0.c` in order to not crash on ELF load.
>
> exec_ctors() is really simple, but it depends on a table of constructor addresses created by the build logic. These are defined by binfmt/libelf/gnu-elf.ld or similar:
>
> ```
> /* Linker defined symbols to .ctors and .dtors */
>
> extern initializer_t _sctors[];
> extern initializer_t _ectors[];
> extern initializer_t _sdtors[];
> extern initializer_t _edtors[];
> ```
>
> Check whatever linker script you use to build the elf modules. Check that those symbols exist in the module.
>
> These define a table of constructor and destructor address. Constructors start at _sctors and end at _ectors. Make sure that there are constructors in the table.
>
> Issues with atexit() may be like those of #1263. I don't know the state of that issue. The function called by atexit() calls the destructors. So more likely that is the same issue as with the list of destructors.
>
> > Regardless, I can live with NuttX kernel build that has no C++ static constructor support given everything else appears to be working flawlessly.
>
> But you shouldn't have to!
I decided not to settle and poked around more to see if I can get static constructor support to work but no luck so far.
I have taken a look and `CONFIG_BINFMT_CONSTRUCTORS` is not set in my config anymore.
The segmentation fault appears to happen inside the function `exec_ctors()` the moment `(*ctor)()` is executed. This is called inside `__start()` that is provided by `crt0.o`.
I dumped helloxx and examined the symbol table to make sure .ctors was there and it is:
My linker script is based on the linker script from the Fully Linking for NuttX apps article by @lupyuen and looks as follows:
```
SECTIONS
{
. = 0xC0000000;
.text :
{
_stext = . ;
*(.text)
*(.text.*)
*(.gnu.warning)
*(.stub)
*(.glue_7)
*(.glue_7t)
*(.jcr)
/* C++ support: The .init and .fini sections contain specific logic
* to manage static constructors and destructors.
*/
*(.gnu.linkonce.t.*)
KEEP(*(.init)) /* Old ABI */
KEEP(*(.fini)) /* Old ABI */
_etext = . ;
}
.rodata :
{
_srodata = . ;
*(.rodata)
*(.rodata1)
*(.rodata.*)
*(.gnu.linkonce.r*)
_erodata = . ;
}
. = 0xC0400000;
.data :
{
_sdata = . ;
*(.data)
*(.data1)
*(.data.*)
*(.gnu.linkonce.d*)
. = ALIGN(4);
_edata = . ;
}
/* C++ support. For each global and static local C++ object,
* GCC creates a small subroutine to construct the object. Pointers
* to these routines (not the routines themselves) are stored as
* simple, linear arrays in the .ctors section of the object file.
* Similarly, pointers to global/static destructor routines are
* stored in .dtors.
*/
.ctors :
{
_sctors = . ;
KEEP(*(SORT_BY_INIT_PRIORITY(.init_array.*) SORT_BY_INIT_PRIORITY(.ctors.*)))
KEEP(*(.init_array .ctors))
_ectors = . ;
}
.dtors :
{
_sdtors = . ;
KEEP (*(.dtors)) /* Old ABI: Unallocated */
KEEP (*(.fini_array)) /* New ABI: Allocated */
KEEP (*(SORT(.fini_array.*)))
_edtors = . ;
}
.bss :
{
_sbss = . ;
*(.bss)
*(.bss.*)
*(.sbss)
*(.sbss.*)
*(.gnu.linkonce.b*)
*(COMMON)
_ebss = . ;
}
/* Thread local storage support */
.tdata : {
_stdata = ABSOLUTE(.);
KEEP (*(.tdata .tdata.* .gnu.linkonce.td.*));
_etdata = ABSOLUTE(.);
}
.tbss : {
_stbss = ABSOLUTE(.);
KEEP (*(.tbss .tbss.* .gnu.linkonce.tb.* .tcommon));
_etbss = ABSOLUTE(.);
}
/* Stabs debugging sections. */
.stab 0 : { *(.stab) }
.stabstr 0 : { *(.stabstr) }
.stab.excl 0 : { *(.stab.excl) }
.stab.exclstr 0 : { *(.stab.exclstr) }
.stab.index 0 : { *(.stab.index) }
.stab.indexstr 0 : { *(.stab.indexstr) }
.comment 0 : { *(.comment) }
.debug_abbrev 0 : { *(.debug_abbrev) }
.debug_info 0 : { *(.debug_info) }
.debug_line 0 : { *(.debug_line) }
.debug_pubnames 0 : { *(.debug_pubnames) }
.debug_aranges 0 : { *(.debug_aranges) }
}
```
I can confirm the initial address `ctor` points to is `0xC0400020` which matches the symbol table so I'm not quite sure what I'm missing now...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Kernel Build Apps [nuttx]
Posted by "patacongo (via GitHub)" <gi...@apache.org>.
patacongo commented on issue #12356:
URL: https://github.com/apache/nuttx/issues/12356#issuecomment-2125637674
> I have seen places where the ctor/dtor code is executed in binfmt
Constructors and destructors should run in crti and crtN, respectively, for general Unix compatibility. crti should support __init and crtN should supporrt __fini.
Lots more detail if you care about this: https://gcc.gnu.org/onlinedocs/gccint/Initialization.html
As the OP mentioned, this is currently done with conditional logic in crt0. That is technically okay but will likely confuse people who are used to thinking about things in the GCC/GLIBC way.
There is a long-open issue #1265 (and related #1263). Perhaps this is only a problem in PROTECTED mode which as a single set of ctors and dtors for the whole user-space blob. KERNEL mode uses only loadable ELF modules, each with their own ctors and dtors.
crt0.c exists only for armv7-a, arm64, and riscv-5 all of which can support the KERNEL build.
So crt0 with ELF modules should never be used in PROTECTED mode right now or else you would get doubly constructed static classes.
> I have seen places where the ctor/dtor code is executed in binfmt (there is a kconfig that controls this I believe, don't remember the name though)
That is CONFIG_BINFMT_CONSTRUCTORS. I think that should be disabled in KERNEL mode to let the ELF module call its own constructors and destructors in the correct context.
This seems awkward and prone to errors to me.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Kernel Build Apps [nuttx]
Posted by "MainframeReboot (via GitHub)" <gi...@apache.org>.
MainframeReboot commented on issue #12356:
URL: https://github.com/apache/nuttx/issues/12356#issuecomment-2123794150
> > EPC says that the NSH Shell is trying to execute the code at Address 0 and failing? This is very odd.
>
> Is zero a valid address in this architecture? In ARM architectures, reset vectors are configurable but often are at virtual address zero and must never be re-mapped.
>
> I suspect this exception occurs immediately on startup of /bin/init. Is zero the correct process startup address in the link? The startup address should be in the ELF header. Is the MMU configured properly to execute from address zero?
>
> Sorry... more questions than answers.
Great questions, they're very helpful in pointing me in directions I wouldn't have looked otherwise.
I took a peak at the output while building the apps and noticed that the linker prints out: `warning: cannot find entry symbol __start; defaulting to 0x00000000C0000000` for every single app that is built. Dumping the init ELF shows that __start is listed as `*UND*` and set to 0x00000000 in the symbol table. The ELF header does list the entry point address as 0xC0000000:
I will look into the MMU configuration next but I'm confused why it's attempting to run from address 0 if the entry point in the header is listed as 0xC0000000.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Kernel Build Apps [nuttx]
Posted by "MainframeReboot (via GitHub)" <gi...@apache.org>.
MainframeReboot commented on issue #12356:
URL: https://github.com/apache/nuttx/issues/12356#issuecomment-2123431521
Thanks for your reply @acassis. I can definitely look into a blog post of my findings once I get this to work!
I took a look, and I am not sure where the init ELF is supposed to go. I built the apps, generated a `boot_romfsimg.h` header file and then recompiled NuttX using the header. I have since tried another approach to try and remove the relocation part out of the equation.
I followed the link that @lupyuen posted [above](https://github.com/lupyuen/quickjs-nuttx#full-linking-for-nuttx-apps) and have been successful in generating fully linked applications that are significantly smaller in size.
I should mention that before I performed the full link of the apps I did some reach into the address environments and the MMU. While doing this I stumbled upon another article written by @lupyuen covering the MMU: [https://lupyuen.codeberg.page/articles/mmu.html](https://lupyuen.codeberg.page/articles/mmu.html).
Using what I learned in this article, I set my address environment as follows:
- `CONFIG_ARCH_TEXT_VBASE=0xC0000000`
- `CONFIG_ARCH_DATA_VBASE=0xC0400000`
- `CONFIG_ARCH_HEAP_VBASE=0xC0800000`
Then I configured the `gnu-elf.ld` script to match these values before I built the apps. This enabled the `init` module to be loaded but ultimately resulted in a segmentation fault:
I am still attempting to get the default icicle knsh config to run with minor tweaks based on articles I read as well as other RISC-V configs. I have noticed that the icicle knsh config has `CONFIG_ARCH_VMA_MAPPING` enabled as well as addresses set for `CONFIG_ARCH_SHM_VBASE` and `CONFIG_ARCH_KMAP_VBASE`, something I haven't seen in other knsh profiles. I will look more into this next.
In the meantime, if anyone has another other suggestions I could try I would greatly appreciate it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Kernel Build Apps [nuttx]
Posted by "patacongo (via GitHub)" <gi...@apache.org>.
patacongo commented on issue #12356:
URL: https://github.com/apache/nuttx/issues/12356#issuecomment-2123714446
> EPC says that the NSH Shell is trying to execute the code at Address 0 and failing? This is very odd.
Is zero a valid address in this architecture? In ARM architectures, reset vectors are at virtual address zero and must never be re-mapped.
I suspect this exception occurs immediately on startup of /bin/init. Is zero the correct process startup address in the link? The startup address should be in the ELF header. Is the MMU configured properly to execute from address zero?
Sorry... more questions than answers.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Kernel Build Apps [nuttx]
Posted by "acassis (via GitHub)" <gi...@apache.org>.
acassis commented on issue #12356:
URL: https://github.com/apache/nuttx/issues/12356#issuecomment-2138386100
@patacongo @MainframeReboot should this arch/risc-v/src/common/crt1.c be submitted to inclusion into nuttx mainline?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Kernel Build Apps [nuttx]
Posted by "MainframeReboot (via GitHub)" <gi...@apache.org>.
MainframeReboot commented on issue #12356:
URL: https://github.com/apache/nuttx/issues/12356#issuecomment-2138234172
@patacongo
I sat down and RTFM to fully understand the differences between crt0, crt1, crti, crtbegin, crtn and crtend. This has helped me immensely and I have managed to get static global instances to run on NuttX kernel by making the necessary changes to NuttX.
I will explain my patches below but the executive summary is _use crt0 for C, use crt1 for C++_ while also manually linking crti, crtbegin, crtn and crtend for C++ applications only.
### crt0.c
The first modification was to remove C++ logic from crt0. This means that all of the .ctors/.dtors arrays at the top (_sctors, sdtors etc.) are removed, along with the the calls to exec_ctors and atexit(exec_dtors). The result is a crt0.c file that looks like this (some comments stripped for size):
``` C
/****************************************************************************
* arch/risc-v/src/common/crt0.c
****************************************************************************/
#include <nuttx/arch.h>
#include <nuttx/addrenv.h>
#include <nuttx/compiler.h>
#include <nuttx/config.h>
#include <sys/types.h>
#include <syscall.h>
#include <stdlib.h>
#include <stdio.h>
#include "riscv_internal.h"
#ifdef CONFIG_BUILD_KERNEL
int main(int argc, char *argv[]);
static void sig_trampoline(void) naked_function;
static void sig_trampoline(void)
{
__asm__ __volatile__
(
" addi sp, sp, -" STACK_FRAME_SIZE "\n" /* Save ra on the stack */
REGSTORE " ra, 0(sp)\n"
" mv t0, a0\n" /* t0=sighand */
" mv a0, a1\n" /* a0=signo */
" mv a1, a2\n" /* a1=info */
" mv a2, a3\n" /* a2=ucontext */
" jalr t0\n" /* Call the signal handler (modifies ra) */
REGLOAD " ra, 0(sp)\n" /* Recover ra in sp */
" addi sp, sp, " STACK_FRAME_SIZE "\n"
" li a0, %0\n" /* SYS_signal_handler_return */
" ecall\n" /* Return from the SYSCALL */
" nop\n"
:
: "i" (SYS_signal_handler_return)
:
);
}
void __start(int argc, char *argv[])
{
int ret;
/* Initialize the reserved area at the beginning of the .bss/.data region
* that is visible to the RTOS.
*/
ARCH_DATA_RESERVE->ar_sigtramp = (addrenv_sigtramp_t)sig_trampoline;
/******************************************************************
* Do NOT include C++ constructor/destructor calls in this file.
* This file is for C applications only. Refer to crt1.c for C++.
*******************************************************************/
/* Call the main() entry point passing argc and argv. */
ret = main(argc, argv);
/* Call exit() if/when the main() returns */
exit(ret);
}
#endif /* CONFIG_BUILD_KERNEL */
```
### crt1.c
All C++ constructor/destructor logic was moved to the new file crt1.c. Note that this is dfference from the original C++ logic in crt0.c. I have learned, during my research, that .ctors and .dtors is the "old" way of doing things and that the new recommended way of doing things is to use .preinit_array, .init_array and .fini_array. The RISC-V toolchain that I use (xPack 13.2.0) supports the new way so I have decided to follow the new convention. From my reading of old RISC-V NuttX issues, it appears the NuttX team has moved on to xPack 13.2.0 for testing so it _should_ work with other RISC-V boards as well. Here is the new crt1.c file:
``` C
/****************************************************************************
* arch/risc-v/src/common/crt1.c
****************************************************************************/
#include <nuttx/arch.h>
#include <nuttx/addrenv.h>
#include <nuttx/compiler.h>
#include <nuttx/config.h>
#include <sys/types.h>
#include <syscall.h>
#include <stdlib.h>
#include "riscv_internal.h"
#ifdef CONFIG_BUILD_KERNEL
int main(int argc, char *argv[]);
static void sig_trampoline(void) naked_function;
static void sig_trampoline(void)
{
__asm__ __volatile__
(
" addi sp, sp, -" STACK_FRAME_SIZE "\n" /* Save ra on the stack */
REGSTORE " ra, 0(sp)\n"
" mv t0, a0\n" /* t0=sighand */
" mv a0, a1\n" /* a0=signo */
" mv a1, a2\n" /* a1=info */
" mv a2, a3\n" /* a2=ucontext */
" jalr t0\n" /* Call the signal handler (modifies ra) */
REGLOAD " ra, 0(sp)\n" /* Recover ra in sp */
" addi sp, sp, " STACK_FRAME_SIZE "\n"
" li a0, %0\n" /* SYS_signal_handler_return */
" ecall\n" /* Return from the SYSCALL */
" nop\n"
:
: "i" (SYS_signal_handler_return)
:
);
}
/****************************************************************************
* Public Data
****************************************************************************/
/*
Linker defined symbols to .preinit_array, .init_array and .fini_array.
.ctors and .dtors are not used by RISC-V.
*/
extern initializer_t __preinit_array_start[];
extern initializer_t __preinit_array_end[];
extern initializer_t __init_array_start[];
extern initializer_t __init_array_end[];
extern initializer_t __fini_array_start[];
extern initializer_t __fini_array_end[];
/****************************************************************************
* Private Functions
****************************************************************************/
#ifdef CONFIG_HAVE_CXX
/****************************************************************************
* Name: exec_preinit
*
* Description:
* Calls startup functions prior to main entry point
*
****************************************************************************/
static void exec_preinit(void)
{
initializer_t *preinit;
for(preinit = __preinit_array_start; preinit < __preinit_array_end; ++preinit)
{
initializer_t initializer = *preinit;
if (initializer)
{
initializer();
}
}
}
/****************************************************************************
* Name: exec_preinit
*
* Description:
* Calls static constructors prior to main entry point
*
****************************************************************************/
static void exec_init(void)
{
initializer_t *init;
for(init = __init_array_start; init < __init_array_end; ++init)
{
initializer_t initializer = *init;
if (initializer)
{
initializer();
}
}
}
/****************************************************************************
* Name: exec_fini
*
* Description:
* Calls static destructors using atexit
*
****************************************************************************/
static void exec_fini(void)
{
initializer_t *fini;
for(fini = __fini_array_start; fini < __fini_array_end; ++fini)
{
initializer_t initializer = *fini;
if (initializer)
{
initializer();
}
}
}
#endif
void __start(int argc, char *argv[])
{
int ret;
/* Initialize the reserved area at the beginning of the .bss/.data region
* that is visible to the RTOS.
*/
ARCH_DATA_RESERVE->ar_sigtramp = (addrenv_sigtramp_t)sig_trampoline;
#ifdef CONFIG_HAVE_CXX
/* Call preinit functions */
exec_preinit();
/* Call C++ constructors */
exec_init();
/* Setup so that C++ destructors called on task exit */
atexit(exec_fini);
#endif
/* Call the main() entry point passing argc and argv. */
ret = main(argc, argv);
/* Call exit() if/when the main() returns */
exit(ret);
}
#endif /* CONFIG_BUILD_KERNEL */
```
### gnu-elf.ld
In order to make sure the arrays in the crt1.c file are populated, the linker script was updated to match:
```
/****************************************************************************
* boards/risc-v/mpfs/icicle/scripts/gnu-elf.ld
****************************************************************************/
SECTIONS
{
.text 0xC0000000 :
{
_stext = . ;
*(.text)
*(.text.*)
*(.gnu.warning)
*(.stub)
*(.glue_7)
*(.glue_7t)
*(.jcr)
/* C++ support: The .init and .fini sections contain specific logic
* to manage static constructors and destructors.
*/
*(.gnu.linkonce.t.*)
*(.init) /* Old ABI */
*(.fini) /* Old ABI */
_etext = . ;
}
.rodata :
{
_srodata = . ;
*(.rodata)
*(.rodata1)
*(.rodata.*)
*(.gnu.linkonce.r*)
_erodata = . ;
}
.data 0xC0101000:
{
_sdata = . ;
*(.data)
*(.data1)
*(.data.*)
*(.gnu.linkonce.d*)
. = ALIGN(4);
_edata = . ;
}
/* C++ support. For each global and static local C++ object,
* GCC creates a small subroutine to construct the object. Pointers
* to these routines (not the routines themselves) are stored as
* simple, linear arrays in the .ctors section of the object file.
* Similarly, pointers to global/static destructor routines are
* stored in .dtors.
*/
.preinit_array :
{
PROVIDE(__preinit_array_start = .);
KEEP(*(.preinit_array*))
PROVIDE(__preinit_array_end = .);
}
.init_array :
{
PROVIDE(__init_array_start = .);
KEEP(*(SORT_BY_INIT_PRIORITY(.init_array.*) SORT_BY_INIT_PRIORITY(.ctors.*)))
KEEP(*(.init_array .ctors))
PROVIDE(__init_array_end = .);
}
.fini_array :
{
PROVIDE(__fini_array_start = .);
KEEP(*(SORT_BY_INIT_PRIORITY(.fini_array.*) SORT_BY_INIT_PRIORITY(.dtors.*)))
KEEP(*(.fini_array .dtors))
PROVIDE(__fini_array_end = .);
}
/* Thread local storage support */
.tdata :
{
_stdata = ABSOLUTE(.);
KEEP (*(.tdata .tdata.* .gnu.linkonce.td.*));
_etdata = ABSOLUTE(.);
}
.tbss :
{
_stbss = ABSOLUTE(.);
KEEP (*(.tbss .tbss.* .gnu.linkonce.tb.* .tcommon));
_etbss = ABSOLUTE(.);
}
.bss :
{
_sbss = . ;
*(.bss)
*(.bss.*)
*(.sbss)
*(.sbss.*)
*(.gnu.linkonce.b*)
*(COMMON)
_ebss = . ;
}
/* Stabs debugging sections. */
.stab 0 : { *(.stab) }
.stabstr 0 : { *(.stabstr) }
.stab.excl 0 : { *(.stab.excl) }
.stab.exclstr 0 : { *(.stab.exclstr) }
.stab.index 0 : { *(.stab.index) }
.stab.indexstr 0 : { *(.stab.indexstr) }
.comment 0 : { *(.comment) }
.debug_abbrev 0 : { *(.debug_abbrev) }
.debug_info 0 : { *(.debug_info) }
.debug_line 0 : { *(.debug_line) }
.debug_pubnames 0 : { *(.debug_pubnames) }
.debug_aranges 0 : { *(.debug_aranges) }
}
```
### Compilation & Linking
I had to modify how my kernel applications were compiled and linked in order to support the new crt0.c and crt1.c files. To do this, I modified the `toolchain.cmake.export` file under `nuttx/tools`. Below is a snippet of the modification:
``` CMake
file(GLOB CSTARTUP_OBJS ${NUTTX_PATH}/startup/*)
file(GLOB CXXSTARTUP_OBJS ${NUTTX_PATH}/startup/*)
add_compile_options($<$COMPILE_LANGUAGE:C>:-nostdlib>)
add_compile_options($<$COMPILE_LANGUAGE:CXX>:-nodefaultlibs$<SEMICONON>-nostartfiles>)
set(CMAKE_C_LINK_EXECUTABLE
"<CMAKE_LINKER> ${LDFLAGS} --entry=__start -T${LINKER_SCRIPT} <OBJECTS> ${CSTARTUP_OBJS} -o <TARGET> <LINK_LIBRARIES> -L${NUTTX_PATH}/libs --start-group ${LDLIBS} ${EXTRA_LIBS} --end-group"
)
set(CMAKE_CXX_LINK_EXECUTABLE
"<CMAKE_LINKER> ${LDFLAGS} --entry=__start -T${LINKER_SCRIPT} <OBJECTS> ${CXXSTARTUP_OBJS} -o <TARGET> <LINK_LIBRARIES> -L${NUTTX_PATH}/libs --start-group ${LDLIBS} ${EXTRA_LIBS} --end-group"
)
```
The above modifications ensure that C++ is not linked with any standard libraries nor is it linked with any toolchain start files (I do this manually later on) while C is. It also splits the STARTUP_OBJS variable into a CSTARTUP_OBJS and a CXXSTARTUP_OBJS variable so I can set one to crt0.o and the other to crt1.o. The modification to STARTUP_OBJS is done in `arch/risc-v/src/common/Make.defs`.
Inside the `Makefile` found within `arch/risc-v/src`, I added a section to ensure crt1 was compiled, right under crt0:
``` Makefile
crt0$(OBJEXT): %$(OBJEXT): %.c
$(call COMPILE, $<, $@)
crt1$(OBJEXT): %$(OBJEXT): %.c
$(call COMPILE, $<, $@)
```
Additionally, I had to make sure crt1.o was exported when `make export` is run so that it appears under apps/import/startup. Inside the same `Makefile` found within `arch/risc-v/src`, I modified `export_startup` to contain CXXSTARTUP_OBJS:
``` Makefile
export_startup: $(CSTARTUP_OBJS) $(CXXSTARTUP_OBJS)
ifneq ($(CSTARTUP_OBJS),)
$(Q) if [ -d "$(EXPORT_DIR)/startup" ]; then \
cp -f $(CSTARTUP_OBJS) "$(EXPORT_DIR)/startup/."; \
else \
echo "$(EXPORT_DIR)/startup does not exist"; \
exit 1; \
fi
endif
ifneq ($(CXXSTARTUP_OBJS),)
$(Q) if [ -d "$(EXPORT_DIR)/startup" ]; then \
cp -f $(CXXSTARTUP_OBJS) "$(EXPORT_DIR)/startup/."; \
else \
echo "$(EXPORT_DIR)/startup does not exist"; \
exit 1; \
fi
endif
```
Now that my crt0.o and crt1.o files are built and exported, I had to make sure they were used while building my apps. To do this, I modified the `Make.defs` file found inside `apps/import/Make.defs` to remove default libs and start up files from only the C++ apps:
``` Makefile
ARCHCFLAGS += -fno-common -pipe
ARCHCXXFLAGS += -fno-common -nostdinc++ -pipe -nodefaultlibs -nostartfiles
```
As I am manually linking the C runtime files for C++, I also added some variables to the top of the file for these objects:
``` Makefile
ARCHCRT0OBJ = $(call CONVERT_PATH,$(TOPDIR)$(DELIM)startup$(DELIM)crt0$(OBJEXT))
ARCHCRT1OBJ = $(call CONVERT_PATH,$(TOPDIR)$(DELIM)startup$(DELIM)crt1$(OBJEXT))
ARCHCRTIOBJ = $(wildcard $(shell $(CC) $(ARCHCPUFLAGS) --print-file-name=crti.o))
ARCHCRTBEGINOBJ = $(wildcard $(shell $(CC) $(ARCHCPUFLAGS) --print-file-name=crtbegin.o))
ARCHCRTENDOBJ = $(wildcard $(shell $(CC) $(ARCHCPUFLAGS) --print-file-name=crtend.o))
ARCHCRTNOBJ = $(wildcard $(shell $(CC) $(ARCHCPUFLAGS) --print-file-name=crtn.o))
```
The last modification involves the `Application.mk` file under `/apps`. Here, I updated the `ELFLD` function to perform a check of the incoming object and then call another function to perform the linking depending on whether it's a pure C application or a C++ application:
``` Makefile
define ELFLDC
$(info From ELFLDC: $1 $2)
$(Q) $(LD) $(LDELFFLAGS) $(LDLIBPATH) $(ARCHCRT0OBJ) $1 $(LDSTARTGROUP) $(LDLIBS) $(LDENDGROUP) -o $2
endef
define ELFLDCXX
$(info From ELFLDCXX: $1 $2)
$(Q) $(LD) $(LDELFFLAGS) $(LDLIBPATH) $(ARCHCRT1OBJ) $(ARCHCRTIOBJ) $(ARCHCRTBEGINOBJ) $1 $(LDSTARTGROUP) $(LDLIBS) $(LDENDGROUP) $(ARCHCRTENDOBJ) $(ARCHCRTNOBJ) -o $2
endef
define ELFLD
$(ECHO_BEGIN)"LD: $2 "
$(if $(MAINCOBJ), $(call ELFLDC, $1, $2), $(call ELFLDCXX, $1, $2))
$(ECHO_END)
endef
```
Now when I perform a `make import`, the C applications are linked to crt0.o while the C++ applications are linked to crt1.o as well as to crti.o, crtbegin.o, crtn.o and crtend.o. The result is C++ applications now work with static global instances:
The additional benefit is that C applications no longer have empty .preinit_array/.init_array/.fini_array sections and no longer call `exec_preinit`, `exec_init` and `atexit(exec_fini)` as there is no reason for them to, they will never have any constructors/destructors in them.
I apologize for this insanely long post, but I wanted to document this in the event anyone else wants to use NuttX kernel build with C++ apps and static global instances. I also apologize if my "hacks" are unsightly, I am not an expert at makefiles so I am sure I broke a number of best practices while coming up with this solution. I am hoping now that I have a solution it can be critiqued, and someone can let me know if this is an OK way of doing it. Nonetheless, I have learned a ton getting the NuttX kernel running on the Polarfire Icicle board and I couldn't have done it without your support, so thank you again!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Kernel Build Apps [nuttx]
Posted by "lupyuen (via GitHub)" <gi...@apache.org>.
lupyuen commented on issue #12356:
URL: https://github.com/apache/nuttx/issues/12356#issuecomment-2115382276
> can you comment on why the size of the generated `boot_romfsimg.h` file is so large? Is there a way to minimize the size of this?
@MainframeReboot I believe NuttX PolarFire Icicle is producing Relocatable ELF Binaries that are not Fully-Linked (which contains Relocation Symbols)? There's a patch for QEMU RISC-V that produces Fully-Linked ELF Binaries, maybe this will help:
https://github.com/lupyuen/quickjs-nuttx#full-linking-for-nuttx-apps
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Kernel Build Apps [nuttx]
Posted by "patacongo (via GitHub)" <gi...@apache.org>.
patacongo commented on issue #12356:
URL: https://github.com/apache/nuttx/issues/12356#issuecomment-2123819530
> I will look into the MMU configuration next but I'm confused why it's attempting to run from address 0 if the entry point in the header is listed as 0xC0000000.
I don't know enough about RISC-V to be much help. For ELF modules, __start is defined in crt0.c. crt0 must be linked into all ELF modules.
```
src/common/crt0.c: * Name: __start
src/common/crt0.c:void __start(int argc, char *argv[])
```
[crt0 is where shared library support would be implemented someday.]
But, perhaps, RISC-V follows a different model??? I think you need to make sure that crt0 is being built and linked into the ELF module.
One possibility is that crt0 with __start is not being including in the ELF module link. That would be the case if nothing references it. It can be forced into the link with arguments on the link command line. In the RISC-V Makefile you will find:
```
./Makefile: LDENTRY ?= -Wl,--entry=__start
./Makefile: LDENTRY ?= --entry=__start
```
That will do the job if crt0 is built and included in the link.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Kernel Build Apps [nuttx]
Posted by "pussuw (via GitHub)" <gi...@apache.org>.
pussuw commented on issue #12356:
URL: https://github.com/apache/nuttx/issues/12356#issuecomment-2125563636
A good candidate for that page fault would be C++ ctors/dtors trying to execute with kernel privileges. The same crt0 file should handle those _per process_, however I have seen places where the ctor/dtor code is executed in binfmt (there is a kconfig that controls this I believe, don't remember the name though). You should check for that.
RISC-V quite smartly prohibits executing user mapped code segments with raised privileges. This is done unconditionally and it cannot be bypassed (not by design, nor by accident).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Kernel Build Apps [nuttx]
Posted by "MainframeReboot (via GitHub)" <gi...@apache.org>.
MainframeReboot commented on issue #12356:
URL: https://github.com/apache/nuttx/issues/12356#issuecomment-2125712553
> > I have seen places where the ctor/dtor code is executed in binfmt
>
> Constructors and destructors should run in crti and crtN, respectively, for general Unix compatibility. crti should support __init and crtN should supporrt __fini.
>
> Lots more detail if you care about this: https://gcc.gnu.org/onlinedocs/gccint/Initialization.html
>
> As the OP mentioned, this is currently done with conditional logic in crt0. That is technically okay but will likely confuse people who are used to thinking about things in the GCC/GLIBC way.
>
> There is a long-open issue #1265 (and related #1263). Perhaps this is only a problem in PROTECTED mode which as a single set of ctors and dtors for the whole user-space blob. KERNEL mode uses only loadable ELF modules, each with their own ctors and dtors.
>
> crt0.c exists only for armv7-a, arm64, and riscv-5 all of which can support the KERNEL build.
>
> So crt0 with C++ ELF modules should never be used in PROTECTED mode right now or else you would get doubly constructed static classes.
>
> > I have seen places where the ctor/dtor code is executed in binfmt (there is a kconfig that controls this I believe, don't remember the name though)
>
> That is CONFIG_BINFMT_CONSTRUCTORS. I think that should be disabled in KERNEL mode to let the ELF module call its own constructors and destructors in the correct context.
>
> This seems awkward and prone to errors to me.
I naively enabled CONFIG_BINFMT_CONSTRUCTORS as the prompt told me that it enables "C++ Static Constructor Support" so it seemed like something I should enable. I have turned this off.
However, that alone did not solve the issue. I had to remove the calls `exec_ctores()` and `atexit(exec_dtors)` from `crt0.c` in order to not crash on ELF load. After doing this, helloxx starts up without issue although there still appears to be some issues with static constructors despite `CONFIG_HAVE_CXXINITIALIZE` being enabled. I am assuming this goes back crt0 not being used with C++ ELF modules. Perhaps I need the ctor/dtors calls inside my ELF files instead, I can try something like this.
Regardless, I can live with NuttX kernel build that has no C++ static constructor support given everything else appears to be working flawlessly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Kernel Build Apps [nuttx]
Posted by "MainframeReboot (via GitHub)" <gi...@apache.org>.
MainframeReboot commented on issue #12356:
URL: https://github.com/apache/nuttx/issues/12356#issuecomment-2133637011
I have dug deeper into the crt0/crt1/crti/crtbegin/crtend/crtn files as well as how .ctors are handled. I've learned that .ctors/.dtors is the old method handling constructors and destructors and that the "new" way is to use the .init_array method. I understand the previous linker script *should* work as, from what I can tell, it's combining .ctors and .init_array into _sctors while combining .dtors and .fini_array into _sdtors.
Nonetheless I decided to update things so my crt0.c source uses the .preinit_array, .init_array and .fini_array sections from my linker script. The update also checks to make sure a pointer isn't NULL before it tries to call it.
``` C
extern initializer_t __preinit_array_start[];
extern initializer_t __preinit_array_end[];
extern initializer_t __init_array_start[];
extern initializer_t __init_array_end[];
extern initializer_t __fini_array_start[];
extern initializer_t __fini_array_end[];
/****************************************************************************
* Private Functions
****************************************************************************/
#ifdef CONFIG_HAVE_CXX
static void exec_preinit(void)
{
initializer_t *preinit;
for(preinit = __preinit_array_start; preinit < __preinit_array_end; preinit++)
{
initializer_t initializer = *preinit;
printf("preinit: %p, initializer: %p\n", preinit, initializer);
if (initializer){
printf("Calling preinit %p\n", initializer);
}else{
printf("preinit initializer %p is NULL!\n", initializer);
}
}
}
static void exec_init(void)
{
initializer_t *init;
for(init = __init_array_start; init < __init_array_end; init++)
{
initializer_t initializer = *init;
printf("init: %p, initializer: %p\n", init, initializer);
if (initializer){
printf("Calling init %p\n", initializer);
}else{
printf("init initializer %p is NULL!\n", initializer);
}
}
}
static void exec_fini(void)
{
initializer_t *fini;
for(fini = __fini_array_start; fini < __fini_array_end; fini++)
{
initializer_t initializer = *fini;
printf("fini: %p, initializer: %p\n", fini, initializer);
if (initializer){
printf("Calling fini %p\n", initializer);
}else{
printf("fini initializer %p is NULL!\n", initializer);
}
}
}
#endif
/****************************************************************************
* Public Functions
****************************************************************************/
void __start(int argc, char *argv[])
{
int ret;
/* Initialize the reserved area at the beginning of the .bss/.data region
* that is visible to the RTOS.
*/
ARCH_DATA_RESERVE->ar_sigtramp = (addrenv_sigtramp_t)sig_trampoline;
#ifdef CONFIG_HAVE_CXX
/* Call C++ constructors */
exec_preinit();
exec_init();
/* Setup so that C++ destructors called on task exit */
atexit(exec_fini);
#endif
/* Call the main() entry point passing argc and argv. */
ret = main(argc, argv);
/* Call exit() if/when the main() returns */
exit(ret);
}
```
The `gnu-elf.ld` linker script was updated to be as follows:
```
SECTIONS
{
. = 0xC0000000;
.text :
{
_stext = . ;
*(.text)
*(.text.*)
*(.gnu.warning)
*(.stub)
*(.glue_7)
*(.glue_7t)
*(.jcr)
/* C++ support: The .init and .fini sections contain specific logic
* to manage static constructors and destructors.
*/
*(.gnu.linkonce.t.*)
KEEP(*(.init)) /* Old ABI */
KEEP(*(.fini)) /* Old ABI */
_etext = . ;
}
.rodata :
{
_srodata = . ;
*(.rodata)
*(.rodata1)
*(.rodata.*)
*(.gnu.linkonce.r*)
_erodata = . ;
}
. = 0xC0400000;
.data :
{
_sdata = . ;
*(.data)
*(.data1)
*(.data.*)
*(.gnu.linkonce.d*)
. = ALIGN(4);
_edata = . ;
}
/* C++ support. For each global and static local C++ object,
* GCC creates a small subroutine to construct the object. Pointers
* to these routines (not the routines themselves) are stored as
* simple, linear arrays in the .ctors section of the object file.
* Similarly, pointers to global/static destructor routines are
* stored in .dtors.
*/
.preinit_array :
{
PROVIDE_HIDDEN (__preinit_array_start = .);
KEEP(*(.preinit_array))
PROVIDE_HIDDEN (__preinit_array_end = .);
}
.init_array :
{
PROVIDE_HIDDEN (__init_array_start = .);
KEEP(*(SORT_BY_INIT_PRIORITY(.init_array.*) SORT_BY_INIT_PRIORITY(.ctors.*)))
KEEP(*(.init_array EXCLUDE_FILE (*crtbegin.o *crtbegin?.o *crtend.o *crtend?.o) .ctors))
PROVIDE_HIDDEN(__init_array_end = .);
}
.fini_array :
{
PROVIDE_HIDDEN (__fini_array_start = .);
KEEP(*(SORT_BY_INIT_PRIORITY(.fini_array.*) SORT_BY_INIT_PRIORITY(.dtors.*)))
KEEP(*(.fini_array EXCLUDE_FILE (*crtbegin.o *crtbegin?.o *crtend.o *crtend?.o) .dtors))
PROVIDE_HIDDEN(__fini_array_end = .);
}
.bss :
{
_sbss = . ;
*(.bss)
*(.bss.*)
*(.sbss)
*(.sbss.*)
*(.gnu.linkonce.b*)
*(COMMON)
_ebss = . ;
}
/* Thread local storage support */
.tdata :
{
_stdata = ABSOLUTE(.);
KEEP (*(.tdata .tdata.* .gnu.linkonce.td.*));
_etdata = ABSOLUTE(.);
}
.tbss :
{
_stbss = ABSOLUTE(.);
KEEP (*(.tbss .tbss.* .gnu.linkonce.tb.* .tcommon));
_etbss = ABSOLUTE(.);
}
/* Stabs debugging sections. */
.stab 0 : { *(.stab) }
.stabstr 0 : { *(.stabstr) }
.stab.excl 0 : { *(.stab.excl) }
.stab.exclstr 0 : { *(.stab.exclstr) }
.stab.index 0 : { *(.stab.index) }
.stab.indexstr 0 : { *(.stab.indexstr) }
.comment 0 : { *(.comment) }
.debug_abbrev 0 : { *(.debug_abbrev) }
.debug_info 0 : { *(.debug_info) }
.debug_line 0 : { *(.debug_line) }
.debug_pubnames 0 : { *(.debug_pubnames) }
.debug_aranges 0 : { *(.debug_aranges) }
}
```
This let me run the helloxx example but the static constructor call failed. Thankfully there was no crash as the null pointer check prevented it.
I dumped the .init_array section of helloxx and it shows that the constructor call isn't there at all, it was replaced with the unimp instruction:
Is there some sort of NuttX configuration setting I am missing that would cause this to occur?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org