[GitHub] [incubator-nuttx] v01d opened a new issue #3031: Unified device interface, callback based initialization and devicetree (DTS)

[GitBox <gi...@apache.org>]

v01d opened a new issue #3031:
URL: https://github.com/apache/incubator-nuttx/issues/3031


   This is an in-depth proposal for the GSoC idea about devicetree support. I have thought this about some time so I this is a proposed roadmap. I actually think the underlying improvement is to develop a unified device interface which can really improve board support. Devicetree support would be a convenience but not a requirement.
   
   # Problem statement
   
   Board logic is in charge of completing initialization of hardware by instantiating drivers (note that I refer to ANY driver not just something that interacts with hardware, e.g. initialization of a nuttx subsystem, although exceptions could be identified). This means that board-level code part of nuttx/ repo usually needs to either: a) be modified by end user for the specific application, b) copy the whole board code and change it into a "custom" board, c) add a Kconfig option for the specific need and port it upstream. These options are all problematic in the long term:
     - a) means one always needs to fork nuttx/ repo, which unloads the burden on the end user
     - b) is a maintenance problem when upstream board logic changes significantly
     - c) is a maintenance (and testing, thanks to the added options) problem to nuttx maintainers
   
   Currently, c) is already an issue: many boards have many options for off-board devices which may or may not be ever used. In this way, board-logic acts more like an example code than the code actually required to support the board and its on-board hardware.
   
   A further problem about "baking in" the initialization of drivers to board-logic based on Kconfig options is that it is restricting in terms of cleanly supporting multiple instances of a driver: each instance would require a specific option, a specific line of code and a copy of all required parameters typically set via Kconfig. One underlying problem here is that Kconfig describes options, not resources. Kconfig is appropriate for enabling/disabling support for a driver, but not for declaring its presence.
   
   Another issue related to driver handling is that driver deinitialization is often overlooked and not even enforced. More generally, drivers are free to define their own api for initialization/deinitialization without any consistency.
   
   # Proposed solution
   
   The proposed solution is based around three changes:
     1. define and adopt a callback-based driver interface (e.g. `struct driver_t`).
     2. rework hardware initialization to be done in a series of stages from within the highest-possible level (only leave to board-logic what can really not be called from the OS itself via these callbacks)
     3. provide a mechanism for a board to instantiate drivers as instances of the device driver interface
   
   Point 3. would look like an array of `struct driver_t` which can be statically defined. Memory requirement would be minimal if the driver struct is kept small and would only occupy space in flash. A set of macros can be used to ease constructing this array declaratively.
   
   Devicetree at this point would allow for a standardized and widely adopted language (text-based .dts) to describe hardware resources (and maybe non-hardware ones, such as a filesystem to be mounted, a subsystem to be initialized). From the .dts we can auto-generate a .c file containing the definition of the array from 3. The benefit of a devicetree over manually populating the array is that it is hierarchical (one can define a "base board", a child arduino shield-like addon, leaving the user to define/refine this on a leaf dts). Use of devicetree itself would only mean some extra tooling, but not much added complexity.
   
   ## Driver based interface
   
   A good starting point would be to define something like:
   ```
   struct device_t
   {
     int (*initialize)(struct device_cfg_s *cfg);
     int (*deinitialize)(void);
   };
   ```
   then, we can define:
   ```
   struct device_cfg_s
   {
     int id;
     /* ... other fields here */
   };
   ```
   as a *base* struct which can be actually used by specific drivers as:
   ```
   struct mydevice_cfg_s
   {
     struct device_cfg_s base_cfg;
     /* custom options */  
   }
   ```
   so it can be instantiated and then passed as-is (via pointer cast) to `initialize()`.
   
   Most likely it would be good to consider classes of drivers, since a common option would be the
   I2C/SPI instance pointer to be passed to driver. So there could actually be a `struct device_i2c_t` or
   even a generic `struct device_bus_t`. 
   
   ## Driver initialization
   
   During later stages of OS initialization, we could have the OS itself make the calls to `initialize()`, for each element of the
   device array (defined by board logic). This should be made in stages (and thus, maybe there could be different sub-arrays for each stage) since for example buses need to be initialized before devices using these buses. Most likely two or three stages could be enough.
   
   The goal here is that the board bringup function wouldn't be really needed (and ideally should be empty). Maybe it is required for some very special calls but these should always be made in the more common points to avoid repeating code between all boards and re-introducing a maintenance problem. The end user should always have the possibility of writing a driver and instantiating it so that the OS itself initializes it (maybe we can even make a special "board initialization driver" for this, which would replace the bringup function).
   
   ## Driver instantiation
   
   As previously mentioned, a const static array of `struct device_t` can be defined, which would be acompanied by const static instances of the configuration structs for each device. In order to bind drivers between themselves, some sort of driver class and driver id needs to be used (for example, to indicate that a given I2C device is to be bound to a given I2C bus device). If all of these are static structs, maybe it is possible to directly pass pointers of a device struct to another device as a field in the corresponding config struct.
   To avoid too much verbosity, macros can be introduced to ease legibility of these struct instantiations,  (e.g. REGISTER_DEVICE() to declare an array entry, etc).
   
   ## Devicetree support
   
   For reference, it would be good if the reader can have a look at the original issue #1020 describing this feature (which was well before on thinking about the previous architecture). I'll try to give the main points on the current status of that effort here as I already worked on the tooling:
   
   - Text-based DTS files are typically compiled into a binary blob, the DTB. The blob is then linked into final binary and processed during runtime at bootup. This is not very good for embedded systems due to performance reasons. Thus, I followed the approach used by Zephyr, which generates static information from the DTS so that performance reduction is minimal.
   - Zephyr's approach was to write python tooling to directly parse the DTS and generate a set of macros that can then be used in code to consume required information. A similar strategy would require us to add python as build dependency and I understand this is not desireable
   - My approach was to use standard tooling and a custom-built C++ code generation tool: [dtgen](https://gitlab.com/nuttx_projects/dtgen).
   
   Initially I looked for a C based DTS parser library but the officially supported library, part of the DTS->DTB compiler project, is actually used to parse a DTB. So to base my work on well supported tools the workflow is as follows:
   
   ```DTS ---(dts compiler) ---> DTB ---- (my tool, based on dtb library) ---> C code```
   
   The tool I wrote basically reads a bunch of YAML files which define how a DTS entry is to be translated into C code, loads the DTB and produces the code. This tool, if used, should be updated to the architecture defined in this issue since it originally was supposed to generate code to call into existing individual device APIs. Most likely it would be much simpler to just generate code that instantiates the structs
   
   I think devicetrees are a good option to support although I don't think we would fully use them as intended as they include information which we already handle (for example gpio pin configuration). But devicetree is just a very basic syntax plus some standard (or, actually, some set of conventions) we can choose to partially follow and appropriate to suite our needs best.
   
   NOTE: I can probably work on the tool if needed and contribute to this effort but of course the main work will be to actually adopt this system, which involves modifying all drivers and board logic. I hope that the GSoC student can be a big help in that effort.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-nuttx] AndrewD commented on issue #3031: Unified device interface, callback based initialization and devicetree (DTS)

[GitBox <gi...@apache.org>]

AndrewD commented on issue #3031:
URL: https://github.com/apache/incubator-nuttx/issues/3031#issuecomment-1066063212


   Have you gone any further with this yourself since this proposal? We are interested to use device tree with nuttx in a product and have cloned your fork referenced in #1020 for initial experimentation. Maybe we can collaborate on this in some way to get some infrastructure integrated in nuttx? Ideally this would be optional for drivers and boards for phased adoption.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-nuttx] xiaoxiang781216 commented on issue #3031: Unified device interface, callback based initialization and devicetree (DTS)

[GitBox <gi...@apache.org>]

xiaoxiang781216 commented on issue #3031:
URL: https://github.com/apache/incubator-nuttx/issues/3031#issuecomment-1066083202


   It's nice to see we can finish @protobits's initiative.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org