Re: MTD device aggregation, questions for other developers

[Sebastien Lorquet <se...@lorquet.fr>]

Hello,

Thanks for these ideas.

I need aggregation at MTD layer, not at filesystem layer.

No NuttX FS at that time was immune against tearing in the way we 
required it, so we wrote our own in app space (the fs layer could not 
accomodate a special append-only file type we require). I cant change 
that design for this product.

Next product will probably use littlefs but it was not available at the 
time.

And we still need the full space for writing, so, a unionfs cannot be 
used in any case :)

Sebastien


Le 30/11/2021 à 18:10, Gregory Nutt a écrit :
> Hi, Sebastian,
>
> I think you need to use the UnionFS (fs/unionfs).  It basically lets 
> you mount two file systems at the same location with the content 
> combined.
>
> There is an example at apps/examples/unionfs:
>
>    140   ret = mount(MOUNT_DEVNAME_A, CONFIG_EXAMPLES_UNIONFS_TMPA,
>    "romfs",
>    141               MS_RDONLY, NULL);
>    168   ret = mount(MOUNT_DEVNAME_B, CONFIG_EXAMPLES_UNIONFS_TMPB,
>    "romfs",
>    169               MS_RDONLY, NULL);
>    178   ret = unionfs_mount(CONFIG_EXAMPLES_UNIONFS_TMPA, NULL,
>    179                       CONFIG_EXAMPLES_UNIONFS_TMPB, "offset",
>    180                       CONFIG_EXAMPLES_UNIONFS_MOUNTPT);
>
> apps/thttpd also has an option to mount a binfs with "real" file 
> system on other media.  This permits in-FLASH builtin apps to be 
> intermixed or replaced by ELF modules in the "real" file system.
>
> Greg
>
> On 11/30/2021 11:01 AM, Sebastien Lorquet wrote:
>> Hello,
>>
>> For history our board need 128 Mbit of flash, but the 128 Mbit parts 
>> are sloooow and have gigantic erase blocks.
>>
>> So as an alternative our plan was to have two 64 Mbit SST parts (fast 
>> and small erases) and aggregate them in software, so that we can 
>> present the other drivers a single 128 Mbit device.
>>
>> I can do this alone my own way, but I thought this would be of 
>> interest for all NuttX developers.
>>
>>
>> This is the reverse of the mtd_partition driver.
>>
>> You give it a list of MTD devices (with similar geometries, this will 
>> be checked) and the result is one larger MTD device that span all the 
>> small ones.
>>
>> This is a kind of raid0 for MTD devices.
>>
>>
>> I have questions about the API. Basically there are two choices:
>>
>> -have an empty initialiser (mtd_agg_init()) that return the larger 
>> device, and a function mtd_agg_adddev(child) to grow the device
>>
>> -have an initializer that takes as parameter the list of devices to 
>> aggregate
>>
>>
>> The assumption is that the returned MTD devices are pointers to 
>> malloc()ed ram. So it makes no sense to have the list of child 
>> pointers as a const array. it is inherently dynamic device construction.
>>
>>
>> What is your preference?
>>
>> The separate add() function requires realloc. This is probably an 
>> argument against it.
>>
>> The initializer with parameters need allocation of a ram array to 
>> store the MTD pointers. Not nice.
>>
>>
>> Should I copy the pointers in my own privately allocated mtd device 
>> or should I rely on a stable, externally allocated device list?
>>
>>
>> Do you have any design advice here, so NuttX can benefit from the 
>> best API?
>>
>> Coding has not started yet, so I have no personal preference.
>>
>>
>> Sebastien
>>
>