You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mynewt.apache.org by marko kiiskila <ma...@runtime.io> on 2016/04/07 18:27:37 UTC
choosing between FCB and NFFS
Hi,
at the moment bootloader always uses NFFS. App can tell bootloader
to pick specific image for next boot by creating a file. Bootloader also
keeps track of image update by writing to another file.
I want to use sys/config and FCB (flash circular buffer) for
this instead. Reason for this change is the impact on code size.
My current idea revolves around creating new versions of apps;
one version uses NFFS and second version uses FCB. Both would
go for the same flash map entry for location of their storage.
And you would pick your target to use either NFFS or FCB depending on
what platform you’re building for.
Not too keen on this though, as it would require a bit of code
duplication (main.c on boot/slinky/blinky).
Better suggestions?
Re: choosing between FCB and NFFS
Posted by marko kiiskila <ma...@runtime.io>.
> On Apr 7, 2016, at 3:03 PM, paul@wrada.com wrote:
>
> I prefer to keep the boot loaders as simple as possible.
Agreed.
> Unless there is good justification, I think we should move all the boot
> loaders to the simplest (and smallest) implementation for flash storage
> that we can and NOT give a choice.
I still want to give user the choice between the matter; quite a few
flashes have big sector sizes and consequently not that many sectors, so
having both NFFS and FCB co-exist might not always be possible.
And the user might still want to insist on using NFFS.
We could make FCB default. That’s a smaller implementation
of the two, so it would a better fit for more platforms.
> As an alternate if we want to keep the functionality is to have two
> different fw-storage libraries that are included in the boot loader and
> app respectively that perform this image stuff with the same API but
> different underlying mechanisms and let users decide which to pull into
> their project.
That’s exactly what I was going to do. I was going to use
sys/config interface for image stuff. This has 2 options for permanent
storage, one going to file system and another one going to FCB.
I was going to change the code to start using this interface.
It will be possible for the user to pick which implementation they
want by setting up sys/config appropriately within their app and
bootloader main().
>
> Paul
>
>
>
>
> On 4/7/16, 2:27 PM, "ray suorsa" <re...@runtime.io> wrote:
>
>> Hi,
>>
>> 1) It seems like moving the bootloader over to FCB would be a good
>> thing. I don¹t understand requiring both a NFFS and FCB implementation.
>> Why not just FCB to keep things simple?
>>
>> 2) I have one feature request or maybe it is a design change.
>> Currently the when talking to the OS and getting a list of images
>> it replies with the version of the image. Versions are fine and
>> I think the OS needs to reply with version info. However I think
>> the images should be indexed and stored via the firmware image¹s
>> embedded buildid. I would like the buildid to be The-One-True-Id
>> for a build. Given that it is a SHA256 of the firmware it would
>> be exceedingly rare to have a SHA256 collision vs having collisions
>> in user supplied versions. Therefore if the image list could reply
>> with something like:
>>
>> {
>> "Images": [
>> [³f50200f8256235434440c73dc79705daf1462d23395e7b6e0f299a78946b5159²,
>> ³1.2.3.4"],
>> [³7723a4bad19e23989ba4dad1b593e44a950a7fe9b10163fdd25a577454caf9e2²,
>> ³1.2.3.4"]
>> ]
>> }
>>
>> or even,
>>
>> {
>> "Images": [
>>
>> ³f50200f8256235434440c73dc79705daf1462d23395e7b6e0f299a78946b5159:1.2.3.4"
>> ,
>>
>> ³7723a4bad19e23989ba4dad1b593e44a950a7fe9b10163fdd25a577454caf9e2:1.2.3.4"
>> ]
>> }
>>
>> then we have good visibility into what the device really has.
>>
>> -res
>>
>>> On Apr 7, 2016, at 12:24 PM, marko kiiskila <ma...@runtime.io> wrote:
>>>
>>> Hi,
>>>
>>> I went through the same though process, and ended in same place :)
>>>
>>> We can avoid code duplication by having a version of slinky that has
>>> #if directives for NFFS vs FCB specific pieces. And then have other
>>> apps that includes this slinky, while selecting between the 2 options.
>>> And then specifying target you could pick one of these.
>>>
>>> But I¹m not keen on that one either, because I feel it would make it
>>> harder to understand what exactly it is that you¹re building.
>>>
>>> And like you say, this is only problem for these sample apps of ours,
>>> which we¹ve tried to make as generic as possible.
>>>
>>>> On Apr 7, 2016, at 11:25 AM, Christopher Collins <cc...@apache.org>
>>>> wrote:
>>>>
>>>> Upon reflection, this idea has problems of its own. The app still
>>>> needs to initialize the specific flash storage package being used. So,
>>>> sorry for posting before thinking! The idea could be salvaged with the
>>>> use of some #if directives in the app code, but I am not sure this is
>>>> any better than duplicating the entire app.
>>>>
>>>> I think the problem you described won't be applicable to most apps.
>>>> The
>>>> example apps are affected by this because they are meant to be as
>>>> generic as possible.
>>>>
>>>> Chris
>>>>
>>>> On Thu, Apr 07, 2016 at 11:03:07AM -0700, Christopher Collins wrote:
>>>>> That mostly sounds good to me, though I agree that the need to
>>>>> duplicate
>>>>> app code is not ideal.
>>>>>
>>>>> Here is an alternative idea:
>>>>> * Both fs and fcb export a suitable API (e.g., "bootapi").
>>>>> * By default, apps depend on nffs.
>>>>> * If a particular feature is set in the target, the app depends on
>>>>> fcb instead.
>>>>>
>>>>> Something like:
>>>>>
>>>>> pkg.deps.base:
>>>>> - libs/os
>>>>> - sys/log
>>>>> - libs/newtmgr
>>>>> - libs/console/full
>>>>> - libs/shell
>>>>>
>>>>> pkg.deps: [pkg.deps.base, fs/nffs]
>>>>> pkg.deps.FCB_BOOT.OVERWRITE: [pkg.deps.base, sys/config, sys/fcb]
>>>>>
>>>>> I may have gotten the yaml syntax wrong, but I think the concept
>>>>> works.
>>>>>
>>>>> Chris
>>>
>>
>
Re: choosing between FCB and NFFS
Posted by "paul@wrada.com" <pa...@wrada.com>.
I prefer to keep the boot loaders as simple as possible.
Unless there is good justification, I think we should move all the boot
loaders to the simplest (and smallest) implementation for flash storage
that we can and NOT give a choice.
As an alternate if we want to keep the functionality is to have two
different fw-storage libraries that are included in the boot loader and
app respectively that perform this image stuff with the same API but
different underlying mechanisms and let users decide which to pull into
their project.
Paul
On 4/7/16, 2:27 PM, "ray suorsa" <re...@runtime.io> wrote:
>Hi,
>
>1) It seems like moving the bootloader over to FCB would be a good
>thing. I don¹t understand requiring both a NFFS and FCB implementation.
>Why not just FCB to keep things simple?
>
>2) I have one feature request or maybe it is a design change.
>Currently the when talking to the OS and getting a list of images
>it replies with the version of the image. Versions are fine and
>I think the OS needs to reply with version info. However I think
>the images should be indexed and stored via the firmware image¹s
>embedded buildid. I would like the buildid to be The-One-True-Id
>for a build. Given that it is a SHA256 of the firmware it would
>be exceedingly rare to have a SHA256 collision vs having collisions
>in user supplied versions. Therefore if the image list could reply
>with something like:
>
>{
> "Images": [
> [³f50200f8256235434440c73dc79705daf1462d23395e7b6e0f299a78946b5159²,
>³1.2.3.4"],
> [³7723a4bad19e23989ba4dad1b593e44a950a7fe9b10163fdd25a577454caf9e2²,
>³1.2.3.4"]
> ]
>}
>
>or even,
>
>{
> "Images": [
>
>³f50200f8256235434440c73dc79705daf1462d23395e7b6e0f299a78946b5159:1.2.3.4"
>,
>
>³7723a4bad19e23989ba4dad1b593e44a950a7fe9b10163fdd25a577454caf9e2:1.2.3.4"
> ]
>}
>
>then we have good visibility into what the device really has.
>
>-res
>
>> On Apr 7, 2016, at 12:24 PM, marko kiiskila <ma...@runtime.io> wrote:
>>
>> Hi,
>>
>> I went through the same though process, and ended in same place :)
>>
>> We can avoid code duplication by having a version of slinky that has
>> #if directives for NFFS vs FCB specific pieces. And then have other
>> apps that includes this slinky, while selecting between the 2 options.
>> And then specifying target you could pick one of these.
>>
>> But I¹m not keen on that one either, because I feel it would make it
>> harder to understand what exactly it is that you¹re building.
>>
>> And like you say, this is only problem for these sample apps of ours,
>> which we¹ve tried to make as generic as possible.
>>
>>> On Apr 7, 2016, at 11:25 AM, Christopher Collins <cc...@apache.org>
>>>wrote:
>>>
>>> Upon reflection, this idea has problems of its own. The app still
>>> needs to initialize the specific flash storage package being used. So,
>>> sorry for posting before thinking! The idea could be salvaged with the
>>> use of some #if directives in the app code, but I am not sure this is
>>> any better than duplicating the entire app.
>>>
>>> I think the problem you described won't be applicable to most apps.
>>>The
>>> example apps are affected by this because they are meant to be as
>>> generic as possible.
>>>
>>> Chris
>>>
>>> On Thu, Apr 07, 2016 at 11:03:07AM -0700, Christopher Collins wrote:
>>>> That mostly sounds good to me, though I agree that the need to
>>>>duplicate
>>>> app code is not ideal.
>>>>
>>>> Here is an alternative idea:
>>>> * Both fs and fcb export a suitable API (e.g., "bootapi").
>>>> * By default, apps depend on nffs.
>>>> * If a particular feature is set in the target, the app depends on
>>>> fcb instead.
>>>>
>>>> Something like:
>>>>
>>>> pkg.deps.base:
>>>> - libs/os
>>>> - sys/log
>>>> - libs/newtmgr
>>>> - libs/console/full
>>>> - libs/shell
>>>>
>>>> pkg.deps: [pkg.deps.base, fs/nffs]
>>>> pkg.deps.FCB_BOOT.OVERWRITE: [pkg.deps.base, sys/config, sys/fcb]
>>>>
>>>> I may have gotten the yaml syntax wrong, but I think the concept
>>>>works.
>>>>
>>>> Chris
>>
>
Re: choosing between FCB and NFFS
Posted by marko kiiskila <ma...@runtime.io>.
Hi,
> On Apr 7, 2016, at 2:27 PM, ray suorsa <re...@runtime.io> wrote:
>
...
> 2) I have one feature request or maybe it is a design change.
> Currently the when talking to the OS and getting a list of images
> it replies with the version of the image. Versions are fine and
> I think the OS needs to reply with version info. However I think
> the images should be indexed and stored via the firmware image’s
> embedded buildid. I would like the buildid to be The-One-True-Id
> for a build. Given that it is a SHA256 of the firmware it would
> be exceedingly rare to have a SHA256 collision vs having collisions
> in user supplied versions. Therefore if the image list could reply
> with something like:
>
> {
> "Images": [
> [“f50200f8256235434440c73dc79705daf1462d23395e7b6e0f299a78946b5159”, “1.2.3.4"],
> [“7723a4bad19e23989ba4dad1b593e44a950a7fe9b10163fdd25a577454caf9e2”, “1.2.3.4"]
> ]
> }
>
> or even,
>
> {
> "Images": [
> “f50200f8256235434440c73dc79705daf1462d23395e7b6e0f299a78946b5159:1.2.3.4",
> “7723a4bad19e23989ba4dad1b593e44a950a7fe9b10163fdd25a577454caf9e2:1.2.3.4"
> ]
> }
>
> then we have good visibility into what the device really has.
>
Getting back to this. I like the idea (except for the above listing having different hash for
same version image :)
I don’t think there’s much benefit in keeping the backwards compatible command/response
support around. No users for this yet, really. So I’m thinking that I would just replace the existing
commands with this. Code space is precious. Unless there’s a compelling reason
to do so.
Let me know,
M
Re: choosing between FCB and NFFS
Posted by ray suorsa <re...@runtime.io>.
Hi,
1) It seems like moving the bootloader over to FCB would be a good
thing. I don’t understand requiring both a NFFS and FCB implementation.
Why not just FCB to keep things simple?
2) I have one feature request or maybe it is a design change.
Currently the when talking to the OS and getting a list of images
it replies with the version of the image. Versions are fine and
I think the OS needs to reply with version info. However I think
the images should be indexed and stored via the firmware image’s
embedded buildid. I would like the buildid to be The-One-True-Id
for a build. Given that it is a SHA256 of the firmware it would
be exceedingly rare to have a SHA256 collision vs having collisions
in user supplied versions. Therefore if the image list could reply
with something like:
{
"Images": [
[“f50200f8256235434440c73dc79705daf1462d23395e7b6e0f299a78946b5159”, “1.2.3.4"],
[“7723a4bad19e23989ba4dad1b593e44a950a7fe9b10163fdd25a577454caf9e2”, “1.2.3.4"]
]
}
or even,
{
"Images": [
“f50200f8256235434440c73dc79705daf1462d23395e7b6e0f299a78946b5159:1.2.3.4",
“7723a4bad19e23989ba4dad1b593e44a950a7fe9b10163fdd25a577454caf9e2:1.2.3.4"
]
}
then we have good visibility into what the device really has.
-res
> On Apr 7, 2016, at 12:24 PM, marko kiiskila <ma...@runtime.io> wrote:
>
> Hi,
>
> I went through the same though process, and ended in same place :)
>
> We can avoid code duplication by having a version of slinky that has
> #if directives for NFFS vs FCB specific pieces. And then have other
> apps that includes this slinky, while selecting between the 2 options.
> And then specifying target you could pick one of these.
>
> But I’m not keen on that one either, because I feel it would make it
> harder to understand what exactly it is that you’re building.
>
> And like you say, this is only problem for these sample apps of ours,
> which we’ve tried to make as generic as possible.
>
>> On Apr 7, 2016, at 11:25 AM, Christopher Collins <cc...@apache.org> wrote:
>>
>> Upon reflection, this idea has problems of its own. The app still
>> needs to initialize the specific flash storage package being used. So,
>> sorry for posting before thinking! The idea could be salvaged with the
>> use of some #if directives in the app code, but I am not sure this is
>> any better than duplicating the entire app.
>>
>> I think the problem you described won't be applicable to most apps. The
>> example apps are affected by this because they are meant to be as
>> generic as possible.
>>
>> Chris
>>
>> On Thu, Apr 07, 2016 at 11:03:07AM -0700, Christopher Collins wrote:
>>> That mostly sounds good to me, though I agree that the need to duplicate
>>> app code is not ideal.
>>>
>>> Here is an alternative idea:
>>> * Both fs and fcb export a suitable API (e.g., "bootapi").
>>> * By default, apps depend on nffs.
>>> * If a particular feature is set in the target, the app depends on
>>> fcb instead.
>>>
>>> Something like:
>>>
>>> pkg.deps.base:
>>> - libs/os
>>> - sys/log
>>> - libs/newtmgr
>>> - libs/console/full
>>> - libs/shell
>>>
>>> pkg.deps: [pkg.deps.base, fs/nffs]
>>> pkg.deps.FCB_BOOT.OVERWRITE: [pkg.deps.base, sys/config, sys/fcb]
>>>
>>> I may have gotten the yaml syntax wrong, but I think the concept works.
>>>
>>> Chris
>
Re: choosing between FCB and NFFS
Posted by marko kiiskila <ma...@runtime.io>.
Hi,
I went through the same though process, and ended in same place :)
We can avoid code duplication by having a version of slinky that has
#if directives for NFFS vs FCB specific pieces. And then have other
apps that includes this slinky, while selecting between the 2 options.
And then specifying target you could pick one of these.
But I’m not keen on that one either, because I feel it would make it
harder to understand what exactly it is that you’re building.
And like you say, this is only problem for these sample apps of ours,
which we’ve tried to make as generic as possible.
> On Apr 7, 2016, at 11:25 AM, Christopher Collins <cc...@apache.org> wrote:
>
> Upon reflection, this idea has problems of its own. The app still
> needs to initialize the specific flash storage package being used. So,
> sorry for posting before thinking! The idea could be salvaged with the
> use of some #if directives in the app code, but I am not sure this is
> any better than duplicating the entire app.
>
> I think the problem you described won't be applicable to most apps. The
> example apps are affected by this because they are meant to be as
> generic as possible.
>
> Chris
>
> On Thu, Apr 07, 2016 at 11:03:07AM -0700, Christopher Collins wrote:
>> That mostly sounds good to me, though I agree that the need to duplicate
>> app code is not ideal.
>>
>> Here is an alternative idea:
>> * Both fs and fcb export a suitable API (e.g., "bootapi").
>> * By default, apps depend on nffs.
>> * If a particular feature is set in the target, the app depends on
>> fcb instead.
>>
>> Something like:
>>
>> pkg.deps.base:
>> - libs/os
>> - sys/log
>> - libs/newtmgr
>> - libs/console/full
>> - libs/shell
>>
>> pkg.deps: [pkg.deps.base, fs/nffs]
>> pkg.deps.FCB_BOOT.OVERWRITE: [pkg.deps.base, sys/config, sys/fcb]
>>
>> I may have gotten the yaml syntax wrong, but I think the concept works.
>>
>> Chris
Re: choosing between FCB and NFFS
Posted by Christopher Collins <cc...@apache.org>.
Upon reflection, this idea has problems of its own. The app still
needs to initialize the specific flash storage package being used. So,
sorry for posting before thinking! The idea could be salvaged with the
use of some #if directives in the app code, but I am not sure this is
any better than duplicating the entire app.
I think the problem you described won't be applicable to most apps. The
example apps are affected by this because they are meant to be as
generic as possible.
Chris
On Thu, Apr 07, 2016 at 11:03:07AM -0700, Christopher Collins wrote:
> That mostly sounds good to me, though I agree that the need to duplicate
> app code is not ideal.
>
> Here is an alternative idea:
> * Both fs and fcb export a suitable API (e.g., "bootapi").
> * By default, apps depend on nffs.
> * If a particular feature is set in the target, the app depends on
> fcb instead.
>
> Something like:
>
> pkg.deps.base:
> - libs/os
> - sys/log
> - libs/newtmgr
> - libs/console/full
> - libs/shell
>
> pkg.deps: [pkg.deps.base, fs/nffs]
> pkg.deps.FCB_BOOT.OVERWRITE: [pkg.deps.base, sys/config, sys/fcb]
>
> I may have gotten the yaml syntax wrong, but I think the concept works.
>
> Chris
Re: choosing between FCB and NFFS
Posted by Christopher Collins <cc...@apache.org>.
That mostly sounds good to me, though I agree that the need to duplicate
app code is not ideal.
Here is an alternative idea:
* Both fs and fcb export a suitable API (e.g., "bootapi").
* By default, apps depend on nffs.
* If a particular feature is set in the target, the app depends on
fcb instead.
Something like:
pkg.deps.base:
- libs/os
- sys/log
- libs/newtmgr
- libs/console/full
- libs/shell
pkg.deps: [pkg.deps.base, fs/nffs]
pkg.deps.FCB_BOOT.OVERWRITE: [pkg.deps.base, sys/config, sys/fcb]
I may have gotten the yaml syntax wrong, but I think the concept works.
Chris
On Thu, Apr 07, 2016 at 09:27:37AM -0700, marko kiiskila wrote:
> Hi,
>
> at the moment bootloader always uses NFFS. App can tell bootloader
> to pick specific image for next boot by creating a file. Bootloader also
> keeps track of image update by writing to another file.
>
> I want to use sys/config and FCB (flash circular buffer) for
> this instead. Reason for this change is the impact on code size.
>
> My current idea revolves around creating new versions of apps;
> one version uses NFFS and second version uses FCB. Both would
> go for the same flash map entry for location of their storage.
>
> And you would pick your target to use either NFFS or FCB depending on
> what platform you’re building for.
>
> Not too keen on this though, as it would require a bit of code
> duplication (main.c on boot/slinky/blinky).
>
> Better suggestions?