You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nuttx.apache.org by "Alan C. Assis" <ac...@gmail.com> on 2023/03/01 12:43:04 UTC
Re: W25Nx NAND flash support
Hi Xiang Xiao,
This is a great news, but I'm afraid yaffs is not a good option for
everybody because it uses GPL license.
But yes, it could be an option for people and companies that can
release all their software as open-source. PX4 project for example
could use it.
BR,
Alan
On 2/28/23, Xiang Xiao <xi...@gmail.com> wrote:
> We are working on yaffs porting, mtd_s need a little bit of extension to
> support NAND flash.
> Here is the patch: https://github.com/apache/nuttx/pull/8670
> LittleFS needs some modification to get the benefit.
>
> On Sun, Feb 26, 2023 at 10:14 PM Nathan Hartman <ha...@gmail.com>
> wrote:
>
>> On Sat, Feb 25, 2023 at 8:53 AM Gregory Nutt <sp...@gmail.com> wrote:
>>
>> >
>> > > Maybe LittleFS or SmartFS could be extended to handle NAND.
>> >
>> > I believe that LittleFS has been used with NAND with a NAND FLASH
>> > translation layer. I am not sure of the state of that work, but I know
>> > that people have tried it. Google "LittleFS on NAND flash" and see the
>> > hits.
>> >
>> > If a NAND flash translation layer (NFTL) were ported to NuttX, then you
>> > should be able to use almost any file system, although some would be
>> > extremely inefficient with NAND. The NAND flash layer would need to do
>> > sparing, wear-leveling, ECC calculations, etc. as needed by NAND.
>> >
>> > I have used NXFFS with NAND and it /almost /works.
>> >
>> > The basic filesystem issue is that all available filesystems need to do
>> > read-modify-write operations on flash sections. And they assume that
>> > you can always write a '1' bit to a '0' bit. SmartFS and NXFFS both
>> > assume this behavior explicitly. It is a good assumption with NOR
>> > flash. The problem is that NAND can only be written a whole sector at
>> > a
>> > time. This is because the error correction coding (ECC): If you
>> > change
>> > even one bit in the NAND sector, you have to re-write the whole sector
>> > with new ECC (because you can't change the '0' bits in the ECC back
>> > into
>> > '1''s without erasing the sector).
>> >
>> > Since SmartFS and NXFFS both do bit twiddling in the sectors they would
>> > be very inefficient with an NFTL: Each bit twiddle would cause a whole
>> > sector re-write. The same is probably true for LittleFS. Other file
>> > systems like FAT could also be used if there were an NFTL, however, FAT
>> > would also be inefficient (each directory update or FAT update and each
>> > file data size change would cause another sector write).
>> >
>> > So the only real solution to support NAND efficiently is use a file
>> > system that is designed specifically around the peculiarities of NAND.
>> > That would require some research (Alan has suggested a couple of places
>> > to start).
>>
>>
>>
>> I seem to recall reading that in NAND flash, one of the challenges is
>> that
>> modifying a bit causes a disturbance that can corrupt other bits, even in
>> unrelated sectors, and that even reading can have this effect. The
>> disturbance only affects bits a little at a time, but after some number
>> of
>> accesses, the affected sectors need to be written to new sectors, keeping
>> track of bad sectors formed in the process, to avoid data loss. This may
>> lead to a cascade effect in which reading a sector may cause numerous
>> sectors to be re-written. This phenomenon is called write amplification.
>> The takeaway from a usage perspective is that you want to over-provision
>> the amount of NAND flash you install, to leave plenty of room for all
>> this
>> swapping, and plenty of extra sectors to increase the amount of accesses
>> before failure. The greater the over-provisioning, the longer until
>> failure.
>>
>> I haven't tried yet, but I would like at some point to support a micro-SD
>> and/or micro-MMC card as the backing store and use one that has built in
>> hardware wear leveling, with some appropriate file system of course. This
>> may add $50 or more to the bill of materials depending on the card
>> capacity, but makes it possible for users to replace the card if it
>> fails,
>> in contrast to soldered-on flash chips, which can be replaced too, but
>> only
>> with the appropriate tools and skills.
>>
>> Cheers
>> Nathan
>>
>
Re: W25Nx NAND flash support
Posted by Xiang Xiao <xi...@gmail.com>.
On Wed, Mar 1, 2023 at 10:43 PM Alan C. Assis <ac...@gmail.com> wrote:
> Hi Xiang,
>
> On 3/1/23, Xiang Xiao <xi...@gmail.com> wrote:
> > On Wed, Mar 1, 2023 at 8:43 PM Alan C. Assis <ac...@gmail.com> wrote:
> >
> >> Hi Xiang Xiao,
> >>
> >> This is a great news, but I'm afraid yaffs is not a good option for
> >> everybody because it uses GPL license.
> >>
> >>
> > Yes, but you can pay some money to switch to a more friendly license.
> >
>
> Do you have more info about it? Is it a reasonable price for small
> companies (let's say: less than U$ 5k to use as you wish or less than
> U$ 1 per device) ?
>
One product costs 10000 Euro without the quantity limitation.
>
> BR,
>
> Alan
>
Re: W25Nx NAND flash support
Posted by "Alan C. Assis" <ac...@gmail.com>.
On 3/1/23, Xiang Xiao <xi...@gmail.com> wrote:
> On Wed, Mar 1, 2023 at 10:45 PM Alan C. Assis <ac...@gmail.com> wrote:
>
>> oh, I just found it:
>>
>> "The fee depends on the number of distinct Products in which Yaffs
>> will be used, and there is a base price of GBP £10,000. If more than
>> one Product is included in the same Licence there is usually a
>> reduction in the second fee."
>>
>> For a big company it is fine, but small companies could be a "don't go!".
>>
>
> If the cost isn't affordable and the company doesn't want to public all
> source code, he can try this approach:
>
> 1. Enable userfs to move yaffs from kernel to userspace
> 2. Build yaffs as wasm module and run it in:
> https://github.com/apache/nuttx-apps/tree/master/interpreters/wamr
>
> This technology could satisfy all GPL requirements but without
> exposing your private source code.
>
Very nice! Good approach!
Maybe we can make it easy to let user to do it.
BR,
Alan
Re: W25Nx NAND flash support
Posted by Xiang Xiao <xi...@gmail.com>.
I will just talk about some possible solutions for small companies. We
already pay the license from Aleph One which is very very cheap from my
point of view.
On Wed, Mar 1, 2023 at 10:53 PM Sebastien Lorquet <se...@lorquet.fr>
wrote:
> Oh come on you are just trying to circumvent the GPL here
>
> Sebastien
>
> Le 01/03/2023 à 15:50, Xiang Xiao a écrit :
> > On Wed, Mar 1, 2023 at 10:45 PM Alan C. Assis <ac...@gmail.com> wrote:
> >
> >> oh, I just found it:
> >>
> >> "The fee depends on the number of distinct Products in which Yaffs
> >> will be used, and there is a base price of GBP £10,000. If more than
> >> one Product is included in the same Licence there is usually a
> >> reduction in the second fee."
> >>
> >> For a big company it is fine, but small companies could be a "don't
> go!".
> >>
> > If the cost isn't affordable and the company doesn't want to public all
> > source code, he can try this approach:
> >
> > 1. Enable userfs to move yaffs from kernel to userspace
> > 2. Build yaffs as wasm module and run it in:
> > https://github.com/apache/nuttx-apps/tree/master/interpreters/wamr
> >
> > This technology could satisfy all GPL requirements but without
> > exposing your private source code.
> >
> >
> >> BR,
> >>
> >> Alan
> >>
> >> On 3/1/23, Alan C. Assis <ac...@gmail.com> wrote:
> >>> Hi Xiang,
> >>>
> >>> On 3/1/23, Xiang Xiao <xi...@gmail.com> wrote:
> >>>> On Wed, Mar 1, 2023 at 8:43 PM Alan C. Assis <ac...@gmail.com>
> wrote:
> >>>>
> >>>>> Hi Xiang Xiao,
> >>>>>
> >>>>> This is a great news, but I'm afraid yaffs is not a good option for
> >>>>> everybody because it uses GPL license.
> >>>>>
> >>>>>
> >>>> Yes, but you can pay some money to switch to a more friendly license.
> >>>>
> >>> Do you have more info about it? Is it a reasonable price for small
> >>> companies (let's say: less than U$ 5k to use as you wish or less than
> >>> U$ 1 per device) ?
> >>>
> >>> BR,
> >>>
> >>> Alan
> >>>
>
Re: W25Nx NAND flash support
Posted by Sebastien Lorquet <se...@lorquet.fr>.
Oh come on you are just trying to circumvent the GPL here
Sebastien
Le 01/03/2023 à 15:50, Xiang Xiao a écrit :
> On Wed, Mar 1, 2023 at 10:45 PM Alan C. Assis <ac...@gmail.com> wrote:
>
>> oh, I just found it:
>>
>> "The fee depends on the number of distinct Products in which Yaffs
>> will be used, and there is a base price of GBP £10,000. If more than
>> one Product is included in the same Licence there is usually a
>> reduction in the second fee."
>>
>> For a big company it is fine, but small companies could be a "don't go!".
>>
> If the cost isn't affordable and the company doesn't want to public all
> source code, he can try this approach:
>
> 1. Enable userfs to move yaffs from kernel to userspace
> 2. Build yaffs as wasm module and run it in:
> https://github.com/apache/nuttx-apps/tree/master/interpreters/wamr
>
> This technology could satisfy all GPL requirements but without
> exposing your private source code.
>
>
>> BR,
>>
>> Alan
>>
>> On 3/1/23, Alan C. Assis <ac...@gmail.com> wrote:
>>> Hi Xiang,
>>>
>>> On 3/1/23, Xiang Xiao <xi...@gmail.com> wrote:
>>>> On Wed, Mar 1, 2023 at 8:43 PM Alan C. Assis <ac...@gmail.com> wrote:
>>>>
>>>>> Hi Xiang Xiao,
>>>>>
>>>>> This is a great news, but I'm afraid yaffs is not a good option for
>>>>> everybody because it uses GPL license.
>>>>>
>>>>>
>>>> Yes, but you can pay some money to switch to a more friendly license.
>>>>
>>> Do you have more info about it? Is it a reasonable price for small
>>> companies (let's say: less than U$ 5k to use as you wish or less than
>>> U$ 1 per device) ?
>>>
>>> BR,
>>>
>>> Alan
>>>
Re: W25Nx NAND flash support
Posted by Gregory Nutt <sp...@gmail.com>.
> If the cost isn't affordable and the company doesn't want to public all
> source code, he can try this approach:
>
> 1. Enable userfs to move yaffs from kernel to userspace
> 2. Build yaffs as wasm module and run it in:
> https://github.com/apache/nuttx-apps/tree/master/interpreters/wamr
>
> This technology could satisfy all GPL requirements but without
> exposing your private source code.
Another option would be to add dynamic registration of file systems
within the OS. Currently in fs/mount/fs_mount.c, the file systems are
in const arrays. But an option could be to very simply add a run-time,
dynamic registration to add file systems to a RAM array.
File systems could then be delivered as kernel modules and they would
register themselves when the kernel module is installed. This capability
is a good new OS feature, regardless of YAFFS.
Re: W25Nx NAND flash support
Posted by Xiang Xiao <xi...@gmail.com>.
On Wed, Mar 1, 2023 at 10:45 PM Alan C. Assis <ac...@gmail.com> wrote:
> oh, I just found it:
>
> "The fee depends on the number of distinct Products in which Yaffs
> will be used, and there is a base price of GBP £10,000. If more than
> one Product is included in the same Licence there is usually a
> reduction in the second fee."
>
> For a big company it is fine, but small companies could be a "don't go!".
>
If the cost isn't affordable and the company doesn't want to public all
source code, he can try this approach:
1. Enable userfs to move yaffs from kernel to userspace
2. Build yaffs as wasm module and run it in:
https://github.com/apache/nuttx-apps/tree/master/interpreters/wamr
This technology could satisfy all GPL requirements but without
exposing your private source code.
> BR,
>
> Alan
>
> On 3/1/23, Alan C. Assis <ac...@gmail.com> wrote:
> > Hi Xiang,
> >
> > On 3/1/23, Xiang Xiao <xi...@gmail.com> wrote:
> >> On Wed, Mar 1, 2023 at 8:43 PM Alan C. Assis <ac...@gmail.com> wrote:
> >>
> >>> Hi Xiang Xiao,
> >>>
> >>> This is a great news, but I'm afraid yaffs is not a good option for
> >>> everybody because it uses GPL license.
> >>>
> >>>
> >> Yes, but you can pay some money to switch to a more friendly license.
> >>
> >
> > Do you have more info about it? Is it a reasonable price for small
> > companies (let's say: less than U$ 5k to use as you wish or less than
> > U$ 1 per device) ?
> >
> > BR,
> >
> > Alan
> >
>
Re: W25Nx NAND flash support
Posted by "Alan C. Assis" <ac...@gmail.com>.
Hi Sebastien,
I think having YAFFS2 support on mainline will be very good.
Companies are free to decide if they want to use GPL code or not.
Many companies don't want to use it, but for some it is completely fine.
Xiang, I'm looking forward to test it. Please just put it dependent of
user enabling the GPL License agreement.
BR,
Alan
On 3/1/23, Sebastien Lorquet <se...@lorquet.fr> wrote:
> I am not confortable with NuttX having features that you have to pay for.
>
> In this case yaffs should be provided as an external patch, but should
> not be in mainline.
>
> At the very least such features should not be advertised on public
> websites.
>
> That is just my opinion but I find it good to bring these objections to
> the light of the mailing list so these can be discussed widely.
>
> Sebastien
>
> Le 01/03/2023 à 15:44, Alan C. Assis a écrit :
>> oh, I just found it:
>>
>> "The fee depends on the number of distinct Products in which Yaffs
>> will be used, and there is a base price of GBP £10,000. If more than
>> one Product is included in the same Licence there is usually a
>> reduction in the second fee."
>>
>> For a big company it is fine, but small companies could be a "don't go!".
>>
>> BR,
>>
>> Alan
>>
>> On 3/1/23, Alan C. Assis <ac...@gmail.com> wrote:
>>> Hi Xiang,
>>>
>>> On 3/1/23, Xiang Xiao <xi...@gmail.com> wrote:
>>>> On Wed, Mar 1, 2023 at 8:43 PM Alan C. Assis <ac...@gmail.com> wrote:
>>>>
>>>>> Hi Xiang Xiao,
>>>>>
>>>>> This is a great news, but I'm afraid yaffs is not a good option for
>>>>> everybody because it uses GPL license.
>>>>>
>>>>>
>>>> Yes, but you can pay some money to switch to a more friendly license.
>>>>
>>> Do you have more info about it? Is it a reasonable price for small
>>> companies (let's say: less than U$ 5k to use as you wish or less than
>>> U$ 1 per device) ?
>>>
>>> BR,
>>>
>>> Alan
>>>
>
Re: W25Nx NAND flash support
Posted by Sebastien Lorquet <se...@lorquet.fr>.
I am not confortable with NuttX having features that you have to pay for.
In this case yaffs should be provided as an external patch, but should
not be in mainline.
At the very least such features should not be advertised on public websites.
That is just my opinion but I find it good to bring these objections to
the light of the mailing list so these can be discussed widely.
Sebastien
Le 01/03/2023 à 15:44, Alan C. Assis a écrit :
> oh, I just found it:
>
> "The fee depends on the number of distinct Products in which Yaffs
> will be used, and there is a base price of GBP £10,000. If more than
> one Product is included in the same Licence there is usually a
> reduction in the second fee."
>
> For a big company it is fine, but small companies could be a "don't go!".
>
> BR,
>
> Alan
>
> On 3/1/23, Alan C. Assis <ac...@gmail.com> wrote:
>> Hi Xiang,
>>
>> On 3/1/23, Xiang Xiao <xi...@gmail.com> wrote:
>>> On Wed, Mar 1, 2023 at 8:43 PM Alan C. Assis <ac...@gmail.com> wrote:
>>>
>>>> Hi Xiang Xiao,
>>>>
>>>> This is a great news, but I'm afraid yaffs is not a good option for
>>>> everybody because it uses GPL license.
>>>>
>>>>
>>> Yes, but you can pay some money to switch to a more friendly license.
>>>
>> Do you have more info about it? Is it a reasonable price for small
>> companies (let's say: less than U$ 5k to use as you wish or less than
>> U$ 1 per device) ?
>>
>> BR,
>>
>> Alan
>>
Re: W25Nx NAND flash support
Posted by "Alan C. Assis" <ac...@gmail.com>.
oh, I just found it:
"The fee depends on the number of distinct Products in which Yaffs
will be used, and there is a base price of GBP £10,000. If more than
one Product is included in the same Licence there is usually a
reduction in the second fee."
For a big company it is fine, but small companies could be a "don't go!".
BR,
Alan
On 3/1/23, Alan C. Assis <ac...@gmail.com> wrote:
> Hi Xiang,
>
> On 3/1/23, Xiang Xiao <xi...@gmail.com> wrote:
>> On Wed, Mar 1, 2023 at 8:43 PM Alan C. Assis <ac...@gmail.com> wrote:
>>
>>> Hi Xiang Xiao,
>>>
>>> This is a great news, but I'm afraid yaffs is not a good option for
>>> everybody because it uses GPL license.
>>>
>>>
>> Yes, but you can pay some money to switch to a more friendly license.
>>
>
> Do you have more info about it? Is it a reasonable price for small
> companies (let's say: less than U$ 5k to use as you wish or less than
> U$ 1 per device) ?
>
> BR,
>
> Alan
>
Re: W25Nx NAND flash support
Posted by "Alan C. Assis" <ac...@gmail.com>.
Hi Xiang,
On 3/1/23, Xiang Xiao <xi...@gmail.com> wrote:
> On Wed, Mar 1, 2023 at 8:43 PM Alan C. Assis <ac...@gmail.com> wrote:
>
>> Hi Xiang Xiao,
>>
>> This is a great news, but I'm afraid yaffs is not a good option for
>> everybody because it uses GPL license.
>>
>>
> Yes, but you can pay some money to switch to a more friendly license.
>
Do you have more info about it? Is it a reasonable price for small
companies (let's say: less than U$ 5k to use as you wish or less than
U$ 1 per device) ?
BR,
Alan
Re: W25Nx NAND flash support
Posted by Xiang Xiao <xi...@gmail.com>.
On Wed, Mar 1, 2023 at 10:44 PM Sebastien Lorquet <se...@lorquet.fr>
wrote:
> Also, sorry for the multiple messages, but why is that not new IOCTL
> calls (used to "Support other, less frequently used commands") so the
> structure is not modified ?
>
>
Because the design mimic Linux mtd interface:
https://github.com/torvalds/linux/blob/master/include/linux/mtd/mtd.h#L352-L353
> That would be a much more compatible way of doing things. Devices that
> dont implement this would just automatically return -ENOSYS or -ENXIO
> (cant remember).
>
>
Yes, it could be. It's better to comment in the PR. before merging. But,
anyway you can provide a patch.
> Sebastien
>
> Le 01/03/2023 à 15:41, Sebastien Lorquet a écrit :
> >
> > the callbacks are not activated by conditional compilation and are in
> > the middle of struct mtd_dev_s , so how are these optional?
> >
>
Yes, it's betteer to move to the end of struct.
> > Sebastien
> >
> > Le 01/03/2023 à 15:36, Xiang Xiao a écrit :
> >>
> >>
> >> On Wed, Mar 1, 2023 at 10:26 PM Sebastien Lorquet
> >> <se...@lorquet.fr> wrote:
> >>
> >> Hi,
> >>
> >> Please dont break the mtd interface, make it compatible with the
> >> previous one.
> >>
> >>
> >> The new callbacks are optional:
> >> https://github.com/apache/nuttx/pull/8683.
> >>
> >>
> >> Sebastien
> >>
> >> Le 01/03/2023 à 14:53, Xiang Xiao a écrit :
> >> > On Wed, Mar 1, 2023 at 8:43 PM Alan C. Assis
> >> <ac...@gmail.com> wrote:
> >> >
> >> >> Hi Xiang Xiao,
> >> >>
> >> >> This is a great news, but I'm afraid yaffs is not a good
> >> option for
> >> >> everybody because it uses GPL license.
> >> >>
> >> >>
> >> > Yes, but you can pay some money to switch to a more friendly
> >> license.
> >> >
> >> >
> >> >> But yes, it could be an option for people and companies that can
> >> >> release all their software as open-source. PX4 project for
> example
> >> >> could use it.
> >> >>
> >> >>
> >> > The enhancement in mtd_s is decoupled from yaffs, actually the new
> >> > interface follows Linux MTD model.
> >> > BTW, We plan to improve LittleFS to utilize the new interface.
> >> >
> >> >
> >> >> BR,
> >> >>
> >> >> Alan
> >> >>
> >> >> On 2/28/23, Xiang Xiao <xi...@gmail.com> wrote:
> >> >>> We are working on yaffs porting, mtd_s need a little bit of
> >> extension to
> >> >>> support NAND flash.
> >> >>> Here is the patch: https://github.com/apache/nuttx/pull/8670
> >> >>> LittleFS needs some modification to get the benefit.
> >> >>>
> >> >>> On Sun, Feb 26, 2023 at 10:14 PM Nathan Hartman <
> >> >> hartman.nathan@gmail.com>
> >> >>> wrote:
> >> >>>
> >> >>>> On Sat, Feb 25, 2023 at 8:53 AM Gregory Nutt
> >> <sp...@gmail.com>
> >> >> wrote:
> >> >>>>>> Maybe LittleFS or SmartFS could be extended to handle NAND.
> >> >>>>> I believe that LittleFS has been used with NAND with a NAND
> >> FLASH
> >> >>>>> translation layer. I am not sure of the state of that
> >> work, but I
> >> >> know
> >> >>>>> that people have tried it. Google "LittleFS on NAND flash"
> >> and see
> >> >> the
> >> >>>>> hits.
> >> >>>>>
> >> >>>>> If a NAND flash translation layer (NFTL) were ported to
> >> NuttX, then
> >> >> you
> >> >>>>> should be able to use almost any file system, although some
> >> would be
> >> >>>>> extremely inefficient with NAND. The NAND flash layer would
> >> need to
> >> >> do
> >> >>>>> sparing, wear-leveling, ECC calculations, etc. as needed by
> >> NAND.
> >> >>>>>
> >> >>>>> I have used NXFFS with NAND and it /almost /works.
> >> >>>>>
> >> >>>>> The basic filesystem issue is that all available
> >> filesystems need to
> >> >> do
> >> >>>>> read-modify-write operations on flash sections. And they
> >> assume that
> >> >>>>> you can always write a '1' bit to a '0' bit. SmartFS and
> >> NXFFS both
> >> >>>>> assume this behavior explicitly. It is a good assumption
> >> with NOR
> >> >>>>> flash. The problem is that NAND can only be written a
> >> whole sector at
> >> >>>>> a
> >> >>>>> time. This is because the error correction coding (ECC):
> >> If you
> >> >>>>> change
> >> >>>>> even one bit in the NAND sector, you have to re-write the
> >> whole sector
> >> >>>>> with new ECC (because you can't change the '0' bits in the
> >> ECC back
> >> >>>>> into
> >> >>>>> '1''s without erasing the sector).
> >> >>>>>
> >> >>>>> Since SmartFS and NXFFS both do bit twiddling in the
> >> sectors they
> >> >> would
> >> >>>>> be very inefficient with an NFTL: Each bit twiddle would
> >> cause a
> >> >> whole
> >> >>>>> sector re-write. The same is probably true for LittleFS.
> >> Other file
> >> >>>>> systems like FAT could also be used if there were an NFTL,
> >> however,
> >> >> FAT
> >> >>>>> would also be inefficient (each directory update or FAT
> >> update and
> >> >> each
> >> >>>>> file data size change would cause another sector write).
> >> >>>>>
> >> >>>>> So the only real solution to support NAND efficiently is
> >> use a file
> >> >>>>> system that is designed specifically around the
> >> peculiarities of NAND.
> >> >>>>> That would require some research (Alan has suggested a
> >> couple of
> >> >> places
> >> >>>>> to start).
> >> >>>>
> >> >>>>
> >> >>>> I seem to recall reading that in NAND flash, one of the
> >> challenges is
> >> >>>> that
> >> >>>> modifying a bit causes a disturbance that can corrupt other
> >> bits, even
> >> >> in
> >> >>>> unrelated sectors, and that even reading can have this
> >> effect. The
> >> >>>> disturbance only affects bits a little at a time, but after
> >> some number
> >> >>>> of
> >> >>>> accesses, the affected sectors need to be written to new
> >> sectors,
> >> >> keeping
> >> >>>> track of bad sectors formed in the process, to avoid data
> >> loss. This may
> >> >>>> lead to a cascade effect in which reading a sector may cause
> >> numerous
> >> >>>> sectors to be re-written. This phenomenon is called write
> >> amplification.
> >> >>>> The takeaway from a usage perspective is that you want to
> >> over-provision
> >> >>>> the amount of NAND flash you install, to leave plenty of
> >> room for all
> >> >>>> this
> >> >>>> swapping, and plenty of extra sectors to increase the amount
> >> of accesses
> >> >>>> before failure. The greater the over-provisioning, the
> >> longer until
> >> >>>> failure.
> >> >>>>
> >> >>>> I haven't tried yet, but I would like at some point to support
> a
> >> >> micro-SD
> >> >>>> and/or micro-MMC card as the backing store and use one that
> >> has built in
> >> >>>> hardware wear leveling, with some appropriate file system of
> >> course.
> >> >> This
> >> >>>> may add $50 or more to the bill of materials depending on
> >> the card
> >> >>>> capacity, but makes it possible for users to replace the
> >> card if it
> >> >>>> fails,
> >> >>>> in contrast to soldered-on flash chips, which can be
> >> replaced too, but
> >> >>>> only
> >> >>>> with the appropriate tools and skills.
> >> >>>>
> >> >>>> Cheers
> >> >>>> Nathan
> >> >>>>
> >>
Re: W25Nx NAND flash support
Posted by Sebastien Lorquet <se...@lorquet.fr>.
Also, sorry for the multiple messages, but why is that not new IOCTL
calls (used to "Support other, less frequently used commands") so the
structure is not modified ?
That would be a much more compatible way of doing things. Devices that
dont implement this would just automatically return -ENOSYS or -ENXIO
(cant remember).
Sebastien
Le 01/03/2023 à 15:41, Sebastien Lorquet a écrit :
>
> the callbacks are not activated by conditional compilation and are in
> the middle of struct mtd_dev_s , so how are these optional?
>
> Sebastien
>
> Le 01/03/2023 à 15:36, Xiang Xiao a écrit :
>>
>>
>> On Wed, Mar 1, 2023 at 10:26 PM Sebastien Lorquet
>> <se...@lorquet.fr> wrote:
>>
>> Hi,
>>
>> Please dont break the mtd interface, make it compatible with the
>> previous one.
>>
>>
>> The new callbacks are optional:
>> https://github.com/apache/nuttx/pull/8683.
>>
>>
>> Sebastien
>>
>> Le 01/03/2023 à 14:53, Xiang Xiao a écrit :
>> > On Wed, Mar 1, 2023 at 8:43 PM Alan C. Assis
>> <ac...@gmail.com> wrote:
>> >
>> >> Hi Xiang Xiao,
>> >>
>> >> This is a great news, but I'm afraid yaffs is not a good
>> option for
>> >> everybody because it uses GPL license.
>> >>
>> >>
>> > Yes, but you can pay some money to switch to a more friendly
>> license.
>> >
>> >
>> >> But yes, it could be an option for people and companies that can
>> >> release all their software as open-source. PX4 project for example
>> >> could use it.
>> >>
>> >>
>> > The enhancement in mtd_s is decoupled from yaffs, actually the new
>> > interface follows Linux MTD model.
>> > BTW, We plan to improve LittleFS to utilize the new interface.
>> >
>> >
>> >> BR,
>> >>
>> >> Alan
>> >>
>> >> On 2/28/23, Xiang Xiao <xi...@gmail.com> wrote:
>> >>> We are working on yaffs porting, mtd_s need a little bit of
>> extension to
>> >>> support NAND flash.
>> >>> Here is the patch: https://github.com/apache/nuttx/pull/8670
>> >>> LittleFS needs some modification to get the benefit.
>> >>>
>> >>> On Sun, Feb 26, 2023 at 10:14 PM Nathan Hartman <
>> >> hartman.nathan@gmail.com>
>> >>> wrote:
>> >>>
>> >>>> On Sat, Feb 25, 2023 at 8:53 AM Gregory Nutt
>> <sp...@gmail.com>
>> >> wrote:
>> >>>>>> Maybe LittleFS or SmartFS could be extended to handle NAND.
>> >>>>> I believe that LittleFS has been used with NAND with a NAND
>> FLASH
>> >>>>> translation layer. I am not sure of the state of that
>> work, but I
>> >> know
>> >>>>> that people have tried it. Google "LittleFS on NAND flash"
>> and see
>> >> the
>> >>>>> hits.
>> >>>>>
>> >>>>> If a NAND flash translation layer (NFTL) were ported to
>> NuttX, then
>> >> you
>> >>>>> should be able to use almost any file system, although some
>> would be
>> >>>>> extremely inefficient with NAND. The NAND flash layer would
>> need to
>> >> do
>> >>>>> sparing, wear-leveling, ECC calculations, etc. as needed by
>> NAND.
>> >>>>>
>> >>>>> I have used NXFFS with NAND and it /almost /works.
>> >>>>>
>> >>>>> The basic filesystem issue is that all available
>> filesystems need to
>> >> do
>> >>>>> read-modify-write operations on flash sections. And they
>> assume that
>> >>>>> you can always write a '1' bit to a '0' bit. SmartFS and
>> NXFFS both
>> >>>>> assume this behavior explicitly. It is a good assumption
>> with NOR
>> >>>>> flash. The problem is that NAND can only be written a
>> whole sector at
>> >>>>> a
>> >>>>> time. This is because the error correction coding (ECC):
>> If you
>> >>>>> change
>> >>>>> even one bit in the NAND sector, you have to re-write the
>> whole sector
>> >>>>> with new ECC (because you can't change the '0' bits in the
>> ECC back
>> >>>>> into
>> >>>>> '1''s without erasing the sector).
>> >>>>>
>> >>>>> Since SmartFS and NXFFS both do bit twiddling in the
>> sectors they
>> >> would
>> >>>>> be very inefficient with an NFTL: Each bit twiddle would
>> cause a
>> >> whole
>> >>>>> sector re-write. The same is probably true for LittleFS.
>> Other file
>> >>>>> systems like FAT could also be used if there were an NFTL,
>> however,
>> >> FAT
>> >>>>> would also be inefficient (each directory update or FAT
>> update and
>> >> each
>> >>>>> file data size change would cause another sector write).
>> >>>>>
>> >>>>> So the only real solution to support NAND efficiently is
>> use a file
>> >>>>> system that is designed specifically around the
>> peculiarities of NAND.
>> >>>>> That would require some research (Alan has suggested a
>> couple of
>> >> places
>> >>>>> to start).
>> >>>>
>> >>>>
>> >>>> I seem to recall reading that in NAND flash, one of the
>> challenges is
>> >>>> that
>> >>>> modifying a bit causes a disturbance that can corrupt other
>> bits, even
>> >> in
>> >>>> unrelated sectors, and that even reading can have this
>> effect. The
>> >>>> disturbance only affects bits a little at a time, but after
>> some number
>> >>>> of
>> >>>> accesses, the affected sectors need to be written to new
>> sectors,
>> >> keeping
>> >>>> track of bad sectors formed in the process, to avoid data
>> loss. This may
>> >>>> lead to a cascade effect in which reading a sector may cause
>> numerous
>> >>>> sectors to be re-written. This phenomenon is called write
>> amplification.
>> >>>> The takeaway from a usage perspective is that you want to
>> over-provision
>> >>>> the amount of NAND flash you install, to leave plenty of
>> room for all
>> >>>> this
>> >>>> swapping, and plenty of extra sectors to increase the amount
>> of accesses
>> >>>> before failure. The greater the over-provisioning, the
>> longer until
>> >>>> failure.
>> >>>>
>> >>>> I haven't tried yet, but I would like at some point to support a
>> >> micro-SD
>> >>>> and/or micro-MMC card as the backing store and use one that
>> has built in
>> >>>> hardware wear leveling, with some appropriate file system of
>> course.
>> >> This
>> >>>> may add $50 or more to the bill of materials depending on
>> the card
>> >>>> capacity, but makes it possible for users to replace the
>> card if it
>> >>>> fails,
>> >>>> in contrast to soldered-on flash chips, which can be
>> replaced too, but
>> >>>> only
>> >>>> with the appropriate tools and skills.
>> >>>>
>> >>>> Cheers
>> >>>> Nathan
>> >>>>
>>
Re: W25Nx NAND flash support
Posted by Sebastien Lorquet <se...@lorquet.fr>.
the callbacks are not activated by conditional compilation and are in
the middle of struct mtd_dev_s , so how are these optional?
Sebastien
Le 01/03/2023 à 15:36, Xiang Xiao a écrit :
>
>
> On Wed, Mar 1, 2023 at 10:26 PM Sebastien Lorquet
> <se...@lorquet.fr> wrote:
>
> Hi,
>
> Please dont break the mtd interface, make it compatible with the
> previous one.
>
>
> The new callbacks are optional: https://github.com/apache/nuttx/pull/8683.
>
>
> Sebastien
>
> Le 01/03/2023 à 14:53, Xiang Xiao a écrit :
> > On Wed, Mar 1, 2023 at 8:43 PM Alan C. Assis <ac...@gmail.com>
> wrote:
> >
> >> Hi Xiang Xiao,
> >>
> >> This is a great news, but I'm afraid yaffs is not a good option for
> >> everybody because it uses GPL license.
> >>
> >>
> > Yes, but you can pay some money to switch to a more friendly
> license.
> >
> >
> >> But yes, it could be an option for people and companies that can
> >> release all their software as open-source. PX4 project for example
> >> could use it.
> >>
> >>
> > The enhancement in mtd_s is decoupled from yaffs, actually the new
> > interface follows Linux MTD model.
> > BTW, We plan to improve LittleFS to utilize the new interface.
> >
> >
> >> BR,
> >>
> >> Alan
> >>
> >> On 2/28/23, Xiang Xiao <xi...@gmail.com> wrote:
> >>> We are working on yaffs porting, mtd_s need a little bit of
> extension to
> >>> support NAND flash.
> >>> Here is the patch: https://github.com/apache/nuttx/pull/8670
> >>> LittleFS needs some modification to get the benefit.
> >>>
> >>> On Sun, Feb 26, 2023 at 10:14 PM Nathan Hartman <
> >> hartman.nathan@gmail.com>
> >>> wrote:
> >>>
> >>>> On Sat, Feb 25, 2023 at 8:53 AM Gregory Nutt
> <sp...@gmail.com>
> >> wrote:
> >>>>>> Maybe LittleFS or SmartFS could be extended to handle NAND.
> >>>>> I believe that LittleFS has been used with NAND with a NAND
> FLASH
> >>>>> translation layer. I am not sure of the state of that work,
> but I
> >> know
> >>>>> that people have tried it. Google "LittleFS on NAND flash"
> and see
> >> the
> >>>>> hits.
> >>>>>
> >>>>> If a NAND flash translation layer (NFTL) were ported to
> NuttX, then
> >> you
> >>>>> should be able to use almost any file system, although some
> would be
> >>>>> extremely inefficient with NAND. The NAND flash layer would
> need to
> >> do
> >>>>> sparing, wear-leveling, ECC calculations, etc. as needed by
> NAND.
> >>>>>
> >>>>> I have used NXFFS with NAND and it /almost /works.
> >>>>>
> >>>>> The basic filesystem issue is that all available filesystems
> need to
> >> do
> >>>>> read-modify-write operations on flash sections. And they
> assume that
> >>>>> you can always write a '1' bit to a '0' bit. SmartFS and
> NXFFS both
> >>>>> assume this behavior explicitly. It is a good assumption
> with NOR
> >>>>> flash. The problem is that NAND can only be written a whole
> sector at
> >>>>> a
> >>>>> time. This is because the error correction coding (ECC):
> If you
> >>>>> change
> >>>>> even one bit in the NAND sector, you have to re-write the
> whole sector
> >>>>> with new ECC (because you can't change the '0' bits in the
> ECC back
> >>>>> into
> >>>>> '1''s without erasing the sector).
> >>>>>
> >>>>> Since SmartFS and NXFFS both do bit twiddling in the sectors
> they
> >> would
> >>>>> be very inefficient with an NFTL: Each bit twiddle would cause a
> >> whole
> >>>>> sector re-write. The same is probably true for LittleFS.
> Other file
> >>>>> systems like FAT could also be used if there were an NFTL,
> however,
> >> FAT
> >>>>> would also be inefficient (each directory update or FAT
> update and
> >> each
> >>>>> file data size change would cause another sector write).
> >>>>>
> >>>>> So the only real solution to support NAND efficiently is use
> a file
> >>>>> system that is designed specifically around the
> peculiarities of NAND.
> >>>>> That would require some research (Alan has suggested a couple of
> >> places
> >>>>> to start).
> >>>>
> >>>>
> >>>> I seem to recall reading that in NAND flash, one of the
> challenges is
> >>>> that
> >>>> modifying a bit causes a disturbance that can corrupt other
> bits, even
> >> in
> >>>> unrelated sectors, and that even reading can have this
> effect. The
> >>>> disturbance only affects bits a little at a time, but after
> some number
> >>>> of
> >>>> accesses, the affected sectors need to be written to new sectors,
> >> keeping
> >>>> track of bad sectors formed in the process, to avoid data
> loss. This may
> >>>> lead to a cascade effect in which reading a sector may cause
> numerous
> >>>> sectors to be re-written. This phenomenon is called write
> amplification.
> >>>> The takeaway from a usage perspective is that you want to
> over-provision
> >>>> the amount of NAND flash you install, to leave plenty of room
> for all
> >>>> this
> >>>> swapping, and plenty of extra sectors to increase the amount
> of accesses
> >>>> before failure. The greater the over-provisioning, the longer
> until
> >>>> failure.
> >>>>
> >>>> I haven't tried yet, but I would like at some point to support a
> >> micro-SD
> >>>> and/or micro-MMC card as the backing store and use one that
> has built in
> >>>> hardware wear leveling, with some appropriate file system of
> course.
> >> This
> >>>> may add $50 or more to the bill of materials depending on the
> card
> >>>> capacity, but makes it possible for users to replace the card
> if it
> >>>> fails,
> >>>> in contrast to soldered-on flash chips, which can be replaced
> too, but
> >>>> only
> >>>> with the appropriate tools and skills.
> >>>>
> >>>> Cheers
> >>>> Nathan
> >>>>
>
Re: W25Nx NAND flash support
Posted by Sebastien Lorquet <se...@lorquet.fr>.
Hi,
Please dont break the mtd interface, make it compatible with the
previous one.
Sebastien
Le 01/03/2023 à 14:53, Xiang Xiao a écrit :
> On Wed, Mar 1, 2023 at 8:43 PM Alan C. Assis <ac...@gmail.com> wrote:
>
>> Hi Xiang Xiao,
>>
>> This is a great news, but I'm afraid yaffs is not a good option for
>> everybody because it uses GPL license.
>>
>>
> Yes, but you can pay some money to switch to a more friendly license.
>
>
>> But yes, it could be an option for people and companies that can
>> release all their software as open-source. PX4 project for example
>> could use it.
>>
>>
> The enhancement in mtd_s is decoupled from yaffs, actually the new
> interface follows Linux MTD model.
> BTW, We plan to improve LittleFS to utilize the new interface.
>
>
>> BR,
>>
>> Alan
>>
>> On 2/28/23, Xiang Xiao <xi...@gmail.com> wrote:
>>> We are working on yaffs porting, mtd_s need a little bit of extension to
>>> support NAND flash.
>>> Here is the patch: https://github.com/apache/nuttx/pull/8670
>>> LittleFS needs some modification to get the benefit.
>>>
>>> On Sun, Feb 26, 2023 at 10:14 PM Nathan Hartman <
>> hartman.nathan@gmail.com>
>>> wrote:
>>>
>>>> On Sat, Feb 25, 2023 at 8:53 AM Gregory Nutt <sp...@gmail.com>
>> wrote:
>>>>>> Maybe LittleFS or SmartFS could be extended to handle NAND.
>>>>> I believe that LittleFS has been used with NAND with a NAND FLASH
>>>>> translation layer. I am not sure of the state of that work, but I
>> know
>>>>> that people have tried it. Google "LittleFS on NAND flash" and see
>> the
>>>>> hits.
>>>>>
>>>>> If a NAND flash translation layer (NFTL) were ported to NuttX, then
>> you
>>>>> should be able to use almost any file system, although some would be
>>>>> extremely inefficient with NAND. The NAND flash layer would need to
>> do
>>>>> sparing, wear-leveling, ECC calculations, etc. as needed by NAND.
>>>>>
>>>>> I have used NXFFS with NAND and it /almost /works.
>>>>>
>>>>> The basic filesystem issue is that all available filesystems need to
>> do
>>>>> read-modify-write operations on flash sections. And they assume that
>>>>> you can always write a '1' bit to a '0' bit. SmartFS and NXFFS both
>>>>> assume this behavior explicitly. It is a good assumption with NOR
>>>>> flash. The problem is that NAND can only be written a whole sector at
>>>>> a
>>>>> time. This is because the error correction coding (ECC): If you
>>>>> change
>>>>> even one bit in the NAND sector, you have to re-write the whole sector
>>>>> with new ECC (because you can't change the '0' bits in the ECC back
>>>>> into
>>>>> '1''s without erasing the sector).
>>>>>
>>>>> Since SmartFS and NXFFS both do bit twiddling in the sectors they
>> would
>>>>> be very inefficient with an NFTL: Each bit twiddle would cause a
>> whole
>>>>> sector re-write. The same is probably true for LittleFS. Other file
>>>>> systems like FAT could also be used if there were an NFTL, however,
>> FAT
>>>>> would also be inefficient (each directory update or FAT update and
>> each
>>>>> file data size change would cause another sector write).
>>>>>
>>>>> So the only real solution to support NAND efficiently is use a file
>>>>> system that is designed specifically around the peculiarities of NAND.
>>>>> That would require some research (Alan has suggested a couple of
>> places
>>>>> to start).
>>>>
>>>>
>>>> I seem to recall reading that in NAND flash, one of the challenges is
>>>> that
>>>> modifying a bit causes a disturbance that can corrupt other bits, even
>> in
>>>> unrelated sectors, and that even reading can have this effect. The
>>>> disturbance only affects bits a little at a time, but after some number
>>>> of
>>>> accesses, the affected sectors need to be written to new sectors,
>> keeping
>>>> track of bad sectors formed in the process, to avoid data loss. This may
>>>> lead to a cascade effect in which reading a sector may cause numerous
>>>> sectors to be re-written. This phenomenon is called write amplification.
>>>> The takeaway from a usage perspective is that you want to over-provision
>>>> the amount of NAND flash you install, to leave plenty of room for all
>>>> this
>>>> swapping, and plenty of extra sectors to increase the amount of accesses
>>>> before failure. The greater the over-provisioning, the longer until
>>>> failure.
>>>>
>>>> I haven't tried yet, but I would like at some point to support a
>> micro-SD
>>>> and/or micro-MMC card as the backing store and use one that has built in
>>>> hardware wear leveling, with some appropriate file system of course.
>> This
>>>> may add $50 or more to the bill of materials depending on the card
>>>> capacity, but makes it possible for users to replace the card if it
>>>> fails,
>>>> in contrast to soldered-on flash chips, which can be replaced too, but
>>>> only
>>>> with the appropriate tools and skills.
>>>>
>>>> Cheers
>>>> Nathan
>>>>
Re: W25Nx NAND flash support
Posted by Xiang Xiao <xi...@gmail.com>.
On Wed, Mar 1, 2023 at 8:43 PM Alan C. Assis <ac...@gmail.com> wrote:
> Hi Xiang Xiao,
>
> This is a great news, but I'm afraid yaffs is not a good option for
> everybody because it uses GPL license.
>
>
Yes, but you can pay some money to switch to a more friendly license.
> But yes, it could be an option for people and companies that can
> release all their software as open-source. PX4 project for example
> could use it.
>
>
The enhancement in mtd_s is decoupled from yaffs, actually the new
interface follows Linux MTD model.
BTW, We plan to improve LittleFS to utilize the new interface.
> BR,
>
> Alan
>
> On 2/28/23, Xiang Xiao <xi...@gmail.com> wrote:
> > We are working on yaffs porting, mtd_s need a little bit of extension to
> > support NAND flash.
> > Here is the patch: https://github.com/apache/nuttx/pull/8670
> > LittleFS needs some modification to get the benefit.
> >
> > On Sun, Feb 26, 2023 at 10:14 PM Nathan Hartman <
> hartman.nathan@gmail.com>
> > wrote:
> >
> >> On Sat, Feb 25, 2023 at 8:53 AM Gregory Nutt <sp...@gmail.com>
> wrote:
> >>
> >> >
> >> > > Maybe LittleFS or SmartFS could be extended to handle NAND.
> >> >
> >> > I believe that LittleFS has been used with NAND with a NAND FLASH
> >> > translation layer. I am not sure of the state of that work, but I
> know
> >> > that people have tried it. Google "LittleFS on NAND flash" and see
> the
> >> > hits.
> >> >
> >> > If a NAND flash translation layer (NFTL) were ported to NuttX, then
> you
> >> > should be able to use almost any file system, although some would be
> >> > extremely inefficient with NAND. The NAND flash layer would need to
> do
> >> > sparing, wear-leveling, ECC calculations, etc. as needed by NAND.
> >> >
> >> > I have used NXFFS with NAND and it /almost /works.
> >> >
> >> > The basic filesystem issue is that all available filesystems need to
> do
> >> > read-modify-write operations on flash sections. And they assume that
> >> > you can always write a '1' bit to a '0' bit. SmartFS and NXFFS both
> >> > assume this behavior explicitly. It is a good assumption with NOR
> >> > flash. The problem is that NAND can only be written a whole sector at
> >> > a
> >> > time. This is because the error correction coding (ECC): If you
> >> > change
> >> > even one bit in the NAND sector, you have to re-write the whole sector
> >> > with new ECC (because you can't change the '0' bits in the ECC back
> >> > into
> >> > '1''s without erasing the sector).
> >> >
> >> > Since SmartFS and NXFFS both do bit twiddling in the sectors they
> would
> >> > be very inefficient with an NFTL: Each bit twiddle would cause a
> whole
> >> > sector re-write. The same is probably true for LittleFS. Other file
> >> > systems like FAT could also be used if there were an NFTL, however,
> FAT
> >> > would also be inefficient (each directory update or FAT update and
> each
> >> > file data size change would cause another sector write).
> >> >
> >> > So the only real solution to support NAND efficiently is use a file
> >> > system that is designed specifically around the peculiarities of NAND.
> >> > That would require some research (Alan has suggested a couple of
> places
> >> > to start).
> >>
> >>
> >>
> >> I seem to recall reading that in NAND flash, one of the challenges is
> >> that
> >> modifying a bit causes a disturbance that can corrupt other bits, even
> in
> >> unrelated sectors, and that even reading can have this effect. The
> >> disturbance only affects bits a little at a time, but after some number
> >> of
> >> accesses, the affected sectors need to be written to new sectors,
> keeping
> >> track of bad sectors formed in the process, to avoid data loss. This may
> >> lead to a cascade effect in which reading a sector may cause numerous
> >> sectors to be re-written. This phenomenon is called write amplification.
> >> The takeaway from a usage perspective is that you want to over-provision
> >> the amount of NAND flash you install, to leave plenty of room for all
> >> this
> >> swapping, and plenty of extra sectors to increase the amount of accesses
> >> before failure. The greater the over-provisioning, the longer until
> >> failure.
> >>
> >> I haven't tried yet, but I would like at some point to support a
> micro-SD
> >> and/or micro-MMC card as the backing store and use one that has built in
> >> hardware wear leveling, with some appropriate file system of course.
> This
> >> may add $50 or more to the bill of materials depending on the card
> >> capacity, but makes it possible for users to replace the card if it
> >> fails,
> >> in contrast to soldered-on flash chips, which can be replaced too, but
> >> only
> >> with the appropriate tools and skills.
> >>
> >> Cheers
> >> Nathan
> >>
> >
>