Re: Transitioning to a better File API implementation

[Michael Gauthier <mi...@silverorange.com>]

Ian,

With the new URLs will it be possible to write to both /Documents and 
/Library for iOS apps? In the old version, the filesystem root resolved 
as /Documents but it was possible to get to /Library by navigating the 
the parent dir.

Cheers,
Mike


On 2013-11-15 15:19, Ian Clelland wrote:
> On Fri, Nov 15, 2013 at 1:40 PM, purplecabbage <pu...@gmail.com>wrote:
>
>> Considering the magnitude of the changes I would have expected that this
>> was just a new file plugin. The previous version was based on a spec, and
>> if we are deviating from it we should probably maintain both, and possibly
>> even make a recommendation to the w3c.
>> I hope we at least do a major version update for this if APIs have changed.
>>
>
> Yes, definitely a major version bump, at least so that devs aren't caught
> by the URL-format-change.
>
> There aren't any changes to external APIs; I've been very careful to
> maintain conformance with the existing tests, except where those tests have
> been for undocumented implementation details. The only app-visible change
> is that URLs returned by the .toURL() method will look like
>
> filesystem://localhost/persistent/my/file.jpg
>
> rather than
>
> file:///var/mobile/Applications/<app id>/Documents/my/file.jpg
>
> We are still following the spec -- in fact, this brings us closer in line
> with the W3C File API spec, on these points:
>
> http://www.w3.org/TR/file-system-api/#widl-Entry-fullPath -- The fullPath
> of an entry should be the path from the root (of the HTML filesystem, not
> the underlying device file system).
>
> http://www.w3.org/TR/file-system-api/#widl-Entry-toURL-DOMString -- This is
> a note, rather than a requirement, but a filesystem url scheme is mentioned
> there.
>
> We're considering extending the spec in one way, which should be compatible
> with the spirit of the spec, and backwards-compatible with the previous
> API. That is the ability to declare new filesystem types, beyond
> "temporary" and "persistent", so that we can access things like device
> media and app bundle assets using the same APIs that we use for accessing
> user-defined files. If we're happy with the way that works, then we should
> definitely propose it for inclusion in the standard (not the specific
> filesystems, perhaps, but the mechanism for requesting and interacting with
> them)
>
> Ian
>
>
>>
>> Sent from my iPhone
>>
>>> On Nov 15, 2013, at 8:36 AM, Ian Clelland <ic...@chromium.org>
>> wrote:
>>>
>>> I've created CB-5403 as the über-ticket for this; various bits of it are
>> in
>>> subtasks.
>>>
>>> Once I have the tests and the JS updated, then platform owners can take a
>>> look and see whether they have anything to do to support their own
>> schemes.
>>>
>>> As general guidelines, I would say:
>>>
>>> * If you can intercept filesystem://* URLs in native code, and return
>>> arbitrary responses, then do it! It will let your platform support new
>>> filesystems in the future as we come up with them. Add a couple of
>> subtasks
>>> for CB-5403 for your platform and go.
>>>
>>> * If you can't do that, but you *can* provide access to things like media
>>> through special urls, then try that! You should be able to create a
>>> FileSystem object that uses that URL as its root, and everything should
>>> work. Add a subtask for that, and see what you can do.
>>>
>>> * If you can't do that either, and you just want to stick with the
>> file:///
>>> urls that we are currently using, then don't do anything; nothing should
>>> change for you.
>>>
>>>
>>>
>>> On Fri, Nov 15, 2013 at 10:36 AM, Ian Clelland <iclelland@chromium.org
>>> wrote:
>>>
>>>>
>>>> On Fri, Nov 15, 2013 at 10:03 AM, Marcel Kinard <cmarcelk@gmail.com
>>> wrote:
>>>>
>>>>> Ian, will there be changes that app developers will need to do? If so,
>>>>> those should be clearly documented in a migration guide. If not, it
>> sounds
>>>>> like the [improved] tests should pass on both the old and new versions,
>>>>> which would be sweet.
>>>>
>>>> The filesystem URLs themselves will change as a result of this. The
>> tests
>>>> should pass on both old and new versions, but the tests mint their own
>> URLs
>>>> for testing against -- each version is internally consistent, but if an
>> app
>>>> is saving internal URLs and tries to dereference an old (file:///) url
>> with
>>>> the new plugin, it will probably not resolve.
>>>>
>>>> Maybe there's a transition path here -- it might be possible to allow
>>>> window.resolveLocalFileSystemURL to access files with the old URL, but
>>>> never actually generate URLs.
>>>>
>>>> That would mean that
>>>>
>>>> (resolveLocalFileSystemURL(a)).toURL() !== a,
>>>>
>>>> but I don't think that we depend on that as an identity.
>>>>
>>>> Is there work which needs to be done on the other platforms (BB, WP8,
>>>>> Win8, FFOS, etc) so that those platforms don't get left behind? If so,
>>>>> would it make sense to reach out to those platform owners and at least
>> get
>>>>> Jira items created with some notes?
>>>>
>>>> Yes. I'm going to create a ticket for this, and we can add
>>>> platform-specific tasks to it.
>>>>
>>>> Other platforms shouldn't *have* to do anything, and some platforms
>>>> *can't* do anything, because they cannot access anything other than
>>>> file:/// urls anyway.
>>>>
>>>> However, if the other platforms want to get in on the goodness, and
>> start
>>>> supporting their own URL schemes (I think the BB folks can use something
>>>> like local:// for their filesystem access), then that'll be the place to
>>>> keep everything organized.
>>>>
>>>> I'll post back once I have that issue set up.
>>>>
>>>> Ian
>>>>
>>>>
>>>>> Sounds like you've built some very significant improvements. Thanks!
>>>>>
>>>>>> On Nov 15, 2013, at 9:29 AM, Ian Clelland <ic...@chromium.org>
>> wrote:
>>>>>>
>>>>>> After working with the File plugin for a week or so, I've gotten it
>>>>> more or
>>>>>> less where I want it to be on iOS, and am working through Android.
>>>>> Looking
>>>>>> at the end result, though, I expect that over 90% of the code has been
>>>>>> touched in some way. (It's a huge diff.)
>>>>
>>>>
>>
>

Re: Transitioning to a better File API implementation

[Shazron <sh...@gmail.com>]

Reminds me of the iOS LocalStorage (CDVLocalStorage plugin) bs that needed
to be handled - we definitely need a migration path (automatic hopefully)
if not users are in for a world of hurt...


On Wed, Dec 11, 2013 at 11:33 AM, Tommy Williams <to...@devgeeks.org> wrote:

> +1
> On 12/12/2013 6:26 am, "Ian Clelland" <ic...@chromium.org> wrote:
>
> > Yeah, it would definitely require some kind of migration support. Not
> > suggesting that this is something that we ever actually do, but we could
> > give developers a bit of code that automatically looks in both places,
> and
> > moves files to the new location on open. Or we do it under a flag that is
> > off for existing apps (off-on-upgrade) and only enabled-by-default for
> new
> > apps.
> >
> > I agree with you: this could cause worlds of pain, with developers, and
> > worse with users. We shouldn't take any steps in that direction without a
> > lot of very careful thought. For now, /Documents, sub-optimal though it
> may
> > be, is where Cordova puts persistent files.
> >
> > Ian
> >
> > On Wed, Dec 11, 2013 at 2:18 PM, Tommy Williams <to...@devgeeks.org>
> > wrote:
> >
> > > Changing where PERSISTENT is located sounds like a very very bad idea.
> > >
> > > I know that would cause me personally a lot of grief with existing user
> > > data.
> > > On 11/12/2013 8:58 am, "Ian Clelland" <ic...@chromium.org> wrote:
> > >
> > > > On Tue, Dec 10, 2013 at 4:37 PM, Michael Gauthier <
> > mike@silverorange.com
> > > > >wrote:
> > > >
> > > > > On 10/12/13 05:02 PM, Ian Clelland wrote:
> > > > >
> > > > >> Yep.
> > > > >>
> > > > >> I think that Library is the more natural place for the HTML
> > persistent
> > > > >> filesystem, since it's used by an app for whatever persistent
> > storage
> > > it
> > > > >> needs, without exposing all of it's implementation details to the
> > > user.
> > > > >> (There's lots of room for debate on this, I'm sure)
> > > > >>
> > > > >> The trouble is that we've historically used /Documents for
> > persistent
> > > > >> storage, and changing that will break apps.
> > > > >>
> > > > >>  I'm fine with a BC break, but I don't have to maintain any legacy
> > > > > applications. /Library does make more sense as the default for
> > > > PERSISTENT.
> > > > > The big problem with BC is for installed apps with existing data on
> > the
> > > > > filesystem, right?
> > > >
> > > >
> > > > Exactly. I'd hate for developers to update their plugins, and push a
> > new
> > > > version of their app that seems fine -- everything works great when
> you
> > > > start from scratch -- but users with existing data find themselves
> > > > completely locked out of it.
> > > >
> > > >
> > > > >
> > > > >  One idea is to allow something like requestFilesystem(DOCUMENT),
> in
> > > > >> addition to PERSISTENT and TEMPORARY. Another suggestion has been
> to
> > > add
> > > > >> extra arguments -- hints -- such as "{sync: true}", or maybe in
> this
> > > > case
> > > > >> "{purpose: documents}" to specify the attributes of the filesystem
> > > that
> > > > is
> > > > >> returned.
> > > > >>
> > > > > >
> > > > > {sync: true} is a bit tricky because /Library/Cache is not synced
> but
> > > > > /Library/Application Data is synced. Having a DOCUMENT type would
> > match
> > > > > /tmp and /Library as the top-level dirs mapping to file-system
> > > constants.
> > > > >
> > > > > All my feedback is only from the iOS perspective though. Not sure
> if
> > > > these
> > > > > ideas make other platforms more or less complicated.
> > > > >
> > > > > Cheers,
> > > > > Mike
> > > > >
> > > > >
> > > > >>
> > > > >>
> > > > >> On Tue, Dec 10, 2013 at 2:39 PM, Michael Gauthier <
> > > > mike@silverorange.com
> > > > >> >wrote:
> > > > >>
> > > > >>  Hmm.. The two directories have different defined roles on iOS and
> > > both
> > > > >>> are
> > > > >>> normal targets for applications to read/write files. Library is
> for
> > > > cache
> > > > >>> data or app-specific resources. Documents is for saving/loading
> > > actual
> > > > >>> things created by users within apps.
> > > > >>>
> > > > >>> See https://developer.apple.com/library/ios/documentation/
> > > > >>>
> > > >
> > FileManagement/Conceptual/FileSystemProgrammingGuide/FileSystemOverview/
> > > > >>> FileSystemOverview.html#//apple_ref/doc/uid/TP40010672-CH2-SW14
> > > > >>>
> > > > >>> Supporting both should be a goal.
> > > > >>>
> > > > >>> Cheers,
> > > > >>> Mike
> > > > >>>
> > > > >>>
> > > > >>>
> > > > >>> On 2013-12-10 14:39, Ian Clelland wrote:
> > > > >>>
> > > > >>>  If you could do that before, it was probably a bug :) (You
> > shouldn't
> > > > be
> > > > >>>> able to use getParent on the filesystem root, and the native
> code
> > > was
> > > > >>>> supposed to be checking for the correct path prefix before
> > allowing
> > > > >>>> access
> > > > >>>> to any files on the device)
> > > > >>>>
> > > > >>>> At the moment, it's not possible to do that -- the filesystems
> > > should
> > > > be
> > > > >>>> properly isolated, and only allow access to correctly formed
> urls,
> > > > like
> > > > >>>> filesystem://localhost/persistent/some-file-here.txt.
> > > > >>>>
> > > > >>>> What we *can* do easily, though, is allow a new URL scheme for
> > > library
> > > > >>>> assets; something like filesystem://localhost/
> > > > >>>> library/some-file-here.txt,
> > > > >>>> and we can have a filesystem handler for those URLs. That'll
> work
> > if
> > > > >>>> your
> > > > >>>> use case is just "I need access to files in /Library", rather
> than
> > > "I
> > > > >>>> need
> > > > >>>> to get to them via string manipulation".
> > > > >>>>
> > > > >>>> I've also had some discussions about making /Library the default
> > > place
> > > > >>>> for
> > > > >>>> the persistent filesystem, and leaving /Documents either just
> for
> > > > legacy
> > > > >>>> apps, or making *that* the target of the special URL scheme.
> > That's
> > > a
> > > > >>>> proposal for a different day, though. There are some pretty big
> > > > >>>> backwards-compatibility issues there.
> > > > >>>>
> > > > >>>>
> > > > >>>>
> > > > >>>> On Tue, Dec 10, 2013 at 11:54 AM, Michael Gauthier <
> > > > >>>> mike@silverorange.com
> > > > >>>>
> > > > >>>>> wrote:
> > > > >>>>>
> > > > >>>>
> > > > >>>>   Ian,
> > > > >>>>
> > > > >>>>>
> > > > >>>>> With the new URLs will it be possible to write to both
> /Documents
> > > and
> > > > >>>>> /Library for iOS apps? In the old version, the filesystem root
> > > > resolved
> > > > >>>>> as
> > > > >>>>> /Documents but it was possible to get to /Library by navigating
> > the
> > > > the
> > > > >>>>> parent dir.
> > > > >>>>>
> > > > >>>>> Cheers,
> > > > >>>>> Mike
> > > > >>>>>
> > > > >>>>>
> > > > >>>>>
> > > > >>>>> On 2013-11-15 15:19, Ian Clelland wrote:
> > > > >>>>>
> > > > >>>>>   On Fri, Nov 15, 2013 at 1:40 PM, purplecabbage <
> > > > >>>>> purplecabbage@gmail.com
> > > > >>>>>
> > > > >>>>>>
> > > > >>>>>>>  wrote:
> > > > >>>>>>
> > > > >>>>>>    Considering the magnitude of the changes I would have
> > expected
> > > > that
> > > > >>>>>> this
> > > > >>>>>>
> > > > >>>>>>  was just a new file plugin. The previous version was based
> on a
> > > > spec,
> > > > >>>>>>> and
> > > > >>>>>>> if we are deviating from it we should probably maintain both,
> > and
> > > > >>>>>>> possibly
> > > > >>>>>>> even make a recommendation to the w3c.
> > > > >>>>>>> I hope we at least do a major version update for this if APIs
> > > have
> > > > >>>>>>> changed.
> > > > >>>>>>>
> > > > >>>>>>>
> > > > >>>>>>>   Yes, definitely a major version bump, at least so that devs
> > > > aren't
> > > > >>>>>>>
> > > > >>>>>> caught
> > > > >>>>>> by the URL-format-change.
> > > > >>>>>>
> > > > >>>>>> There aren't any changes to external APIs; I've been very
> > careful
> > > to
> > > > >>>>>> maintain conformance with the existing tests, except where
> those
> > > > tests
> > > > >>>>>> have
> > > > >>>>>> been for undocumented implementation details. The only
> > app-visible
> > > > >>>>>> change
> > > > >>>>>> is that URLs returned by the .toURL() method will look like
> > > > >>>>>>
> > > > >>>>>> filesystem://localhost/persistent/my/file.jpg
> > > > >>>>>>
> > > > >>>>>> rather than
> > > > >>>>>>
> > > > >>>>>> file:///var/mobile/Applications/<app id>/Documents/my/file.jpg
> > > > >>>>>>
> > > > >>>>>> We are still following the spec -- in fact, this brings us
> > closer
> > > in
> > > > >>>>>> line
> > > > >>>>>> with the W3C File API spec, on these points:
> > > > >>>>>>
> > > > >>>>>> http://www.w3.org/TR/file-system-api/#widl-Entry-fullPath --
> > The
> > > > >>>>>> fullPath
> > > > >>>>>> of an entry should be the path from the root (of the HTML
> > > > filesystem,
> > > > >>>>>> not
> > > > >>>>>> the underlying device file system).
> > > > >>>>>>
> > > > >>>>>>
> > http://www.w3.org/TR/file-system-api/#widl-Entry-toURL-DOMString--
> > > > >>>>>> This
> > > > >>>>>> is
> > > > >>>>>> a note, rather than a requirement, but a filesystem url scheme
> > is
> > > > >>>>>> mentioned
> > > > >>>>>> there.
> > > > >>>>>>
> > > > >>>>>> We're considering extending the spec in one way, which should
> be
> > > > >>>>>> compatible
> > > > >>>>>> with the spirit of the spec, and backwards-compatible with the
> > > > >>>>>> previous
> > > > >>>>>> API. That is the ability to declare new filesystem types,
> beyond
> > > > >>>>>> "temporary" and "persistent", so that we can access things
> like
> > > > device
> > > > >>>>>> media and app bundle assets using the same APIs that we use
> for
> > > > >>>>>> accessing
> > > > >>>>>> user-defined files. If we're happy with the way that works,
> then
> > > we
> > > > >>>>>> should
> > > > >>>>>> definitely propose it for inclusion in the standard (not the
> > > > specific
> > > > >>>>>> filesystems, perhaps, but the mechanism for requesting and
> > > > interacting
> > > > >>>>>> with
> > > > >>>>>> them)
> > > > >>>>>>
> > > > >>>>>> Ian
> > > > >>>>>>
> > > > >>>>>>
> > > > >>>>>>
> > > > >>>>>>   Sent from my iPhone
> > > > >>>>>>
> > > > >>>>>>>
> > > > >>>>>>>    On Nov 15, 2013, at 8:36 AM, Ian Clelland <
> > > > iclelland@chromium.org
> > > > >>>>>>> >
> > > > >>>>>>>
> > > > >>>>>>>
> > > > >>>>>>>>   wrote:
> > > > >>>>>>>>
> > > > >>>>>>>
> > > > >>>>>>>
> > > > >>>>>>>  I've created CB-5403 as the über-ticket for this; various
> bits
> > > of
> > > > it
> > > > >>>>>>>> are
> > > > >>>>>>>>
> > > > >>>>>>>>   in
> > > > >>>>>>>>
> > > > >>>>>>>
> > > > >>>>>>>   subtasks.
> > > > >>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>> Once I have the tests and the JS updated, then platform
> owners
> > > can
> > > > >>>>>>>> take
> > > > >>>>>>>> a
> > > > >>>>>>>> look and see whether they have anything to do to support
> their
> > > own
> > > > >>>>>>>>
> > > > >>>>>>>>   schemes.
> > > > >>>>>>>>
> > > > >>>>>>>
> > > > >>>>>>>
> > > > >>>>>>>  As general guidelines, I would say:
> > > > >>>>>>>>
> > > > >>>>>>>> * If you can intercept filesystem://* URLs in native code,
> and
> > > > >>>>>>>> return
> > > > >>>>>>>> arbitrary responses, then do it! It will let your platform
> > > support
> > > > >>>>>>>> new
> > > > >>>>>>>> filesystems in the future as we come up with them. Add a
> > couple
> > > of
> > > > >>>>>>>>
> > > > >>>>>>>>   subtasks
> > > > >>>>>>>>
> > > > >>>>>>>
> > > > >>>>>>>   for CB-5403 for your platform and go.
> > > > >>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>> * If you can't do that, but you *can* provide access to
> things
> > > > like
> > > > >>>>>>>> media
> > > > >>>>>>>> through special urls, then try that! You should be able to
> > > create
> > > > a
> > > > >>>>>>>> FileSystem object that uses that URL as its root, and
> > everything
> > > > >>>>>>>> should
> > > > >>>>>>>> work. Add a subtask for that, and see what you can do.
> > > > >>>>>>>>
> > > > >>>>>>>> * If you can't do that either, and you just want to stick
> with
> > > the
> > > > >>>>>>>>
> > > > >>>>>>>>   file:///
> > > > >>>>>>>>
> > > > >>>>>>>
> > > > >>>>>>>   urls that we are currently using, then don't do anything;
> > > nothing
> > > > >>>>>>>
> > > > >>>>>>>> should
> > > > >>>>>>>> change for you.
> > > > >>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>> On Fri, Nov 15, 2013 at 10:36 AM, Ian Clelland <
> > > > >>>>>>>> iclelland@chromium.org
> > > > >>>>>>>> wrote:
> > > > >>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>>   On Fri, Nov 15, 2013 at 10:03 AM, Marcel Kinard <
> > > > >>>>>>>> cmarcelk@gmail.com
> > > > >>>>>>>>
> > > > >>>>>>>>>
> > > > >>>>>>>>>   wrote:
> > > > >>>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>>     Ian, will there be changes that app developers will need
> > to
> > > > do?
> > > > >>>>>>>>> If
> > > > >>>>>>>>> so,
> > > > >>>>>>>>>
> > > > >>>>>>>>>  those should be clearly documented in a migration guide.
> If
> > > not,
> > > > >>>>>>>>>> it
> > > > >>>>>>>>>>
> > > > >>>>>>>>>>   sounds
> > > > >>>>>>>>>>
> > > > >>>>>>>>>
> > > > >>>>>>>>>
> > > > >>>>>>>>    like the [improved] tests should pass on both the old and
> > new
> > > > >>>>>>>
> > > > >>>>>>>> versions,
> > > > >>>>>>>>
> > > > >>>>>>>>  which would be sweet.
> > > > >>>>>>>>>
> > > > >>>>>>>>>>
> > > > >>>>>>>>>>
> > > > >>>>>>>>>>  The filesystem URLs themselves will change as a result of
> > > this.
> > > > >>>>>>>>> The
> > > > >>>>>>>>>
> > > > >>>>>>>>>   tests
> > > > >>>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>   should pass on both old and new versions, but the tests
> mint
> > > > their
> > > > >>>>>>> own
> > > > >>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>>>   URLs
> > > > >>>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>   for testing against -- each version is internally
> consistent,
> > > but
> > > > >>>>>>> if
> > > > >>>>>>>
> > > > >>>>>>>> an
> > > > >>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>>>   app
> > > > >>>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>   is saving internal URLs and tries to dereference an old
> > > > (file:///)
> > > > >>>>>>> url
> > > > >>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>>>   with
> > > > >>>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>   the new plugin, it will probably not resolve.
> > > > >>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>>> Maybe there's a transition path here -- it might be
> possible
> > to
> > > > >>>>>>>>> allow
> > > > >>>>>>>>> window.resolveLocalFileSystemURL to access files with the
> old
> > > > URL,
> > > > >>>>>>>>> but
> > > > >>>>>>>>> never actually generate URLs.
> > > > >>>>>>>>>
> > > > >>>>>>>>> That would mean that
> > > > >>>>>>>>>
> > > > >>>>>>>>> (resolveLocalFileSystemURL(a)).toURL() !== a,
> > > > >>>>>>>>>
> > > > >>>>>>>>> but I don't think that we depend on that as an identity.
> > > > >>>>>>>>>
> > > > >>>>>>>>> Is there work which needs to be done on the other platforms
> > > (BB,
> > > > >>>>>>>>> WP8,
> > > > >>>>>>>>>
> > > > >>>>>>>>>   Win8, FFOS, etc) so that those platforms don't get left
> > > behind?
> > > > >>>>>>>>> If
> > > > >>>>>>>>>
> > > > >>>>>>>>>> so,
> > > > >>>>>>>>>> would it make sense to reach out to those platform owners
> > and
> > > at
> > > > >>>>>>>>>> least
> > > > >>>>>>>>>>
> > > > >>>>>>>>>>   get
> > > > >>>>>>>>>>
> > > > >>>>>>>>>
> > > > >>>>>>>>>
> > > > >>>>>>>>    Jira items created with some notes?
> > > > >>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>>>
> > > > >>>>>>>>>>  Yes. I'm going to create a ticket for this, and we can
> add
> > > > >>>>>>>>> platform-specific tasks to it.
> > > > >>>>>>>>>
> > > > >>>>>>>>> Other platforms shouldn't *have* to do anything, and some
> > > > platforms
> > > > >>>>>>>>> *can't* do anything, because they cannot access anything
> > other
> > > > than
> > > > >>>>>>>>> file:/// urls anyway.
> > > > >>>>>>>>>
> > > > >>>>>>>>> However, if the other platforms want to get in on the
> > goodness,
> > > > and
> > > > >>>>>>>>>
> > > > >>>>>>>>>   start
> > > > >>>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>   supporting their own URL schemes (I think the BB folks can
> > use
> > > > >>>>>>>
> > > > >>>>>>>> something
> > > > >>>>>>>>
> > > > >>>>>>>>  like local:// for their filesystem access), then that'll be
> > the
> > > > >>>>>>>>> place
> > > > >>>>>>>>> to
> > > > >>>>>>>>> keep everything organized.
> > > > >>>>>>>>>
> > > > >>>>>>>>> I'll post back once I have that issue set up.
> > > > >>>>>>>>>
> > > > >>>>>>>>> Ian
> > > > >>>>>>>>>
> > > > >>>>>>>>>
> > > > >>>>>>>>>    Sounds like you've built some very significant
> > improvements.
> > > > >>>>>>>>> Thanks!
> > > > >>>>>>>>>
> > > > >>>>>>>>>
> > > > >>>>>>>>>>    On Nov 15, 2013, at 9:29 AM, Ian Clelland <
> > > > >>>>>>>>>> iclelland@chromium.org
> > > > >>>>>>>>>>
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>
> > > > >>>>>>>>>>>   wrote:
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>
> > > > >>>>>>>>>>
> > > > >>>>>>>>>
> > > > >>>>>>>    After working with the File plugin for a week or so, I've
> > > gotten
> > > > >>>>>>>> it
> > > > >>>>>>>>
> > > > >>>>>>>>>
> > > > >>>>>>>>>>>   more or
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>
> > > > >>>>>>>>>>   less where I want it to be on iOS, and am working
> through
> > > > >>>>>>>>>> Android.
> > > > >>>>>>>>>>
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>>   Looking
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>
> > > > >>>>>>>>>>   at the end result, though, I expect that over 90% of the
> > > code
> > > > >>>>>>>>>> has
> > > > >>>>>>>>>>
> > > > >>>>>>>>>>> been
> > > > >>>>>>>>>>> touched in some way. (It's a huge diff.)
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>
> > > > >>>>>>>>>
> > > > >>>>>>>>>
> > > > >>>>>>>
> > > > >>>>>>
> > > > >>>>>
> > > > >>>>
> > > > >>>
> > > > >>
> > > > >
> > > >
> > >
> >
>

Re: Transitioning to a better File API implementation

[Tommy Williams <to...@devgeeks.org>]

+1
On 12/12/2013 6:26 am, "Ian Clelland" <ic...@chromium.org> wrote:

> Yeah, it would definitely require some kind of migration support. Not
> suggesting that this is something that we ever actually do, but we could
> give developers a bit of code that automatically looks in both places, and
> moves files to the new location on open. Or we do it under a flag that is
> off for existing apps (off-on-upgrade) and only enabled-by-default for new
> apps.
>
> I agree with you: this could cause worlds of pain, with developers, and
> worse with users. We shouldn't take any steps in that direction without a
> lot of very careful thought. For now, /Documents, sub-optimal though it may
> be, is where Cordova puts persistent files.
>
> Ian
>
> On Wed, Dec 11, 2013 at 2:18 PM, Tommy Williams <to...@devgeeks.org>
> wrote:
>
> > Changing where PERSISTENT is located sounds like a very very bad idea.
> >
> > I know that would cause me personally a lot of grief with existing user
> > data.
> > On 11/12/2013 8:58 am, "Ian Clelland" <ic...@chromium.org> wrote:
> >
> > > On Tue, Dec 10, 2013 at 4:37 PM, Michael Gauthier <
> mike@silverorange.com
> > > >wrote:
> > >
> > > > On 10/12/13 05:02 PM, Ian Clelland wrote:
> > > >
> > > >> Yep.
> > > >>
> > > >> I think that Library is the more natural place for the HTML
> persistent
> > > >> filesystem, since it's used by an app for whatever persistent
> storage
> > it
> > > >> needs, without exposing all of it's implementation details to the
> > user.
> > > >> (There's lots of room for debate on this, I'm sure)
> > > >>
> > > >> The trouble is that we've historically used /Documents for
> persistent
> > > >> storage, and changing that will break apps.
> > > >>
> > > >>  I'm fine with a BC break, but I don't have to maintain any legacy
> > > > applications. /Library does make more sense as the default for
> > > PERSISTENT.
> > > > The big problem with BC is for installed apps with existing data on
> the
> > > > filesystem, right?
> > >
> > >
> > > Exactly. I'd hate for developers to update their plugins, and push a
> new
> > > version of their app that seems fine -- everything works great when you
> > > start from scratch -- but users with existing data find themselves
> > > completely locked out of it.
> > >
> > >
> > > >
> > > >  One idea is to allow something like requestFilesystem(DOCUMENT), in
> > > >> addition to PERSISTENT and TEMPORARY. Another suggestion has been to
> > add
> > > >> extra arguments -- hints -- such as "{sync: true}", or maybe in this
> > > case
> > > >> "{purpose: documents}" to specify the attributes of the filesystem
> > that
> > > is
> > > >> returned.
> > > >>
> > > > >
> > > > {sync: true} is a bit tricky because /Library/Cache is not synced but
> > > > /Library/Application Data is synced. Having a DOCUMENT type would
> match
> > > > /tmp and /Library as the top-level dirs mapping to file-system
> > constants.
> > > >
> > > > All my feedback is only from the iOS perspective though. Not sure if
> > > these
> > > > ideas make other platforms more or less complicated.
> > > >
> > > > Cheers,
> > > > Mike
> > > >
> > > >
> > > >>
> > > >>
> > > >> On Tue, Dec 10, 2013 at 2:39 PM, Michael Gauthier <
> > > mike@silverorange.com
> > > >> >wrote:
> > > >>
> > > >>  Hmm.. The two directories have different defined roles on iOS and
> > both
> > > >>> are
> > > >>> normal targets for applications to read/write files. Library is for
> > > cache
> > > >>> data or app-specific resources. Documents is for saving/loading
> > actual
> > > >>> things created by users within apps.
> > > >>>
> > > >>> See https://developer.apple.com/library/ios/documentation/
> > > >>>
> > >
> FileManagement/Conceptual/FileSystemProgrammingGuide/FileSystemOverview/
> > > >>> FileSystemOverview.html#//apple_ref/doc/uid/TP40010672-CH2-SW14
> > > >>>
> > > >>> Supporting both should be a goal.
> > > >>>
> > > >>> Cheers,
> > > >>> Mike
> > > >>>
> > > >>>
> > > >>>
> > > >>> On 2013-12-10 14:39, Ian Clelland wrote:
> > > >>>
> > > >>>  If you could do that before, it was probably a bug :) (You
> shouldn't
> > > be
> > > >>>> able to use getParent on the filesystem root, and the native code
> > was
> > > >>>> supposed to be checking for the correct path prefix before
> allowing
> > > >>>> access
> > > >>>> to any files on the device)
> > > >>>>
> > > >>>> At the moment, it's not possible to do that -- the filesystems
> > should
> > > be
> > > >>>> properly isolated, and only allow access to correctly formed urls,
> > > like
> > > >>>> filesystem://localhost/persistent/some-file-here.txt.
> > > >>>>
> > > >>>> What we *can* do easily, though, is allow a new URL scheme for
> > library
> > > >>>> assets; something like filesystem://localhost/
> > > >>>> library/some-file-here.txt,
> > > >>>> and we can have a filesystem handler for those URLs. That'll work
> if
> > > >>>> your
> > > >>>> use case is just "I need access to files in /Library", rather than
> > "I
> > > >>>> need
> > > >>>> to get to them via string manipulation".
> > > >>>>
> > > >>>> I've also had some discussions about making /Library the default
> > place
> > > >>>> for
> > > >>>> the persistent filesystem, and leaving /Documents either just for
> > > legacy
> > > >>>> apps, or making *that* the target of the special URL scheme.
> That's
> > a
> > > >>>> proposal for a different day, though. There are some pretty big
> > > >>>> backwards-compatibility issues there.
> > > >>>>
> > > >>>>
> > > >>>>
> > > >>>> On Tue, Dec 10, 2013 at 11:54 AM, Michael Gauthier <
> > > >>>> mike@silverorange.com
> > > >>>>
> > > >>>>> wrote:
> > > >>>>>
> > > >>>>
> > > >>>>   Ian,
> > > >>>>
> > > >>>>>
> > > >>>>> With the new URLs will it be possible to write to both /Documents
> > and
> > > >>>>> /Library for iOS apps? In the old version, the filesystem root
> > > resolved
> > > >>>>> as
> > > >>>>> /Documents but it was possible to get to /Library by navigating
> the
> > > the
> > > >>>>> parent dir.
> > > >>>>>
> > > >>>>> Cheers,
> > > >>>>> Mike
> > > >>>>>
> > > >>>>>
> > > >>>>>
> > > >>>>> On 2013-11-15 15:19, Ian Clelland wrote:
> > > >>>>>
> > > >>>>>   On Fri, Nov 15, 2013 at 1:40 PM, purplecabbage <
> > > >>>>> purplecabbage@gmail.com
> > > >>>>>
> > > >>>>>>
> > > >>>>>>>  wrote:
> > > >>>>>>
> > > >>>>>>    Considering the magnitude of the changes I would have
> expected
> > > that
> > > >>>>>> this
> > > >>>>>>
> > > >>>>>>  was just a new file plugin. The previous version was based on a
> > > spec,
> > > >>>>>>> and
> > > >>>>>>> if we are deviating from it we should probably maintain both,
> and
> > > >>>>>>> possibly
> > > >>>>>>> even make a recommendation to the w3c.
> > > >>>>>>> I hope we at least do a major version update for this if APIs
> > have
> > > >>>>>>> changed.
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>>   Yes, definitely a major version bump, at least so that devs
> > > aren't
> > > >>>>>>>
> > > >>>>>> caught
> > > >>>>>> by the URL-format-change.
> > > >>>>>>
> > > >>>>>> There aren't any changes to external APIs; I've been very
> careful
> > to
> > > >>>>>> maintain conformance with the existing tests, except where those
> > > tests
> > > >>>>>> have
> > > >>>>>> been for undocumented implementation details. The only
> app-visible
> > > >>>>>> change
> > > >>>>>> is that URLs returned by the .toURL() method will look like
> > > >>>>>>
> > > >>>>>> filesystem://localhost/persistent/my/file.jpg
> > > >>>>>>
> > > >>>>>> rather than
> > > >>>>>>
> > > >>>>>> file:///var/mobile/Applications/<app id>/Documents/my/file.jpg
> > > >>>>>>
> > > >>>>>> We are still following the spec -- in fact, this brings us
> closer
> > in
> > > >>>>>> line
> > > >>>>>> with the W3C File API spec, on these points:
> > > >>>>>>
> > > >>>>>> http://www.w3.org/TR/file-system-api/#widl-Entry-fullPath --
> The
> > > >>>>>> fullPath
> > > >>>>>> of an entry should be the path from the root (of the HTML
> > > filesystem,
> > > >>>>>> not
> > > >>>>>> the underlying device file system).
> > > >>>>>>
> > > >>>>>>
> http://www.w3.org/TR/file-system-api/#widl-Entry-toURL-DOMString--
> > > >>>>>> This
> > > >>>>>> is
> > > >>>>>> a note, rather than a requirement, but a filesystem url scheme
> is
> > > >>>>>> mentioned
> > > >>>>>> there.
> > > >>>>>>
> > > >>>>>> We're considering extending the spec in one way, which should be
> > > >>>>>> compatible
> > > >>>>>> with the spirit of the spec, and backwards-compatible with the
> > > >>>>>> previous
> > > >>>>>> API. That is the ability to declare new filesystem types, beyond
> > > >>>>>> "temporary" and "persistent", so that we can access things like
> > > device
> > > >>>>>> media and app bundle assets using the same APIs that we use for
> > > >>>>>> accessing
> > > >>>>>> user-defined files. If we're happy with the way that works, then
> > we
> > > >>>>>> should
> > > >>>>>> definitely propose it for inclusion in the standard (not the
> > > specific
> > > >>>>>> filesystems, perhaps, but the mechanism for requesting and
> > > interacting
> > > >>>>>> with
> > > >>>>>> them)
> > > >>>>>>
> > > >>>>>> Ian
> > > >>>>>>
> > > >>>>>>
> > > >>>>>>
> > > >>>>>>   Sent from my iPhone
> > > >>>>>>
> > > >>>>>>>
> > > >>>>>>>    On Nov 15, 2013, at 8:36 AM, Ian Clelland <
> > > iclelland@chromium.org
> > > >>>>>>> >
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>>>   wrote:
> > > >>>>>>>>
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>>  I've created CB-5403 as the über-ticket for this; various bits
> > of
> > > it
> > > >>>>>>>> are
> > > >>>>>>>>
> > > >>>>>>>>   in
> > > >>>>>>>>
> > > >>>>>>>
> > > >>>>>>>   subtasks.
> > > >>>>>>>
> > > >>>>>>>>
> > > >>>>>>>> Once I have the tests and the JS updated, then platform owners
> > can
> > > >>>>>>>> take
> > > >>>>>>>> a
> > > >>>>>>>> look and see whether they have anything to do to support their
> > own
> > > >>>>>>>>
> > > >>>>>>>>   schemes.
> > > >>>>>>>>
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>>  As general guidelines, I would say:
> > > >>>>>>>>
> > > >>>>>>>> * If you can intercept filesystem://* URLs in native code, and
> > > >>>>>>>> return
> > > >>>>>>>> arbitrary responses, then do it! It will let your platform
> > support
> > > >>>>>>>> new
> > > >>>>>>>> filesystems in the future as we come up with them. Add a
> couple
> > of
> > > >>>>>>>>
> > > >>>>>>>>   subtasks
> > > >>>>>>>>
> > > >>>>>>>
> > > >>>>>>>   for CB-5403 for your platform and go.
> > > >>>>>>>
> > > >>>>>>>>
> > > >>>>>>>> * If you can't do that, but you *can* provide access to things
> > > like
> > > >>>>>>>> media
> > > >>>>>>>> through special urls, then try that! You should be able to
> > create
> > > a
> > > >>>>>>>> FileSystem object that uses that URL as its root, and
> everything
> > > >>>>>>>> should
> > > >>>>>>>> work. Add a subtask for that, and see what you can do.
> > > >>>>>>>>
> > > >>>>>>>> * If you can't do that either, and you just want to stick with
> > the
> > > >>>>>>>>
> > > >>>>>>>>   file:///
> > > >>>>>>>>
> > > >>>>>>>
> > > >>>>>>>   urls that we are currently using, then don't do anything;
> > nothing
> > > >>>>>>>
> > > >>>>>>>> should
> > > >>>>>>>> change for you.
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>> On Fri, Nov 15, 2013 at 10:36 AM, Ian Clelland <
> > > >>>>>>>> iclelland@chromium.org
> > > >>>>>>>> wrote:
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>>   On Fri, Nov 15, 2013 at 10:03 AM, Marcel Kinard <
> > > >>>>>>>> cmarcelk@gmail.com
> > > >>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>>   wrote:
> > > >>>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>>     Ian, will there be changes that app developers will need
> to
> > > do?
> > > >>>>>>>>> If
> > > >>>>>>>>> so,
> > > >>>>>>>>>
> > > >>>>>>>>>  those should be clearly documented in a migration guide. If
> > not,
> > > >>>>>>>>>> it
> > > >>>>>>>>>>
> > > >>>>>>>>>>   sounds
> > > >>>>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>    like the [improved] tests should pass on both the old and
> new
> > > >>>>>>>
> > > >>>>>>>> versions,
> > > >>>>>>>>
> > > >>>>>>>>  which would be sweet.
> > > >>>>>>>>>
> > > >>>>>>>>>>
> > > >>>>>>>>>>
> > > >>>>>>>>>>  The filesystem URLs themselves will change as a result of
> > this.
> > > >>>>>>>>> The
> > > >>>>>>>>>
> > > >>>>>>>>>   tests
> > > >>>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>   should pass on both old and new versions, but the tests mint
> > > their
> > > >>>>>>> own
> > > >>>>>>>
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>>>   URLs
> > > >>>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>   for testing against -- each version is internally consistent,
> > but
> > > >>>>>>> if
> > > >>>>>>>
> > > >>>>>>>> an
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>>>   app
> > > >>>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>   is saving internal URLs and tries to dereference an old
> > > (file:///)
> > > >>>>>>> url
> > > >>>>>>>
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>>>   with
> > > >>>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>   the new plugin, it will probably not resolve.
> > > >>>>>>>
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>>> Maybe there's a transition path here -- it might be possible
> to
> > > >>>>>>>>> allow
> > > >>>>>>>>> window.resolveLocalFileSystemURL to access files with the old
> > > URL,
> > > >>>>>>>>> but
> > > >>>>>>>>> never actually generate URLs.
> > > >>>>>>>>>
> > > >>>>>>>>> That would mean that
> > > >>>>>>>>>
> > > >>>>>>>>> (resolveLocalFileSystemURL(a)).toURL() !== a,
> > > >>>>>>>>>
> > > >>>>>>>>> but I don't think that we depend on that as an identity.
> > > >>>>>>>>>
> > > >>>>>>>>> Is there work which needs to be done on the other platforms
> > (BB,
> > > >>>>>>>>> WP8,
> > > >>>>>>>>>
> > > >>>>>>>>>   Win8, FFOS, etc) so that those platforms don't get left
> > behind?
> > > >>>>>>>>> If
> > > >>>>>>>>>
> > > >>>>>>>>>> so,
> > > >>>>>>>>>> would it make sense to reach out to those platform owners
> and
> > at
> > > >>>>>>>>>> least
> > > >>>>>>>>>>
> > > >>>>>>>>>>   get
> > > >>>>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>    Jira items created with some notes?
> > > >>>>>>>
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>>>  Yes. I'm going to create a ticket for this, and we can add
> > > >>>>>>>>> platform-specific tasks to it.
> > > >>>>>>>>>
> > > >>>>>>>>> Other platforms shouldn't *have* to do anything, and some
> > > platforms
> > > >>>>>>>>> *can't* do anything, because they cannot access anything
> other
> > > than
> > > >>>>>>>>> file:/// urls anyway.
> > > >>>>>>>>>
> > > >>>>>>>>> However, if the other platforms want to get in on the
> goodness,
> > > and
> > > >>>>>>>>>
> > > >>>>>>>>>   start
> > > >>>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>   supporting their own URL schemes (I think the BB folks can
> use
> > > >>>>>>>
> > > >>>>>>>> something
> > > >>>>>>>>
> > > >>>>>>>>  like local:// for their filesystem access), then that'll be
> the
> > > >>>>>>>>> place
> > > >>>>>>>>> to
> > > >>>>>>>>> keep everything organized.
> > > >>>>>>>>>
> > > >>>>>>>>> I'll post back once I have that issue set up.
> > > >>>>>>>>>
> > > >>>>>>>>> Ian
> > > >>>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>>    Sounds like you've built some very significant
> improvements.
> > > >>>>>>>>> Thanks!
> > > >>>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>>>    On Nov 15, 2013, at 9:29 AM, Ian Clelland <
> > > >>>>>>>>>> iclelland@chromium.org
> > > >>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>>>>>>
> > > >>>>>>>>>>>   wrote:
> > > >>>>>>>>>>>
> > > >>>>>>>>>>
> > > >>>>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>    After working with the File plugin for a week or so, I've
> > gotten
> > > >>>>>>>> it
> > > >>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>>>>   more or
> > > >>>>>>>>>>>
> > > >>>>>>>>>>
> > > >>>>>>>>>>   less where I want it to be on iOS, and am working through
> > > >>>>>>>>>> Android.
> > > >>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>>>>>>>   Looking
> > > >>>>>>>>>>>
> > > >>>>>>>>>>
> > > >>>>>>>>>>   at the end result, though, I expect that over 90% of the
> > code
> > > >>>>>>>>>> has
> > > >>>>>>>>>>
> > > >>>>>>>>>>> been
> > > >>>>>>>>>>> touched in some way. (It's a huge diff.)
> > > >>>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>
> > > >>>>>>
> > > >>>>>
> > > >>>>
> > > >>>
> > > >>
> > > >
> > >
> >
>

Re: Transitioning to a better File API implementation

[Ian Clelland <ic...@chromium.org>]

Yeah, it would definitely require some kind of migration support. Not
suggesting that this is something that we ever actually do, but we could
give developers a bit of code that automatically looks in both places, and
moves files to the new location on open. Or we do it under a flag that is
off for existing apps (off-on-upgrade) and only enabled-by-default for new
apps.

I agree with you: this could cause worlds of pain, with developers, and
worse with users. We shouldn't take any steps in that direction without a
lot of very careful thought. For now, /Documents, sub-optimal though it may
be, is where Cordova puts persistent files.

Ian

On Wed, Dec 11, 2013 at 2:18 PM, Tommy Williams <to...@devgeeks.org> wrote:

> Changing where PERSISTENT is located sounds like a very very bad idea.
>
> I know that would cause me personally a lot of grief with existing user
> data.
> On 11/12/2013 8:58 am, "Ian Clelland" <ic...@chromium.org> wrote:
>
> > On Tue, Dec 10, 2013 at 4:37 PM, Michael Gauthier <mike@silverorange.com
> > >wrote:
> >
> > > On 10/12/13 05:02 PM, Ian Clelland wrote:
> > >
> > >> Yep.
> > >>
> > >> I think that Library is the more natural place for the HTML persistent
> > >> filesystem, since it's used by an app for whatever persistent storage
> it
> > >> needs, without exposing all of it's implementation details to the
> user.
> > >> (There's lots of room for debate on this, I'm sure)
> > >>
> > >> The trouble is that we've historically used /Documents for persistent
> > >> storage, and changing that will break apps.
> > >>
> > >>  I'm fine with a BC break, but I don't have to maintain any legacy
> > > applications. /Library does make more sense as the default for
> > PERSISTENT.
> > > The big problem with BC is for installed apps with existing data on the
> > > filesystem, right?
> >
> >
> > Exactly. I'd hate for developers to update their plugins, and push a new
> > version of their app that seems fine -- everything works great when you
> > start from scratch -- but users with existing data find themselves
> > completely locked out of it.
> >
> >
> > >
> > >  One idea is to allow something like requestFilesystem(DOCUMENT), in
> > >> addition to PERSISTENT and TEMPORARY. Another suggestion has been to
> add
> > >> extra arguments -- hints -- such as "{sync: true}", or maybe in this
> > case
> > >> "{purpose: documents}" to specify the attributes of the filesystem
> that
> > is
> > >> returned.
> > >>
> > > >
> > > {sync: true} is a bit tricky because /Library/Cache is not synced but
> > > /Library/Application Data is synced. Having a DOCUMENT type would match
> > > /tmp and /Library as the top-level dirs mapping to file-system
> constants.
> > >
> > > All my feedback is only from the iOS perspective though. Not sure if
> > these
> > > ideas make other platforms more or less complicated.
> > >
> > > Cheers,
> > > Mike
> > >
> > >
> > >>
> > >>
> > >> On Tue, Dec 10, 2013 at 2:39 PM, Michael Gauthier <
> > mike@silverorange.com
> > >> >wrote:
> > >>
> > >>  Hmm.. The two directories have different defined roles on iOS and
> both
> > >>> are
> > >>> normal targets for applications to read/write files. Library is for
> > cache
> > >>> data or app-specific resources. Documents is for saving/loading
> actual
> > >>> things created by users within apps.
> > >>>
> > >>> See https://developer.apple.com/library/ios/documentation/
> > >>>
> > FileManagement/Conceptual/FileSystemProgrammingGuide/FileSystemOverview/
> > >>> FileSystemOverview.html#//apple_ref/doc/uid/TP40010672-CH2-SW14
> > >>>
> > >>> Supporting both should be a goal.
> > >>>
> > >>> Cheers,
> > >>> Mike
> > >>>
> > >>>
> > >>>
> > >>> On 2013-12-10 14:39, Ian Clelland wrote:
> > >>>
> > >>>  If you could do that before, it was probably a bug :) (You shouldn't
> > be
> > >>>> able to use getParent on the filesystem root, and the native code
> was
> > >>>> supposed to be checking for the correct path prefix before allowing
> > >>>> access
> > >>>> to any files on the device)
> > >>>>
> > >>>> At the moment, it's not possible to do that -- the filesystems
> should
> > be
> > >>>> properly isolated, and only allow access to correctly formed urls,
> > like
> > >>>> filesystem://localhost/persistent/some-file-here.txt.
> > >>>>
> > >>>> What we *can* do easily, though, is allow a new URL scheme for
> library
> > >>>> assets; something like filesystem://localhost/
> > >>>> library/some-file-here.txt,
> > >>>> and we can have a filesystem handler for those URLs. That'll work if
> > >>>> your
> > >>>> use case is just "I need access to files in /Library", rather than
> "I
> > >>>> need
> > >>>> to get to them via string manipulation".
> > >>>>
> > >>>> I've also had some discussions about making /Library the default
> place
> > >>>> for
> > >>>> the persistent filesystem, and leaving /Documents either just for
> > legacy
> > >>>> apps, or making *that* the target of the special URL scheme. That's
> a
> > >>>> proposal for a different day, though. There are some pretty big
> > >>>> backwards-compatibility issues there.
> > >>>>
> > >>>>
> > >>>>
> > >>>> On Tue, Dec 10, 2013 at 11:54 AM, Michael Gauthier <
> > >>>> mike@silverorange.com
> > >>>>
> > >>>>> wrote:
> > >>>>>
> > >>>>
> > >>>>   Ian,
> > >>>>
> > >>>>>
> > >>>>> With the new URLs will it be possible to write to both /Documents
> and
> > >>>>> /Library for iOS apps? In the old version, the filesystem root
> > resolved
> > >>>>> as
> > >>>>> /Documents but it was possible to get to /Library by navigating the
> > the
> > >>>>> parent dir.
> > >>>>>
> > >>>>> Cheers,
> > >>>>> Mike
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>> On 2013-11-15 15:19, Ian Clelland wrote:
> > >>>>>
> > >>>>>   On Fri, Nov 15, 2013 at 1:40 PM, purplecabbage <
> > >>>>> purplecabbage@gmail.com
> > >>>>>
> > >>>>>>
> > >>>>>>>  wrote:
> > >>>>>>
> > >>>>>>    Considering the magnitude of the changes I would have expected
> > that
> > >>>>>> this
> > >>>>>>
> > >>>>>>  was just a new file plugin. The previous version was based on a
> > spec,
> > >>>>>>> and
> > >>>>>>> if we are deviating from it we should probably maintain both, and
> > >>>>>>> possibly
> > >>>>>>> even make a recommendation to the w3c.
> > >>>>>>> I hope we at least do a major version update for this if APIs
> have
> > >>>>>>> changed.
> > >>>>>>>
> > >>>>>>>
> > >>>>>>>   Yes, definitely a major version bump, at least so that devs
> > aren't
> > >>>>>>>
> > >>>>>> caught
> > >>>>>> by the URL-format-change.
> > >>>>>>
> > >>>>>> There aren't any changes to external APIs; I've been very careful
> to
> > >>>>>> maintain conformance with the existing tests, except where those
> > tests
> > >>>>>> have
> > >>>>>> been for undocumented implementation details. The only app-visible
> > >>>>>> change
> > >>>>>> is that URLs returned by the .toURL() method will look like
> > >>>>>>
> > >>>>>> filesystem://localhost/persistent/my/file.jpg
> > >>>>>>
> > >>>>>> rather than
> > >>>>>>
> > >>>>>> file:///var/mobile/Applications/<app id>/Documents/my/file.jpg
> > >>>>>>
> > >>>>>> We are still following the spec -- in fact, this brings us closer
> in
> > >>>>>> line
> > >>>>>> with the W3C File API spec, on these points:
> > >>>>>>
> > >>>>>> http://www.w3.org/TR/file-system-api/#widl-Entry-fullPath -- The
> > >>>>>> fullPath
> > >>>>>> of an entry should be the path from the root (of the HTML
> > filesystem,
> > >>>>>> not
> > >>>>>> the underlying device file system).
> > >>>>>>
> > >>>>>> http://www.w3.org/TR/file-system-api/#widl-Entry-toURL-DOMString--
> > >>>>>> This
> > >>>>>> is
> > >>>>>> a note, rather than a requirement, but a filesystem url scheme is
> > >>>>>> mentioned
> > >>>>>> there.
> > >>>>>>
> > >>>>>> We're considering extending the spec in one way, which should be
> > >>>>>> compatible
> > >>>>>> with the spirit of the spec, and backwards-compatible with the
> > >>>>>> previous
> > >>>>>> API. That is the ability to declare new filesystem types, beyond
> > >>>>>> "temporary" and "persistent", so that we can access things like
> > device
> > >>>>>> media and app bundle assets using the same APIs that we use for
> > >>>>>> accessing
> > >>>>>> user-defined files. If we're happy with the way that works, then
> we
> > >>>>>> should
> > >>>>>> definitely propose it for inclusion in the standard (not the
> > specific
> > >>>>>> filesystems, perhaps, but the mechanism for requesting and
> > interacting
> > >>>>>> with
> > >>>>>> them)
> > >>>>>>
> > >>>>>> Ian
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>>   Sent from my iPhone
> > >>>>>>
> > >>>>>>>
> > >>>>>>>    On Nov 15, 2013, at 8:36 AM, Ian Clelland <
> > iclelland@chromium.org
> > >>>>>>> >
> > >>>>>>>
> > >>>>>>>
> > >>>>>>>>   wrote:
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>>  I've created CB-5403 as the über-ticket for this; various bits
> of
> > it
> > >>>>>>>> are
> > >>>>>>>>
> > >>>>>>>>   in
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>>   subtasks.
> > >>>>>>>
> > >>>>>>>>
> > >>>>>>>> Once I have the tests and the JS updated, then platform owners
> can
> > >>>>>>>> take
> > >>>>>>>> a
> > >>>>>>>> look and see whether they have anything to do to support their
> own
> > >>>>>>>>
> > >>>>>>>>   schemes.
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>>  As general guidelines, I would say:
> > >>>>>>>>
> > >>>>>>>> * If you can intercept filesystem://* URLs in native code, and
> > >>>>>>>> return
> > >>>>>>>> arbitrary responses, then do it! It will let your platform
> support
> > >>>>>>>> new
> > >>>>>>>> filesystems in the future as we come up with them. Add a couple
> of
> > >>>>>>>>
> > >>>>>>>>   subtasks
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>>   for CB-5403 for your platform and go.
> > >>>>>>>
> > >>>>>>>>
> > >>>>>>>> * If you can't do that, but you *can* provide access to things
> > like
> > >>>>>>>> media
> > >>>>>>>> through special urls, then try that! You should be able to
> create
> > a
> > >>>>>>>> FileSystem object that uses that URL as its root, and everything
> > >>>>>>>> should
> > >>>>>>>> work. Add a subtask for that, and see what you can do.
> > >>>>>>>>
> > >>>>>>>> * If you can't do that either, and you just want to stick with
> the
> > >>>>>>>>
> > >>>>>>>>   file:///
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>>   urls that we are currently using, then don't do anything;
> nothing
> > >>>>>>>
> > >>>>>>>> should
> > >>>>>>>> change for you.
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> On Fri, Nov 15, 2013 at 10:36 AM, Ian Clelland <
> > >>>>>>>> iclelland@chromium.org
> > >>>>>>>> wrote:
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>   On Fri, Nov 15, 2013 at 10:03 AM, Marcel Kinard <
> > >>>>>>>> cmarcelk@gmail.com
> > >>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>   wrote:
> > >>>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>     Ian, will there be changes that app developers will need to
> > do?
> > >>>>>>>>> If
> > >>>>>>>>> so,
> > >>>>>>>>>
> > >>>>>>>>>  those should be clearly documented in a migration guide. If
> not,
> > >>>>>>>>>> it
> > >>>>>>>>>>
> > >>>>>>>>>>   sounds
> > >>>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>    like the [improved] tests should pass on both the old and new
> > >>>>>>>
> > >>>>>>>> versions,
> > >>>>>>>>
> > >>>>>>>>  which would be sweet.
> > >>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>  The filesystem URLs themselves will change as a result of
> this.
> > >>>>>>>>> The
> > >>>>>>>>>
> > >>>>>>>>>   tests
> > >>>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>   should pass on both old and new versions, but the tests mint
> > their
> > >>>>>>> own
> > >>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>>   URLs
> > >>>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>   for testing against -- each version is internally consistent,
> but
> > >>>>>>> if
> > >>>>>>>
> > >>>>>>>> an
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>>   app
> > >>>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>   is saving internal URLs and tries to dereference an old
> > (file:///)
> > >>>>>>> url
> > >>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>>   with
> > >>>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>   the new plugin, it will probably not resolve.
> > >>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>> Maybe there's a transition path here -- it might be possible to
> > >>>>>>>>> allow
> > >>>>>>>>> window.resolveLocalFileSystemURL to access files with the old
> > URL,
> > >>>>>>>>> but
> > >>>>>>>>> never actually generate URLs.
> > >>>>>>>>>
> > >>>>>>>>> That would mean that
> > >>>>>>>>>
> > >>>>>>>>> (resolveLocalFileSystemURL(a)).toURL() !== a,
> > >>>>>>>>>
> > >>>>>>>>> but I don't think that we depend on that as an identity.
> > >>>>>>>>>
> > >>>>>>>>> Is there work which needs to be done on the other platforms
> (BB,
> > >>>>>>>>> WP8,
> > >>>>>>>>>
> > >>>>>>>>>   Win8, FFOS, etc) so that those platforms don't get left
> behind?
> > >>>>>>>>> If
> > >>>>>>>>>
> > >>>>>>>>>> so,
> > >>>>>>>>>> would it make sense to reach out to those platform owners and
> at
> > >>>>>>>>>> least
> > >>>>>>>>>>
> > >>>>>>>>>>   get
> > >>>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>    Jira items created with some notes?
> > >>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>>  Yes. I'm going to create a ticket for this, and we can add
> > >>>>>>>>> platform-specific tasks to it.
> > >>>>>>>>>
> > >>>>>>>>> Other platforms shouldn't *have* to do anything, and some
> > platforms
> > >>>>>>>>> *can't* do anything, because they cannot access anything other
> > than
> > >>>>>>>>> file:/// urls anyway.
> > >>>>>>>>>
> > >>>>>>>>> However, if the other platforms want to get in on the goodness,
> > and
> > >>>>>>>>>
> > >>>>>>>>>   start
> > >>>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>   supporting their own URL schemes (I think the BB folks can use
> > >>>>>>>
> > >>>>>>>> something
> > >>>>>>>>
> > >>>>>>>>  like local:// for their filesystem access), then that'll be the
> > >>>>>>>>> place
> > >>>>>>>>> to
> > >>>>>>>>> keep everything organized.
> > >>>>>>>>>
> > >>>>>>>>> I'll post back once I have that issue set up.
> > >>>>>>>>>
> > >>>>>>>>> Ian
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>    Sounds like you've built some very significant improvements.
> > >>>>>>>>> Thanks!
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>>    On Nov 15, 2013, at 9:29 AM, Ian Clelland <
> > >>>>>>>>>> iclelland@chromium.org
> > >>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>>   wrote:
> > >>>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>    After working with the File plugin for a week or so, I've
> gotten
> > >>>>>>>> it
> > >>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>>>   more or
> > >>>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>   less where I want it to be on iOS, and am working through
> > >>>>>>>>>> Android.
> > >>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>>   Looking
> > >>>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>   at the end result, though, I expect that over 90% of the
> code
> > >>>>>>>>>> has
> > >>>>>>>>>>
> > >>>>>>>>>>> been
> > >>>>>>>>>>> touched in some way. (It's a huge diff.)
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>
> > >>>>>>
> > >>>>>
> > >>>>
> > >>>
> > >>
> > >
> >
>

Re: Transitioning to a better File API implementation

[Tommy Williams <to...@devgeeks.org>]

Changing where PERSISTENT is located sounds like a very very bad idea.

I know that would cause me personally a lot of grief with existing user
data.
On 11/12/2013 8:58 am, "Ian Clelland" <ic...@chromium.org> wrote:

> On Tue, Dec 10, 2013 at 4:37 PM, Michael Gauthier <mike@silverorange.com
> >wrote:
>
> > On 10/12/13 05:02 PM, Ian Clelland wrote:
> >
> >> Yep.
> >>
> >> I think that Library is the more natural place for the HTML persistent
> >> filesystem, since it's used by an app for whatever persistent storage it
> >> needs, without exposing all of it's implementation details to the user.
> >> (There's lots of room for debate on this, I'm sure)
> >>
> >> The trouble is that we've historically used /Documents for persistent
> >> storage, and changing that will break apps.
> >>
> >>  I'm fine with a BC break, but I don't have to maintain any legacy
> > applications. /Library does make more sense as the default for
> PERSISTENT.
> > The big problem with BC is for installed apps with existing data on the
> > filesystem, right?
>
>
> Exactly. I'd hate for developers to update their plugins, and push a new
> version of their app that seems fine -- everything works great when you
> start from scratch -- but users with existing data find themselves
> completely locked out of it.
>
>
> >
> >  One idea is to allow something like requestFilesystem(DOCUMENT), in
> >> addition to PERSISTENT and TEMPORARY. Another suggestion has been to add
> >> extra arguments -- hints -- such as "{sync: true}", or maybe in this
> case
> >> "{purpose: documents}" to specify the attributes of the filesystem that
> is
> >> returned.
> >>
> > >
> > {sync: true} is a bit tricky because /Library/Cache is not synced but
> > /Library/Application Data is synced. Having a DOCUMENT type would match
> > /tmp and /Library as the top-level dirs mapping to file-system constants.
> >
> > All my feedback is only from the iOS perspective though. Not sure if
> these
> > ideas make other platforms more or less complicated.
> >
> > Cheers,
> > Mike
> >
> >
> >>
> >>
> >> On Tue, Dec 10, 2013 at 2:39 PM, Michael Gauthier <
> mike@silverorange.com
> >> >wrote:
> >>
> >>  Hmm.. The two directories have different defined roles on iOS and both
> >>> are
> >>> normal targets for applications to read/write files. Library is for
> cache
> >>> data or app-specific resources. Documents is for saving/loading actual
> >>> things created by users within apps.
> >>>
> >>> See https://developer.apple.com/library/ios/documentation/
> >>>
> FileManagement/Conceptual/FileSystemProgrammingGuide/FileSystemOverview/
> >>> FileSystemOverview.html#//apple_ref/doc/uid/TP40010672-CH2-SW14
> >>>
> >>> Supporting both should be a goal.
> >>>
> >>> Cheers,
> >>> Mike
> >>>
> >>>
> >>>
> >>> On 2013-12-10 14:39, Ian Clelland wrote:
> >>>
> >>>  If you could do that before, it was probably a bug :) (You shouldn't
> be
> >>>> able to use getParent on the filesystem root, and the native code was
> >>>> supposed to be checking for the correct path prefix before allowing
> >>>> access
> >>>> to any files on the device)
> >>>>
> >>>> At the moment, it's not possible to do that -- the filesystems should
> be
> >>>> properly isolated, and only allow access to correctly formed urls,
> like
> >>>> filesystem://localhost/persistent/some-file-here.txt.
> >>>>
> >>>> What we *can* do easily, though, is allow a new URL scheme for library
> >>>> assets; something like filesystem://localhost/
> >>>> library/some-file-here.txt,
> >>>> and we can have a filesystem handler for those URLs. That'll work if
> >>>> your
> >>>> use case is just "I need access to files in /Library", rather than "I
> >>>> need
> >>>> to get to them via string manipulation".
> >>>>
> >>>> I've also had some discussions about making /Library the default place
> >>>> for
> >>>> the persistent filesystem, and leaving /Documents either just for
> legacy
> >>>> apps, or making *that* the target of the special URL scheme. That's a
> >>>> proposal for a different day, though. There are some pretty big
> >>>> backwards-compatibility issues there.
> >>>>
> >>>>
> >>>>
> >>>> On Tue, Dec 10, 2013 at 11:54 AM, Michael Gauthier <
> >>>> mike@silverorange.com
> >>>>
> >>>>> wrote:
> >>>>>
> >>>>
> >>>>   Ian,
> >>>>
> >>>>>
> >>>>> With the new URLs will it be possible to write to both /Documents and
> >>>>> /Library for iOS apps? In the old version, the filesystem root
> resolved
> >>>>> as
> >>>>> /Documents but it was possible to get to /Library by navigating the
> the
> >>>>> parent dir.
> >>>>>
> >>>>> Cheers,
> >>>>> Mike
> >>>>>
> >>>>>
> >>>>>
> >>>>> On 2013-11-15 15:19, Ian Clelland wrote:
> >>>>>
> >>>>>   On Fri, Nov 15, 2013 at 1:40 PM, purplecabbage <
> >>>>> purplecabbage@gmail.com
> >>>>>
> >>>>>>
> >>>>>>>  wrote:
> >>>>>>
> >>>>>>    Considering the magnitude of the changes I would have expected
> that
> >>>>>> this
> >>>>>>
> >>>>>>  was just a new file plugin. The previous version was based on a
> spec,
> >>>>>>> and
> >>>>>>> if we are deviating from it we should probably maintain both, and
> >>>>>>> possibly
> >>>>>>> even make a recommendation to the w3c.
> >>>>>>> I hope we at least do a major version update for this if APIs have
> >>>>>>> changed.
> >>>>>>>
> >>>>>>>
> >>>>>>>   Yes, definitely a major version bump, at least so that devs
> aren't
> >>>>>>>
> >>>>>> caught
> >>>>>> by the URL-format-change.
> >>>>>>
> >>>>>> There aren't any changes to external APIs; I've been very careful to
> >>>>>> maintain conformance with the existing tests, except where those
> tests
> >>>>>> have
> >>>>>> been for undocumented implementation details. The only app-visible
> >>>>>> change
> >>>>>> is that URLs returned by the .toURL() method will look like
> >>>>>>
> >>>>>> filesystem://localhost/persistent/my/file.jpg
> >>>>>>
> >>>>>> rather than
> >>>>>>
> >>>>>> file:///var/mobile/Applications/<app id>/Documents/my/file.jpg
> >>>>>>
> >>>>>> We are still following the spec -- in fact, this brings us closer in
> >>>>>> line
> >>>>>> with the W3C File API spec, on these points:
> >>>>>>
> >>>>>> http://www.w3.org/TR/file-system-api/#widl-Entry-fullPath -- The
> >>>>>> fullPath
> >>>>>> of an entry should be the path from the root (of the HTML
> filesystem,
> >>>>>> not
> >>>>>> the underlying device file system).
> >>>>>>
> >>>>>> http://www.w3.org/TR/file-system-api/#widl-Entry-toURL-DOMString --
> >>>>>> This
> >>>>>> is
> >>>>>> a note, rather than a requirement, but a filesystem url scheme is
> >>>>>> mentioned
> >>>>>> there.
> >>>>>>
> >>>>>> We're considering extending the spec in one way, which should be
> >>>>>> compatible
> >>>>>> with the spirit of the spec, and backwards-compatible with the
> >>>>>> previous
> >>>>>> API. That is the ability to declare new filesystem types, beyond
> >>>>>> "temporary" and "persistent", so that we can access things like
> device
> >>>>>> media and app bundle assets using the same APIs that we use for
> >>>>>> accessing
> >>>>>> user-defined files. If we're happy with the way that works, then we
> >>>>>> should
> >>>>>> definitely propose it for inclusion in the standard (not the
> specific
> >>>>>> filesystems, perhaps, but the mechanism for requesting and
> interacting
> >>>>>> with
> >>>>>> them)
> >>>>>>
> >>>>>> Ian
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>   Sent from my iPhone
> >>>>>>
> >>>>>>>
> >>>>>>>    On Nov 15, 2013, at 8:36 AM, Ian Clelland <
> iclelland@chromium.org
> >>>>>>> >
> >>>>>>>
> >>>>>>>
> >>>>>>>>   wrote:
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>  I've created CB-5403 as the über-ticket for this; various bits of
> it
> >>>>>>>> are
> >>>>>>>>
> >>>>>>>>   in
> >>>>>>>>
> >>>>>>>
> >>>>>>>   subtasks.
> >>>>>>>
> >>>>>>>>
> >>>>>>>> Once I have the tests and the JS updated, then platform owners can
> >>>>>>>> take
> >>>>>>>> a
> >>>>>>>> look and see whether they have anything to do to support their own
> >>>>>>>>
> >>>>>>>>   schemes.
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>  As general guidelines, I would say:
> >>>>>>>>
> >>>>>>>> * If you can intercept filesystem://* URLs in native code, and
> >>>>>>>> return
> >>>>>>>> arbitrary responses, then do it! It will let your platform support
> >>>>>>>> new
> >>>>>>>> filesystems in the future as we come up with them. Add a couple of
> >>>>>>>>
> >>>>>>>>   subtasks
> >>>>>>>>
> >>>>>>>
> >>>>>>>   for CB-5403 for your platform and go.
> >>>>>>>
> >>>>>>>>
> >>>>>>>> * If you can't do that, but you *can* provide access to things
> like
> >>>>>>>> media
> >>>>>>>> through special urls, then try that! You should be able to create
> a
> >>>>>>>> FileSystem object that uses that URL as its root, and everything
> >>>>>>>> should
> >>>>>>>> work. Add a subtask for that, and see what you can do.
> >>>>>>>>
> >>>>>>>> * If you can't do that either, and you just want to stick with the
> >>>>>>>>
> >>>>>>>>   file:///
> >>>>>>>>
> >>>>>>>
> >>>>>>>   urls that we are currently using, then don't do anything; nothing
> >>>>>>>
> >>>>>>>> should
> >>>>>>>> change for you.
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Fri, Nov 15, 2013 at 10:36 AM, Ian Clelland <
> >>>>>>>> iclelland@chromium.org
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>   On Fri, Nov 15, 2013 at 10:03 AM, Marcel Kinard <
> >>>>>>>> cmarcelk@gmail.com
> >>>>>>>>
> >>>>>>>>>
> >>>>>>>>>   wrote:
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>     Ian, will there be changes that app developers will need to
> do?
> >>>>>>>>> If
> >>>>>>>>> so,
> >>>>>>>>>
> >>>>>>>>>  those should be clearly documented in a migration guide. If not,
> >>>>>>>>>> it
> >>>>>>>>>>
> >>>>>>>>>>   sounds
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>    like the [improved] tests should pass on both the old and new
> >>>>>>>
> >>>>>>>> versions,
> >>>>>>>>
> >>>>>>>>  which would be sweet.
> >>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>  The filesystem URLs themselves will change as a result of this.
> >>>>>>>>> The
> >>>>>>>>>
> >>>>>>>>>   tests
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>   should pass on both old and new versions, but the tests mint
> their
> >>>>>>> own
> >>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>>   URLs
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>   for testing against -- each version is internally consistent, but
> >>>>>>> if
> >>>>>>>
> >>>>>>>> an
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>>   app
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>   is saving internal URLs and tries to dereference an old
> (file:///)
> >>>>>>> url
> >>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>>   with
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>   the new plugin, it will probably not resolve.
> >>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>> Maybe there's a transition path here -- it might be possible to
> >>>>>>>>> allow
> >>>>>>>>> window.resolveLocalFileSystemURL to access files with the old
> URL,
> >>>>>>>>> but
> >>>>>>>>> never actually generate URLs.
> >>>>>>>>>
> >>>>>>>>> That would mean that
> >>>>>>>>>
> >>>>>>>>> (resolveLocalFileSystemURL(a)).toURL() !== a,
> >>>>>>>>>
> >>>>>>>>> but I don't think that we depend on that as an identity.
> >>>>>>>>>
> >>>>>>>>> Is there work which needs to be done on the other platforms (BB,
> >>>>>>>>> WP8,
> >>>>>>>>>
> >>>>>>>>>   Win8, FFOS, etc) so that those platforms don't get left behind?
> >>>>>>>>> If
> >>>>>>>>>
> >>>>>>>>>> so,
> >>>>>>>>>> would it make sense to reach out to those platform owners and at
> >>>>>>>>>> least
> >>>>>>>>>>
> >>>>>>>>>>   get
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>    Jira items created with some notes?
> >>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>>
> >>>>>>>>>>  Yes. I'm going to create a ticket for this, and we can add
> >>>>>>>>> platform-specific tasks to it.
> >>>>>>>>>
> >>>>>>>>> Other platforms shouldn't *have* to do anything, and some
> platforms
> >>>>>>>>> *can't* do anything, because they cannot access anything other
> than
> >>>>>>>>> file:/// urls anyway.
> >>>>>>>>>
> >>>>>>>>> However, if the other platforms want to get in on the goodness,
> and
> >>>>>>>>>
> >>>>>>>>>   start
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>   supporting their own URL schemes (I think the BB folks can use
> >>>>>>>
> >>>>>>>> something
> >>>>>>>>
> >>>>>>>>  like local:// for their filesystem access), then that'll be the
> >>>>>>>>> place
> >>>>>>>>> to
> >>>>>>>>> keep everything organized.
> >>>>>>>>>
> >>>>>>>>> I'll post back once I have that issue set up.
> >>>>>>>>>
> >>>>>>>>> Ian
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>    Sounds like you've built some very significant improvements.
> >>>>>>>>> Thanks!
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>>    On Nov 15, 2013, at 9:29 AM, Ian Clelland <
> >>>>>>>>>> iclelland@chromium.org
> >>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>   wrote:
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>    After working with the File plugin for a week or so, I've gotten
> >>>>>>>> it
> >>>>>>>>
> >>>>>>>>>
> >>>>>>>>>>>   more or
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>   less where I want it to be on iOS, and am working through
> >>>>>>>>>> Android.
> >>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>   Looking
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>   at the end result, though, I expect that over 90% of the code
> >>>>>>>>>> has
> >>>>>>>>>>
> >>>>>>>>>>> been
> >>>>>>>>>>> touched in some way. (It's a huge diff.)
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>>
> >>
> >
>

Re: Transitioning to a better File API implementation

[Ian Clelland <ic...@chromium.org>]

On Tue, Dec 10, 2013 at 4:37 PM, Michael Gauthier <mi...@silverorange.com>wrote:

> On 10/12/13 05:02 PM, Ian Clelland wrote:
>
>> Yep.
>>
>> I think that Library is the more natural place for the HTML persistent
>> filesystem, since it's used by an app for whatever persistent storage it
>> needs, without exposing all of it's implementation details to the user.
>> (There's lots of room for debate on this, I'm sure)
>>
>> The trouble is that we've historically used /Documents for persistent
>> storage, and changing that will break apps.
>>
>>  I'm fine with a BC break, but I don't have to maintain any legacy
> applications. /Library does make more sense as the default for PERSISTENT.
> The big problem with BC is for installed apps with existing data on the
> filesystem, right?


Exactly. I'd hate for developers to update their plugins, and push a new
version of their app that seems fine -- everything works great when you
start from scratch -- but users with existing data find themselves
completely locked out of it.


>
>  One idea is to allow something like requestFilesystem(DOCUMENT), in
>> addition to PERSISTENT and TEMPORARY. Another suggestion has been to add
>> extra arguments -- hints -- such as "{sync: true}", or maybe in this case
>> "{purpose: documents}" to specify the attributes of the filesystem that is
>> returned.
>>
> >
> {sync: true} is a bit tricky because /Library/Cache is not synced but
> /Library/Application Data is synced. Having a DOCUMENT type would match
> /tmp and /Library as the top-level dirs mapping to file-system constants.
>
> All my feedback is only from the iOS perspective though. Not sure if these
> ideas make other platforms more or less complicated.
>
> Cheers,
> Mike
>
>
>>
>>
>> On Tue, Dec 10, 2013 at 2:39 PM, Michael Gauthier <mike@silverorange.com
>> >wrote:
>>
>>  Hmm.. The two directories have different defined roles on iOS and both
>>> are
>>> normal targets for applications to read/write files. Library is for cache
>>> data or app-specific resources. Documents is for saving/loading actual
>>> things created by users within apps.
>>>
>>> See https://developer.apple.com/library/ios/documentation/
>>> FileManagement/Conceptual/FileSystemProgrammingGuide/FileSystemOverview/
>>> FileSystemOverview.html#//apple_ref/doc/uid/TP40010672-CH2-SW14
>>>
>>> Supporting both should be a goal.
>>>
>>> Cheers,
>>> Mike
>>>
>>>
>>>
>>> On 2013-12-10 14:39, Ian Clelland wrote:
>>>
>>>  If you could do that before, it was probably a bug :) (You shouldn't be
>>>> able to use getParent on the filesystem root, and the native code was
>>>> supposed to be checking for the correct path prefix before allowing
>>>> access
>>>> to any files on the device)
>>>>
>>>> At the moment, it's not possible to do that -- the filesystems should be
>>>> properly isolated, and only allow access to correctly formed urls, like
>>>> filesystem://localhost/persistent/some-file-here.txt.
>>>>
>>>> What we *can* do easily, though, is allow a new URL scheme for library
>>>> assets; something like filesystem://localhost/
>>>> library/some-file-here.txt,
>>>> and we can have a filesystem handler for those URLs. That'll work if
>>>> your
>>>> use case is just "I need access to files in /Library", rather than "I
>>>> need
>>>> to get to them via string manipulation".
>>>>
>>>> I've also had some discussions about making /Library the default place
>>>> for
>>>> the persistent filesystem, and leaving /Documents either just for legacy
>>>> apps, or making *that* the target of the special URL scheme. That's a
>>>> proposal for a different day, though. There are some pretty big
>>>> backwards-compatibility issues there.
>>>>
>>>>
>>>>
>>>> On Tue, Dec 10, 2013 at 11:54 AM, Michael Gauthier <
>>>> mike@silverorange.com
>>>>
>>>>> wrote:
>>>>>
>>>>
>>>>   Ian,
>>>>
>>>>>
>>>>> With the new URLs will it be possible to write to both /Documents and
>>>>> /Library for iOS apps? In the old version, the filesystem root resolved
>>>>> as
>>>>> /Documents but it was possible to get to /Library by navigating the the
>>>>> parent dir.
>>>>>
>>>>> Cheers,
>>>>> Mike
>>>>>
>>>>>
>>>>>
>>>>> On 2013-11-15 15:19, Ian Clelland wrote:
>>>>>
>>>>>   On Fri, Nov 15, 2013 at 1:40 PM, purplecabbage <
>>>>> purplecabbage@gmail.com
>>>>>
>>>>>>
>>>>>>>  wrote:
>>>>>>
>>>>>>    Considering the magnitude of the changes I would have expected that
>>>>>> this
>>>>>>
>>>>>>  was just a new file plugin. The previous version was based on a spec,
>>>>>>> and
>>>>>>> if we are deviating from it we should probably maintain both, and
>>>>>>> possibly
>>>>>>> even make a recommendation to the w3c.
>>>>>>> I hope we at least do a major version update for this if APIs have
>>>>>>> changed.
>>>>>>>
>>>>>>>
>>>>>>>   Yes, definitely a major version bump, at least so that devs aren't
>>>>>>>
>>>>>> caught
>>>>>> by the URL-format-change.
>>>>>>
>>>>>> There aren't any changes to external APIs; I've been very careful to
>>>>>> maintain conformance with the existing tests, except where those tests
>>>>>> have
>>>>>> been for undocumented implementation details. The only app-visible
>>>>>> change
>>>>>> is that URLs returned by the .toURL() method will look like
>>>>>>
>>>>>> filesystem://localhost/persistent/my/file.jpg
>>>>>>
>>>>>> rather than
>>>>>>
>>>>>> file:///var/mobile/Applications/<app id>/Documents/my/file.jpg
>>>>>>
>>>>>> We are still following the spec -- in fact, this brings us closer in
>>>>>> line
>>>>>> with the W3C File API spec, on these points:
>>>>>>
>>>>>> http://www.w3.org/TR/file-system-api/#widl-Entry-fullPath -- The
>>>>>> fullPath
>>>>>> of an entry should be the path from the root (of the HTML filesystem,
>>>>>> not
>>>>>> the underlying device file system).
>>>>>>
>>>>>> http://www.w3.org/TR/file-system-api/#widl-Entry-toURL-DOMString --
>>>>>> This
>>>>>> is
>>>>>> a note, rather than a requirement, but a filesystem url scheme is
>>>>>> mentioned
>>>>>> there.
>>>>>>
>>>>>> We're considering extending the spec in one way, which should be
>>>>>> compatible
>>>>>> with the spirit of the spec, and backwards-compatible with the
>>>>>> previous
>>>>>> API. That is the ability to declare new filesystem types, beyond
>>>>>> "temporary" and "persistent", so that we can access things like device
>>>>>> media and app bundle assets using the same APIs that we use for
>>>>>> accessing
>>>>>> user-defined files. If we're happy with the way that works, then we
>>>>>> should
>>>>>> definitely propose it for inclusion in the standard (not the specific
>>>>>> filesystems, perhaps, but the mechanism for requesting and interacting
>>>>>> with
>>>>>> them)
>>>>>>
>>>>>> Ian
>>>>>>
>>>>>>
>>>>>>
>>>>>>   Sent from my iPhone
>>>>>>
>>>>>>>
>>>>>>>    On Nov 15, 2013, at 8:36 AM, Ian Clelland <iclelland@chromium.org
>>>>>>> >
>>>>>>>
>>>>>>>
>>>>>>>>   wrote:
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>  I've created CB-5403 as the über-ticket for this; various bits of it
>>>>>>>> are
>>>>>>>>
>>>>>>>>   in
>>>>>>>>
>>>>>>>
>>>>>>>   subtasks.
>>>>>>>
>>>>>>>>
>>>>>>>> Once I have the tests and the JS updated, then platform owners can
>>>>>>>> take
>>>>>>>> a
>>>>>>>> look and see whether they have anything to do to support their own
>>>>>>>>
>>>>>>>>   schemes.
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>  As general guidelines, I would say:
>>>>>>>>
>>>>>>>> * If you can intercept filesystem://* URLs in native code, and
>>>>>>>> return
>>>>>>>> arbitrary responses, then do it! It will let your platform support
>>>>>>>> new
>>>>>>>> filesystems in the future as we come up with them. Add a couple of
>>>>>>>>
>>>>>>>>   subtasks
>>>>>>>>
>>>>>>>
>>>>>>>   for CB-5403 for your platform and go.
>>>>>>>
>>>>>>>>
>>>>>>>> * If you can't do that, but you *can* provide access to things like
>>>>>>>> media
>>>>>>>> through special urls, then try that! You should be able to create a
>>>>>>>> FileSystem object that uses that URL as its root, and everything
>>>>>>>> should
>>>>>>>> work. Add a subtask for that, and see what you can do.
>>>>>>>>
>>>>>>>> * If you can't do that either, and you just want to stick with the
>>>>>>>>
>>>>>>>>   file:///
>>>>>>>>
>>>>>>>
>>>>>>>   urls that we are currently using, then don't do anything; nothing
>>>>>>>
>>>>>>>> should
>>>>>>>> change for you.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Fri, Nov 15, 2013 at 10:36 AM, Ian Clelland <
>>>>>>>> iclelland@chromium.org
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>   On Fri, Nov 15, 2013 at 10:03 AM, Marcel Kinard <
>>>>>>>> cmarcelk@gmail.com
>>>>>>>>
>>>>>>>>>
>>>>>>>>>   wrote:
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>     Ian, will there be changes that app developers will need to do?
>>>>>>>>> If
>>>>>>>>> so,
>>>>>>>>>
>>>>>>>>>  those should be clearly documented in a migration guide. If not,
>>>>>>>>>> it
>>>>>>>>>>
>>>>>>>>>>   sounds
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>    like the [improved] tests should pass on both the old and new
>>>>>>>
>>>>>>>> versions,
>>>>>>>>
>>>>>>>>  which would be sweet.
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>  The filesystem URLs themselves will change as a result of this.
>>>>>>>>> The
>>>>>>>>>
>>>>>>>>>   tests
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>   should pass on both old and new versions, but the tests mint their
>>>>>>> own
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>>   URLs
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>   for testing against -- each version is internally consistent, but
>>>>>>> if
>>>>>>>
>>>>>>>> an
>>>>>>>>
>>>>>>>>
>>>>>>>>>   app
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>   is saving internal URLs and tries to dereference an old (file:///)
>>>>>>> url
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>>   with
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>   the new plugin, it will probably not resolve.
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> Maybe there's a transition path here -- it might be possible to
>>>>>>>>> allow
>>>>>>>>> window.resolveLocalFileSystemURL to access files with the old URL,
>>>>>>>>> but
>>>>>>>>> never actually generate URLs.
>>>>>>>>>
>>>>>>>>> That would mean that
>>>>>>>>>
>>>>>>>>> (resolveLocalFileSystemURL(a)).toURL() !== a,
>>>>>>>>>
>>>>>>>>> but I don't think that we depend on that as an identity.
>>>>>>>>>
>>>>>>>>> Is there work which needs to be done on the other platforms (BB,
>>>>>>>>> WP8,
>>>>>>>>>
>>>>>>>>>   Win8, FFOS, etc) so that those platforms don't get left behind?
>>>>>>>>> If
>>>>>>>>>
>>>>>>>>>> so,
>>>>>>>>>> would it make sense to reach out to those platform owners and at
>>>>>>>>>> least
>>>>>>>>>>
>>>>>>>>>>   get
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>    Jira items created with some notes?
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>>>  Yes. I'm going to create a ticket for this, and we can add
>>>>>>>>> platform-specific tasks to it.
>>>>>>>>>
>>>>>>>>> Other platforms shouldn't *have* to do anything, and some platforms
>>>>>>>>> *can't* do anything, because they cannot access anything other than
>>>>>>>>> file:/// urls anyway.
>>>>>>>>>
>>>>>>>>> However, if the other platforms want to get in on the goodness, and
>>>>>>>>>
>>>>>>>>>   start
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>   supporting their own URL schemes (I think the BB folks can use
>>>>>>>
>>>>>>>> something
>>>>>>>>
>>>>>>>>  like local:// for their filesystem access), then that'll be the
>>>>>>>>> place
>>>>>>>>> to
>>>>>>>>> keep everything organized.
>>>>>>>>>
>>>>>>>>> I'll post back once I have that issue set up.
>>>>>>>>>
>>>>>>>>> Ian
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>    Sounds like you've built some very significant improvements.
>>>>>>>>> Thanks!
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>    On Nov 15, 2013, at 9:29 AM, Ian Clelland <
>>>>>>>>>> iclelland@chromium.org
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>   wrote:
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>    After working with the File plugin for a week or so, I've gotten
>>>>>>>> it
>>>>>>>>
>>>>>>>>>
>>>>>>>>>>>   more or
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>   less where I want it to be on iOS, and am working through
>>>>>>>>>> Android.
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>   Looking
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>   at the end result, though, I expect that over 90% of the code
>>>>>>>>>> has
>>>>>>>>>>
>>>>>>>>>>> been
>>>>>>>>>>> touched in some way. (It's a huge diff.)
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>

Re: Transitioning to a better File API implementation

[Michael Gauthier <mi...@silverorange.com>]

On 10/12/13 05:02 PM, Ian Clelland wrote:
> Yep.
>
> I think that Library is the more natural place for the HTML persistent
> filesystem, since it's used by an app for whatever persistent storage it
> needs, without exposing all of it's implementation details to the user.
> (There's lots of room for debate on this, I'm sure)
>
> The trouble is that we've historically used /Documents for persistent
> storage, and changing that will break apps.
>
I'm fine with a BC break, but I don't have to maintain any legacy 
applications. /Library does make more sense as the default for 
PERSISTENT. The big problem with BC is for installed apps with existing 
data on the filesystem, right?

> One idea is to allow something like requestFilesystem(DOCUMENT), in
> addition to PERSISTENT and TEMPORARY. Another suggestion has been to add
> extra arguments -- hints -- such as "{sync: true}", or maybe in this case
> "{purpose: documents}" to specify the attributes of the filesystem that is
> returned.
 >
{sync: true} is a bit tricky because /Library/Cache is not synced but 
/Library/Application Data is synced. Having a DOCUMENT type would match 
/tmp and /Library as the top-level dirs mapping to file-system constants.

All my feedback is only from the iOS perspective though. Not sure if 
these ideas make other platforms more or less complicated.

Cheers,
Mike
>
>
>
> On Tue, Dec 10, 2013 at 2:39 PM, Michael Gauthier <mi...@silverorange.com>wrote:
>
>> Hmm.. The two directories have different defined roles on iOS and both are
>> normal targets for applications to read/write files. Library is for cache
>> data or app-specific resources. Documents is for saving/loading actual
>> things created by users within apps.
>>
>> See https://developer.apple.com/library/ios/documentation/
>> FileManagement/Conceptual/FileSystemProgrammingGuide/FileSystemOverview/
>> FileSystemOverview.html#//apple_ref/doc/uid/TP40010672-CH2-SW14
>>
>> Supporting both should be a goal.
>>
>> Cheers,
>> Mike
>>
>>
>>
>> On 2013-12-10 14:39, Ian Clelland wrote:
>>
>>> If you could do that before, it was probably a bug :) (You shouldn't be
>>> able to use getParent on the filesystem root, and the native code was
>>> supposed to be checking for the correct path prefix before allowing access
>>> to any files on the device)
>>>
>>> At the moment, it's not possible to do that -- the filesystems should be
>>> properly isolated, and only allow access to correctly formed urls, like
>>> filesystem://localhost/persistent/some-file-here.txt.
>>>
>>> What we *can* do easily, though, is allow a new URL scheme for library
>>> assets; something like filesystem://localhost/library/some-file-here.txt,
>>> and we can have a filesystem handler for those URLs. That'll work if your
>>> use case is just "I need access to files in /Library", rather than "I need
>>> to get to them via string manipulation".
>>>
>>> I've also had some discussions about making /Library the default place for
>>> the persistent filesystem, and leaving /Documents either just for legacy
>>> apps, or making *that* the target of the special URL scheme. That's a
>>> proposal for a different day, though. There are some pretty big
>>> backwards-compatibility issues there.
>>>
>>>
>>>
>>> On Tue, Dec 10, 2013 at 11:54 AM, Michael Gauthier <mike@silverorange.com
>>>> wrote:
>>>
>>>   Ian,
>>>>
>>>> With the new URLs will it be possible to write to both /Documents and
>>>> /Library for iOS apps? In the old version, the filesystem root resolved
>>>> as
>>>> /Documents but it was possible to get to /Library by navigating the the
>>>> parent dir.
>>>>
>>>> Cheers,
>>>> Mike
>>>>
>>>>
>>>>
>>>> On 2013-11-15 15:19, Ian Clelland wrote:
>>>>
>>>>   On Fri, Nov 15, 2013 at 1:40 PM, purplecabbage <purplecabbage@gmail.com
>>>>>>
>>>>> wrote:
>>>>>
>>>>>    Considering the magnitude of the changes I would have expected that
>>>>> this
>>>>>
>>>>>> was just a new file plugin. The previous version was based on a spec,
>>>>>> and
>>>>>> if we are deviating from it we should probably maintain both, and
>>>>>> possibly
>>>>>> even make a recommendation to the w3c.
>>>>>> I hope we at least do a major version update for this if APIs have
>>>>>> changed.
>>>>>>
>>>>>>
>>>>>>   Yes, definitely a major version bump, at least so that devs aren't
>>>>> caught
>>>>> by the URL-format-change.
>>>>>
>>>>> There aren't any changes to external APIs; I've been very careful to
>>>>> maintain conformance with the existing tests, except where those tests
>>>>> have
>>>>> been for undocumented implementation details. The only app-visible
>>>>> change
>>>>> is that URLs returned by the .toURL() method will look like
>>>>>
>>>>> filesystem://localhost/persistent/my/file.jpg
>>>>>
>>>>> rather than
>>>>>
>>>>> file:///var/mobile/Applications/<app id>/Documents/my/file.jpg
>>>>>
>>>>> We are still following the spec -- in fact, this brings us closer in
>>>>> line
>>>>> with the W3C File API spec, on these points:
>>>>>
>>>>> http://www.w3.org/TR/file-system-api/#widl-Entry-fullPath -- The
>>>>> fullPath
>>>>> of an entry should be the path from the root (of the HTML filesystem,
>>>>> not
>>>>> the underlying device file system).
>>>>>
>>>>> http://www.w3.org/TR/file-system-api/#widl-Entry-toURL-DOMString --
>>>>> This
>>>>> is
>>>>> a note, rather than a requirement, but a filesystem url scheme is
>>>>> mentioned
>>>>> there.
>>>>>
>>>>> We're considering extending the spec in one way, which should be
>>>>> compatible
>>>>> with the spirit of the spec, and backwards-compatible with the previous
>>>>> API. That is the ability to declare new filesystem types, beyond
>>>>> "temporary" and "persistent", so that we can access things like device
>>>>> media and app bundle assets using the same APIs that we use for
>>>>> accessing
>>>>> user-defined files. If we're happy with the way that works, then we
>>>>> should
>>>>> definitely propose it for inclusion in the standard (not the specific
>>>>> filesystems, perhaps, but the mechanism for requesting and interacting
>>>>> with
>>>>> them)
>>>>>
>>>>> Ian
>>>>>
>>>>>
>>>>>
>>>>>   Sent from my iPhone
>>>>>>
>>>>>>    On Nov 15, 2013, at 8:36 AM, Ian Clelland <ic...@chromium.org>
>>>>>>
>>>>>>>
>>>>>>>   wrote:
>>>>>>
>>>>>>
>>>>>>> I've created CB-5403 as the über-ticket for this; various bits of it
>>>>>>> are
>>>>>>>
>>>>>>>   in
>>>>>>
>>>>>>   subtasks.
>>>>>>>
>>>>>>> Once I have the tests and the JS updated, then platform owners can
>>>>>>> take
>>>>>>> a
>>>>>>> look and see whether they have anything to do to support their own
>>>>>>>
>>>>>>>   schemes.
>>>>>>
>>>>>>
>>>>>>> As general guidelines, I would say:
>>>>>>>
>>>>>>> * If you can intercept filesystem://* URLs in native code, and return
>>>>>>> arbitrary responses, then do it! It will let your platform support new
>>>>>>> filesystems in the future as we come up with them. Add a couple of
>>>>>>>
>>>>>>>   subtasks
>>>>>>
>>>>>>   for CB-5403 for your platform and go.
>>>>>>>
>>>>>>> * If you can't do that, but you *can* provide access to things like
>>>>>>> media
>>>>>>> through special urls, then try that! You should be able to create a
>>>>>>> FileSystem object that uses that URL as its root, and everything
>>>>>>> should
>>>>>>> work. Add a subtask for that, and see what you can do.
>>>>>>>
>>>>>>> * If you can't do that either, and you just want to stick with the
>>>>>>>
>>>>>>>   file:///
>>>>>>
>>>>>>   urls that we are currently using, then don't do anything; nothing
>>>>>>> should
>>>>>>> change for you.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Fri, Nov 15, 2013 at 10:36 AM, Ian Clelland <
>>>>>>> iclelland@chromium.org
>>>>>>> wrote:
>>>>>>>
>>>>>>>
>>>>>>>   On Fri, Nov 15, 2013 at 10:03 AM, Marcel Kinard <cmarcelk@gmail.com
>>>>>>>>
>>>>>>>>   wrote:
>>>>>>>
>>>>>>>
>>>>>>>>    Ian, will there be changes that app developers will need to do? If
>>>>>>>> so,
>>>>>>>>
>>>>>>>>> those should be clearly documented in a migration guide. If not, it
>>>>>>>>>
>>>>>>>>>   sounds
>>>>>>>>
>>>>>>>
>>>>>>   like the [improved] tests should pass on both the old and new
>>>>>>> versions,
>>>>>>>
>>>>>>>> which would be sweet.
>>>>>>>>>
>>>>>>>>>
>>>>>>>> The filesystem URLs themselves will change as a result of this. The
>>>>>>>>
>>>>>>>>   tests
>>>>>>>
>>>>>>
>>>>>>   should pass on both old and new versions, but the tests mint their own
>>>>>>>
>>>>>>>>
>>>>>>>>   URLs
>>>>>>>
>>>>>>
>>>>>>   for testing against -- each version is internally consistent, but if
>>>>>>> an
>>>>>>>
>>>>>>>>
>>>>>>>>   app
>>>>>>>
>>>>>>
>>>>>>   is saving internal URLs and tries to dereference an old (file:///) url
>>>>>>>
>>>>>>>>
>>>>>>>>   with
>>>>>>>
>>>>>>
>>>>>>   the new plugin, it will probably not resolve.
>>>>>>>
>>>>>>>>
>>>>>>>> Maybe there's a transition path here -- it might be possible to allow
>>>>>>>> window.resolveLocalFileSystemURL to access files with the old URL,
>>>>>>>> but
>>>>>>>> never actually generate URLs.
>>>>>>>>
>>>>>>>> That would mean that
>>>>>>>>
>>>>>>>> (resolveLocalFileSystemURL(a)).toURL() !== a,
>>>>>>>>
>>>>>>>> but I don't think that we depend on that as an identity.
>>>>>>>>
>>>>>>>> Is there work which needs to be done on the other platforms (BB, WP8,
>>>>>>>>
>>>>>>>>   Win8, FFOS, etc) so that those platforms don't get left behind? If
>>>>>>>>> so,
>>>>>>>>> would it make sense to reach out to those platform owners and at
>>>>>>>>> least
>>>>>>>>>
>>>>>>>>>   get
>>>>>>>>
>>>>>>>
>>>>>>   Jira items created with some notes?
>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>> Yes. I'm going to create a ticket for this, and we can add
>>>>>>>> platform-specific tasks to it.
>>>>>>>>
>>>>>>>> Other platforms shouldn't *have* to do anything, and some platforms
>>>>>>>> *can't* do anything, because they cannot access anything other than
>>>>>>>> file:/// urls anyway.
>>>>>>>>
>>>>>>>> However, if the other platforms want to get in on the goodness, and
>>>>>>>>
>>>>>>>>   start
>>>>>>>
>>>>>>
>>>>>>   supporting their own URL schemes (I think the BB folks can use
>>>>>>> something
>>>>>>>
>>>>>>>> like local:// for their filesystem access), then that'll be the place
>>>>>>>> to
>>>>>>>> keep everything organized.
>>>>>>>>
>>>>>>>> I'll post back once I have that issue set up.
>>>>>>>>
>>>>>>>> Ian
>>>>>>>>
>>>>>>>>
>>>>>>>>    Sounds like you've built some very significant improvements.
>>>>>>>> Thanks!
>>>>>>>>
>>>>>>>>>
>>>>>>>>>    On Nov 15, 2013, at 9:29 AM, Ian Clelland <iclelland@chromium.org
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>   wrote:
>>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>>>   After working with the File plugin for a week or so, I've gotten it
>>>>>>>>>>
>>>>>>>>>>   more or
>>>>>>>>>
>>>>>>>>>   less where I want it to be on iOS, and am working through Android.
>>>>>>>>>>
>>>>>>>>>>   Looking
>>>>>>>>>
>>>>>>>>>   at the end result, though, I expect that over 90% of the code has
>>>>>>>>>> been
>>>>>>>>>> touched in some way. (It's a huge diff.)
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>

Re: Transitioning to a better File API implementation

[Ian Clelland <ic...@chromium.org>]

Yep.

I think that Library is the more natural place for the HTML persistent
filesystem, since it's used by an app for whatever persistent storage it
needs, without exposing all of it's implementation details to the user.
(There's lots of room for debate on this, I'm sure)

The trouble is that we've historically used /Documents for persistent
storage, and changing that will break apps.

One idea is to allow something like requestFilesystem(DOCUMENT), in
addition to PERSISTENT and TEMPORARY. Another suggestion has been to add
extra arguments -- hints -- such as "{sync: true}", or maybe in this case
"{purpose: documents}" to specify the attributes of the filesystem that is
returned.



On Tue, Dec 10, 2013 at 2:39 PM, Michael Gauthier <mi...@silverorange.com>wrote:

> Hmm.. The two directories have different defined roles on iOS and both are
> normal targets for applications to read/write files. Library is for cache
> data or app-specific resources. Documents is for saving/loading actual
> things created by users within apps.
>
> See https://developer.apple.com/library/ios/documentation/
> FileManagement/Conceptual/FileSystemProgrammingGuide/FileSystemOverview/
> FileSystemOverview.html#//apple_ref/doc/uid/TP40010672-CH2-SW14
>
> Supporting both should be a goal.
>
> Cheers,
> Mike
>
>
>
> On 2013-12-10 14:39, Ian Clelland wrote:
>
>> If you could do that before, it was probably a bug :) (You shouldn't be
>> able to use getParent on the filesystem root, and the native code was
>> supposed to be checking for the correct path prefix before allowing access
>> to any files on the device)
>>
>> At the moment, it's not possible to do that -- the filesystems should be
>> properly isolated, and only allow access to correctly formed urls, like
>> filesystem://localhost/persistent/some-file-here.txt.
>>
>> What we *can* do easily, though, is allow a new URL scheme for library
>> assets; something like filesystem://localhost/library/some-file-here.txt,
>> and we can have a filesystem handler for those URLs. That'll work if your
>> use case is just "I need access to files in /Library", rather than "I need
>> to get to them via string manipulation".
>>
>> I've also had some discussions about making /Library the default place for
>> the persistent filesystem, and leaving /Documents either just for legacy
>> apps, or making *that* the target of the special URL scheme. That's a
>> proposal for a different day, though. There are some pretty big
>> backwards-compatibility issues there.
>>
>>
>>
>> On Tue, Dec 10, 2013 at 11:54 AM, Michael Gauthier <mike@silverorange.com
>> >wrote:
>>
>>  Ian,
>>>
>>> With the new URLs will it be possible to write to both /Documents and
>>> /Library for iOS apps? In the old version, the filesystem root resolved
>>> as
>>> /Documents but it was possible to get to /Library by navigating the the
>>> parent dir.
>>>
>>> Cheers,
>>> Mike
>>>
>>>
>>>
>>> On 2013-11-15 15:19, Ian Clelland wrote:
>>>
>>>  On Fri, Nov 15, 2013 at 1:40 PM, purplecabbage <purplecabbage@gmail.com
>>>> >
>>>> wrote:
>>>>
>>>>   Considering the magnitude of the changes I would have expected that
>>>> this
>>>>
>>>>> was just a new file plugin. The previous version was based on a spec,
>>>>> and
>>>>> if we are deviating from it we should probably maintain both, and
>>>>> possibly
>>>>> even make a recommendation to the w3c.
>>>>> I hope we at least do a major version update for this if APIs have
>>>>> changed.
>>>>>
>>>>>
>>>>>  Yes, definitely a major version bump, at least so that devs aren't
>>>> caught
>>>> by the URL-format-change.
>>>>
>>>> There aren't any changes to external APIs; I've been very careful to
>>>> maintain conformance with the existing tests, except where those tests
>>>> have
>>>> been for undocumented implementation details. The only app-visible
>>>> change
>>>> is that URLs returned by the .toURL() method will look like
>>>>
>>>> filesystem://localhost/persistent/my/file.jpg
>>>>
>>>> rather than
>>>>
>>>> file:///var/mobile/Applications/<app id>/Documents/my/file.jpg
>>>>
>>>> We are still following the spec -- in fact, this brings us closer in
>>>> line
>>>> with the W3C File API spec, on these points:
>>>>
>>>> http://www.w3.org/TR/file-system-api/#widl-Entry-fullPath -- The
>>>> fullPath
>>>> of an entry should be the path from the root (of the HTML filesystem,
>>>> not
>>>> the underlying device file system).
>>>>
>>>> http://www.w3.org/TR/file-system-api/#widl-Entry-toURL-DOMString --
>>>> This
>>>> is
>>>> a note, rather than a requirement, but a filesystem url scheme is
>>>> mentioned
>>>> there.
>>>>
>>>> We're considering extending the spec in one way, which should be
>>>> compatible
>>>> with the spirit of the spec, and backwards-compatible with the previous
>>>> API. That is the ability to declare new filesystem types, beyond
>>>> "temporary" and "persistent", so that we can access things like device
>>>> media and app bundle assets using the same APIs that we use for
>>>> accessing
>>>> user-defined files. If we're happy with the way that works, then we
>>>> should
>>>> definitely propose it for inclusion in the standard (not the specific
>>>> filesystems, perhaps, but the mechanism for requesting and interacting
>>>> with
>>>> them)
>>>>
>>>> Ian
>>>>
>>>>
>>>>
>>>>  Sent from my iPhone
>>>>>
>>>>>   On Nov 15, 2013, at 8:36 AM, Ian Clelland <ic...@chromium.org>
>>>>>
>>>>>>
>>>>>>  wrote:
>>>>>
>>>>>
>>>>>> I've created CB-5403 as the über-ticket for this; various bits of it
>>>>>> are
>>>>>>
>>>>>>  in
>>>>>
>>>>>  subtasks.
>>>>>>
>>>>>> Once I have the tests and the JS updated, then platform owners can
>>>>>> take
>>>>>> a
>>>>>> look and see whether they have anything to do to support their own
>>>>>>
>>>>>>  schemes.
>>>>>
>>>>>
>>>>>> As general guidelines, I would say:
>>>>>>
>>>>>> * If you can intercept filesystem://* URLs in native code, and return
>>>>>> arbitrary responses, then do it! It will let your platform support new
>>>>>> filesystems in the future as we come up with them. Add a couple of
>>>>>>
>>>>>>  subtasks
>>>>>
>>>>>  for CB-5403 for your platform and go.
>>>>>>
>>>>>> * If you can't do that, but you *can* provide access to things like
>>>>>> media
>>>>>> through special urls, then try that! You should be able to create a
>>>>>> FileSystem object that uses that URL as its root, and everything
>>>>>> should
>>>>>> work. Add a subtask for that, and see what you can do.
>>>>>>
>>>>>> * If you can't do that either, and you just want to stick with the
>>>>>>
>>>>>>  file:///
>>>>>
>>>>>  urls that we are currently using, then don't do anything; nothing
>>>>>> should
>>>>>> change for you.
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Fri, Nov 15, 2013 at 10:36 AM, Ian Clelland <
>>>>>> iclelland@chromium.org
>>>>>> wrote:
>>>>>>
>>>>>>
>>>>>>  On Fri, Nov 15, 2013 at 10:03 AM, Marcel Kinard <cmarcelk@gmail.com
>>>>>>>
>>>>>>>  wrote:
>>>>>>
>>>>>>
>>>>>>>   Ian, will there be changes that app developers will need to do? If
>>>>>>> so,
>>>>>>>
>>>>>>>> those should be clearly documented in a migration guide. If not, it
>>>>>>>>
>>>>>>>>  sounds
>>>>>>>
>>>>>>
>>>>>  like the [improved] tests should pass on both the old and new
>>>>>> versions,
>>>>>>
>>>>>>> which would be sweet.
>>>>>>>>
>>>>>>>>
>>>>>>> The filesystem URLs themselves will change as a result of this. The
>>>>>>>
>>>>>>>  tests
>>>>>>
>>>>>
>>>>>  should pass on both old and new versions, but the tests mint their own
>>>>>>
>>>>>>>
>>>>>>>  URLs
>>>>>>
>>>>>
>>>>>  for testing against -- each version is internally consistent, but if
>>>>>> an
>>>>>>
>>>>>>>
>>>>>>>  app
>>>>>>
>>>>>
>>>>>  is saving internal URLs and tries to dereference an old (file:///) url
>>>>>>
>>>>>>>
>>>>>>>  with
>>>>>>
>>>>>
>>>>>  the new plugin, it will probably not resolve.
>>>>>>
>>>>>>>
>>>>>>> Maybe there's a transition path here -- it might be possible to allow
>>>>>>> window.resolveLocalFileSystemURL to access files with the old URL,
>>>>>>> but
>>>>>>> never actually generate URLs.
>>>>>>>
>>>>>>> That would mean that
>>>>>>>
>>>>>>> (resolveLocalFileSystemURL(a)).toURL() !== a,
>>>>>>>
>>>>>>> but I don't think that we depend on that as an identity.
>>>>>>>
>>>>>>> Is there work which needs to be done on the other platforms (BB, WP8,
>>>>>>>
>>>>>>>  Win8, FFOS, etc) so that those platforms don't get left behind? If
>>>>>>>> so,
>>>>>>>> would it make sense to reach out to those platform owners and at
>>>>>>>> least
>>>>>>>>
>>>>>>>>  get
>>>>>>>
>>>>>>
>>>>>  Jira items created with some notes?
>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>> Yes. I'm going to create a ticket for this, and we can add
>>>>>>> platform-specific tasks to it.
>>>>>>>
>>>>>>> Other platforms shouldn't *have* to do anything, and some platforms
>>>>>>> *can't* do anything, because they cannot access anything other than
>>>>>>> file:/// urls anyway.
>>>>>>>
>>>>>>> However, if the other platforms want to get in on the goodness, and
>>>>>>>
>>>>>>>  start
>>>>>>
>>>>>
>>>>>  supporting their own URL schemes (I think the BB folks can use
>>>>>> something
>>>>>>
>>>>>>> like local:// for their filesystem access), then that'll be the place
>>>>>>> to
>>>>>>> keep everything organized.
>>>>>>>
>>>>>>> I'll post back once I have that issue set up.
>>>>>>>
>>>>>>> Ian
>>>>>>>
>>>>>>>
>>>>>>>   Sounds like you've built some very significant improvements.
>>>>>>> Thanks!
>>>>>>>
>>>>>>>>
>>>>>>>>   On Nov 15, 2013, at 9:29 AM, Ian Clelland <iclelland@chromium.org
>>>>>>>> >
>>>>>>>>
>>>>>>>>>
>>>>>>>>>  wrote:
>>>>>>>>
>>>>>>>
>>>>>
>>>>>>  After working with the File plugin for a week or so, I've gotten it
>>>>>>>>>
>>>>>>>>>  more or
>>>>>>>>
>>>>>>>>  less where I want it to be on iOS, and am working through Android.
>>>>>>>>>
>>>>>>>>>  Looking
>>>>>>>>
>>>>>>>>  at the end result, though, I expect that over 90% of the code has
>>>>>>>>> been
>>>>>>>>> touched in some way. (It's a huge diff.)
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>
>>>>
>>>
>>
>

Re: Transitioning to a better File API implementation

[Michael Gauthier <mi...@silverorange.com>]

Hmm.. The two directories have different defined roles on iOS and both 
are normal targets for applications to read/write files. Library is for 
cache data or app-specific resources. Documents is for saving/loading 
actual things created by users within apps.

See 
https://developer.apple.com/library/ios/documentation/FileManagement/Conceptual/FileSystemProgrammingGuide/FileSystemOverview/FileSystemOverview.html#//apple_ref/doc/uid/TP40010672-CH2-SW14

Supporting both should be a goal.

Cheers,
Mike


On 2013-12-10 14:39, Ian Clelland wrote:
> If you could do that before, it was probably a bug :) (You shouldn't be
> able to use getParent on the filesystem root, and the native code was
> supposed to be checking for the correct path prefix before allowing access
> to any files on the device)
>
> At the moment, it's not possible to do that -- the filesystems should be
> properly isolated, and only allow access to correctly formed urls, like
> filesystem://localhost/persistent/some-file-here.txt.
>
> What we *can* do easily, though, is allow a new URL scheme for library
> assets; something like filesystem://localhost/library/some-file-here.txt,
> and we can have a filesystem handler for those URLs. That'll work if your
> use case is just "I need access to files in /Library", rather than "I need
> to get to them via string manipulation".
>
> I've also had some discussions about making /Library the default place for
> the persistent filesystem, and leaving /Documents either just for legacy
> apps, or making *that* the target of the special URL scheme. That's a
> proposal for a different day, though. There are some pretty big
> backwards-compatibility issues there.
>
>
>
> On Tue, Dec 10, 2013 at 11:54 AM, Michael Gauthier <mi...@silverorange.com>wrote:
>
>> Ian,
>>
>> With the new URLs will it be possible to write to both /Documents and
>> /Library for iOS apps? In the old version, the filesystem root resolved as
>> /Documents but it was possible to get to /Library by navigating the the
>> parent dir.
>>
>> Cheers,
>> Mike
>>
>>
>>
>> On 2013-11-15 15:19, Ian Clelland wrote:
>>
>>> On Fri, Nov 15, 2013 at 1:40 PM, purplecabbage <pu...@gmail.com>
>>> wrote:
>>>
>>>   Considering the magnitude of the changes I would have expected that this
>>>> was just a new file plugin. The previous version was based on a spec, and
>>>> if we are deviating from it we should probably maintain both, and
>>>> possibly
>>>> even make a recommendation to the w3c.
>>>> I hope we at least do a major version update for this if APIs have
>>>> changed.
>>>>
>>>>
>>> Yes, definitely a major version bump, at least so that devs aren't caught
>>> by the URL-format-change.
>>>
>>> There aren't any changes to external APIs; I've been very careful to
>>> maintain conformance with the existing tests, except where those tests
>>> have
>>> been for undocumented implementation details. The only app-visible change
>>> is that URLs returned by the .toURL() method will look like
>>>
>>> filesystem://localhost/persistent/my/file.jpg
>>>
>>> rather than
>>>
>>> file:///var/mobile/Applications/<app id>/Documents/my/file.jpg
>>>
>>> We are still following the spec -- in fact, this brings us closer in line
>>> with the W3C File API spec, on these points:
>>>
>>> http://www.w3.org/TR/file-system-api/#widl-Entry-fullPath -- The fullPath
>>> of an entry should be the path from the root (of the HTML filesystem, not
>>> the underlying device file system).
>>>
>>> http://www.w3.org/TR/file-system-api/#widl-Entry-toURL-DOMString -- This
>>> is
>>> a note, rather than a requirement, but a filesystem url scheme is
>>> mentioned
>>> there.
>>>
>>> We're considering extending the spec in one way, which should be
>>> compatible
>>> with the spirit of the spec, and backwards-compatible with the previous
>>> API. That is the ability to declare new filesystem types, beyond
>>> "temporary" and "persistent", so that we can access things like device
>>> media and app bundle assets using the same APIs that we use for accessing
>>> user-defined files. If we're happy with the way that works, then we should
>>> definitely propose it for inclusion in the standard (not the specific
>>> filesystems, perhaps, but the mechanism for requesting and interacting
>>> with
>>> them)
>>>
>>> Ian
>>>
>>>
>>>
>>>> Sent from my iPhone
>>>>
>>>>   On Nov 15, 2013, at 8:36 AM, Ian Clelland <ic...@chromium.org>
>>>>>
>>>> wrote:
>>>>
>>>>>
>>>>> I've created CB-5403 as the über-ticket for this; various bits of it are
>>>>>
>>>> in
>>>>
>>>>> subtasks.
>>>>>
>>>>> Once I have the tests and the JS updated, then platform owners can take
>>>>> a
>>>>> look and see whether they have anything to do to support their own
>>>>>
>>>> schemes.
>>>>
>>>>>
>>>>> As general guidelines, I would say:
>>>>>
>>>>> * If you can intercept filesystem://* URLs in native code, and return
>>>>> arbitrary responses, then do it! It will let your platform support new
>>>>> filesystems in the future as we come up with them. Add a couple of
>>>>>
>>>> subtasks
>>>>
>>>>> for CB-5403 for your platform and go.
>>>>>
>>>>> * If you can't do that, but you *can* provide access to things like
>>>>> media
>>>>> through special urls, then try that! You should be able to create a
>>>>> FileSystem object that uses that URL as its root, and everything should
>>>>> work. Add a subtask for that, and see what you can do.
>>>>>
>>>>> * If you can't do that either, and you just want to stick with the
>>>>>
>>>> file:///
>>>>
>>>>> urls that we are currently using, then don't do anything; nothing should
>>>>> change for you.
>>>>>
>>>>>
>>>>>
>>>>> On Fri, Nov 15, 2013 at 10:36 AM, Ian Clelland <iclelland@chromium.org
>>>>> wrote:
>>>>>
>>>>>
>>>>>> On Fri, Nov 15, 2013 at 10:03 AM, Marcel Kinard <cmarcelk@gmail.com
>>>>>>
>>>>> wrote:
>>>>>
>>>>>>
>>>>>>   Ian, will there be changes that app developers will need to do? If so,
>>>>>>> those should be clearly documented in a migration guide. If not, it
>>>>>>>
>>>>>> sounds
>>>>
>>>>> like the [improved] tests should pass on both the old and new versions,
>>>>>>> which would be sweet.
>>>>>>>
>>>>>>
>>>>>> The filesystem URLs themselves will change as a result of this. The
>>>>>>
>>>>> tests
>>>>
>>>>> should pass on both old and new versions, but the tests mint their own
>>>>>>
>>>>> URLs
>>>>
>>>>> for testing against -- each version is internally consistent, but if an
>>>>>>
>>>>> app
>>>>
>>>>> is saving internal URLs and tries to dereference an old (file:///) url
>>>>>>
>>>>> with
>>>>
>>>>> the new plugin, it will probably not resolve.
>>>>>>
>>>>>> Maybe there's a transition path here -- it might be possible to allow
>>>>>> window.resolveLocalFileSystemURL to access files with the old URL, but
>>>>>> never actually generate URLs.
>>>>>>
>>>>>> That would mean that
>>>>>>
>>>>>> (resolveLocalFileSystemURL(a)).toURL() !== a,
>>>>>>
>>>>>> but I don't think that we depend on that as an identity.
>>>>>>
>>>>>> Is there work which needs to be done on the other platforms (BB, WP8,
>>>>>>
>>>>>>> Win8, FFOS, etc) so that those platforms don't get left behind? If so,
>>>>>>> would it make sense to reach out to those platform owners and at least
>>>>>>>
>>>>>> get
>>>>
>>>>> Jira items created with some notes?
>>>>>>>
>>>>>>
>>>>>> Yes. I'm going to create a ticket for this, and we can add
>>>>>> platform-specific tasks to it.
>>>>>>
>>>>>> Other platforms shouldn't *have* to do anything, and some platforms
>>>>>> *can't* do anything, because they cannot access anything other than
>>>>>> file:/// urls anyway.
>>>>>>
>>>>>> However, if the other platforms want to get in on the goodness, and
>>>>>>
>>>>> start
>>>>
>>>>> supporting their own URL schemes (I think the BB folks can use something
>>>>>> like local:// for their filesystem access), then that'll be the place
>>>>>> to
>>>>>> keep everything organized.
>>>>>>
>>>>>> I'll post back once I have that issue set up.
>>>>>>
>>>>>> Ian
>>>>>>
>>>>>>
>>>>>>   Sounds like you've built some very significant improvements. Thanks!
>>>>>>>
>>>>>>>   On Nov 15, 2013, at 9:29 AM, Ian Clelland <ic...@chromium.org>
>>>>>>>>
>>>>>>> wrote:
>>>>
>>>>>
>>>>>>>> After working with the File plugin for a week or so, I've gotten it
>>>>>>>>
>>>>>>> more or
>>>>>>>
>>>>>>>> less where I want it to be on iOS, and am working through Android.
>>>>>>>>
>>>>>>> Looking
>>>>>>>
>>>>>>>> at the end result, though, I expect that over 90% of the code has
>>>>>>>> been
>>>>>>>> touched in some way. (It's a huge diff.)
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>
>>>
>>
>

Re: Transitioning to a better File API implementation

[Ian Clelland <ic...@chromium.org>]

If you could do that before, it was probably a bug :) (You shouldn't be
able to use getParent on the filesystem root, and the native code was
supposed to be checking for the correct path prefix before allowing access
to any files on the device)

At the moment, it's not possible to do that -- the filesystems should be
properly isolated, and only allow access to correctly formed urls, like
filesystem://localhost/persistent/some-file-here.txt.

What we *can* do easily, though, is allow a new URL scheme for library
assets; something like filesystem://localhost/library/some-file-here.txt,
and we can have a filesystem handler for those URLs. That'll work if your
use case is just "I need access to files in /Library", rather than "I need
to get to them via string manipulation".

I've also had some discussions about making /Library the default place for
the persistent filesystem, and leaving /Documents either just for legacy
apps, or making *that* the target of the special URL scheme. That's a
proposal for a different day, though. There are some pretty big
backwards-compatibility issues there.



On Tue, Dec 10, 2013 at 11:54 AM, Michael Gauthier <mi...@silverorange.com>wrote:

> Ian,
>
> With the new URLs will it be possible to write to both /Documents and
> /Library for iOS apps? In the old version, the filesystem root resolved as
> /Documents but it was possible to get to /Library by navigating the the
> parent dir.
>
> Cheers,
> Mike
>
>
>
> On 2013-11-15 15:19, Ian Clelland wrote:
>
>> On Fri, Nov 15, 2013 at 1:40 PM, purplecabbage <pu...@gmail.com>
>> wrote:
>>
>>  Considering the magnitude of the changes I would have expected that this
>>> was just a new file plugin. The previous version was based on a spec, and
>>> if we are deviating from it we should probably maintain both, and
>>> possibly
>>> even make a recommendation to the w3c.
>>> I hope we at least do a major version update for this if APIs have
>>> changed.
>>>
>>>
>> Yes, definitely a major version bump, at least so that devs aren't caught
>> by the URL-format-change.
>>
>> There aren't any changes to external APIs; I've been very careful to
>> maintain conformance with the existing tests, except where those tests
>> have
>> been for undocumented implementation details. The only app-visible change
>> is that URLs returned by the .toURL() method will look like
>>
>> filesystem://localhost/persistent/my/file.jpg
>>
>> rather than
>>
>> file:///var/mobile/Applications/<app id>/Documents/my/file.jpg
>>
>> We are still following the spec -- in fact, this brings us closer in line
>> with the W3C File API spec, on these points:
>>
>> http://www.w3.org/TR/file-system-api/#widl-Entry-fullPath -- The fullPath
>> of an entry should be the path from the root (of the HTML filesystem, not
>> the underlying device file system).
>>
>> http://www.w3.org/TR/file-system-api/#widl-Entry-toURL-DOMString -- This
>> is
>> a note, rather than a requirement, but a filesystem url scheme is
>> mentioned
>> there.
>>
>> We're considering extending the spec in one way, which should be
>> compatible
>> with the spirit of the spec, and backwards-compatible with the previous
>> API. That is the ability to declare new filesystem types, beyond
>> "temporary" and "persistent", so that we can access things like device
>> media and app bundle assets using the same APIs that we use for accessing
>> user-defined files. If we're happy with the way that works, then we should
>> definitely propose it for inclusion in the standard (not the specific
>> filesystems, perhaps, but the mechanism for requesting and interacting
>> with
>> them)
>>
>> Ian
>>
>>
>>
>>> Sent from my iPhone
>>>
>>>  On Nov 15, 2013, at 8:36 AM, Ian Clelland <ic...@chromium.org>
>>>>
>>> wrote:
>>>
>>>>
>>>> I've created CB-5403 as the über-ticket for this; various bits of it are
>>>>
>>> in
>>>
>>>> subtasks.
>>>>
>>>> Once I have the tests and the JS updated, then platform owners can take
>>>> a
>>>> look and see whether they have anything to do to support their own
>>>>
>>> schemes.
>>>
>>>>
>>>> As general guidelines, I would say:
>>>>
>>>> * If you can intercept filesystem://* URLs in native code, and return
>>>> arbitrary responses, then do it! It will let your platform support new
>>>> filesystems in the future as we come up with them. Add a couple of
>>>>
>>> subtasks
>>>
>>>> for CB-5403 for your platform and go.
>>>>
>>>> * If you can't do that, but you *can* provide access to things like
>>>> media
>>>> through special urls, then try that! You should be able to create a
>>>> FileSystem object that uses that URL as its root, and everything should
>>>> work. Add a subtask for that, and see what you can do.
>>>>
>>>> * If you can't do that either, and you just want to stick with the
>>>>
>>> file:///
>>>
>>>> urls that we are currently using, then don't do anything; nothing should
>>>> change for you.
>>>>
>>>>
>>>>
>>>> On Fri, Nov 15, 2013 at 10:36 AM, Ian Clelland <iclelland@chromium.org
>>>> wrote:
>>>>
>>>>
>>>>> On Fri, Nov 15, 2013 at 10:03 AM, Marcel Kinard <cmarcelk@gmail.com
>>>>>
>>>> wrote:
>>>>
>>>>>
>>>>>  Ian, will there be changes that app developers will need to do? If so,
>>>>>> those should be clearly documented in a migration guide. If not, it
>>>>>>
>>>>> sounds
>>>
>>>> like the [improved] tests should pass on both the old and new versions,
>>>>>> which would be sweet.
>>>>>>
>>>>>
>>>>> The filesystem URLs themselves will change as a result of this. The
>>>>>
>>>> tests
>>>
>>>> should pass on both old and new versions, but the tests mint their own
>>>>>
>>>> URLs
>>>
>>>> for testing against -- each version is internally consistent, but if an
>>>>>
>>>> app
>>>
>>>> is saving internal URLs and tries to dereference an old (file:///) url
>>>>>
>>>> with
>>>
>>>> the new plugin, it will probably not resolve.
>>>>>
>>>>> Maybe there's a transition path here -- it might be possible to allow
>>>>> window.resolveLocalFileSystemURL to access files with the old URL, but
>>>>> never actually generate URLs.
>>>>>
>>>>> That would mean that
>>>>>
>>>>> (resolveLocalFileSystemURL(a)).toURL() !== a,
>>>>>
>>>>> but I don't think that we depend on that as an identity.
>>>>>
>>>>> Is there work which needs to be done on the other platforms (BB, WP8,
>>>>>
>>>>>> Win8, FFOS, etc) so that those platforms don't get left behind? If so,
>>>>>> would it make sense to reach out to those platform owners and at least
>>>>>>
>>>>> get
>>>
>>>> Jira items created with some notes?
>>>>>>
>>>>>
>>>>> Yes. I'm going to create a ticket for this, and we can add
>>>>> platform-specific tasks to it.
>>>>>
>>>>> Other platforms shouldn't *have* to do anything, and some platforms
>>>>> *can't* do anything, because they cannot access anything other than
>>>>> file:/// urls anyway.
>>>>>
>>>>> However, if the other platforms want to get in on the goodness, and
>>>>>
>>>> start
>>>
>>>> supporting their own URL schemes (I think the BB folks can use something
>>>>> like local:// for their filesystem access), then that'll be the place
>>>>> to
>>>>> keep everything organized.
>>>>>
>>>>> I'll post back once I have that issue set up.
>>>>>
>>>>> Ian
>>>>>
>>>>>
>>>>>  Sounds like you've built some very significant improvements. Thanks!
>>>>>>
>>>>>>  On Nov 15, 2013, at 9:29 AM, Ian Clelland <ic...@chromium.org>
>>>>>>>
>>>>>> wrote:
>>>
>>>>
>>>>>>> After working with the File plugin for a week or so, I've gotten it
>>>>>>>
>>>>>> more or
>>>>>>
>>>>>>> less where I want it to be on iOS, and am working through Android.
>>>>>>>
>>>>>> Looking
>>>>>>
>>>>>>> at the end result, though, I expect that over 90% of the code has
>>>>>>> been
>>>>>>> touched in some way. (It's a huge diff.)
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>
>>
>