You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cordova.apache.org by Ian Clelland <ic...@chromium.org> on 2013/11/15 15:29:44 UTC
Transitioning to a better File API implementation
After working with the File plugin for a week or so, I've gotten it more or
less where I want it to be on iOS, and am working through Android. Looking
at the end result, though, I expect that over 90% of the code has been
touched in some way. (It's a huge diff.) I'd rather not simply dump this in
to the repo as a monolithic change, so I'm planing on splitting it up into
smaller, sensible steps.
I think we can transition from one to the other by doing this:
1. Harden the tests
Many of the tests in the test suite are fragile, and depend on
implementation-dependent things like whether root directory paths end with
a "/" or not, or whether a particular URL scheme is valid. I'm first going
to fix those tests which don't pass under both the old and new systems.
2. Update the JS code to allow the FileSystem object to format paths used
in exec() calls.
Since CB-5129, all Entry objects should have a filesystem object attached.
By allowing this FS object to determine what information goes across the
bridge, we are 90% of the way to using URLs internally, rather than (real)
filesystem paths, which was one of the goals of the refactoring.
3. Switch native code to use filesystem://localhost/* URLs internally, and
change the FileSystem JS object to use those URLs for the bridge.
(This gets us the rest of the way.) We can do this one platform at a time,
as well, by providing platform-specific JS code for FileSystem.
4. Add URL handlers for filesystem:// URLs in native code
This last step lets us close CB-5007, by intercepting network requests for
filesystem://* and returning the correct data. At this point, we can
actually use the results returned by FileEntry.toURL() as the source for
images, script tags, etc. Like step #3, this is opt-in, for platforms which
can support it. Platforms that don't can just continue to have their
FileSystem objects product URLs which *can* be accessed from the webview.
One final step, which can be done as part of #3, or afterwards, is to
modularize the code so that additional filesystems can be added. For iOS,
this means Asset-Library URLs, which no longer have to be special-cased in
every API call. Future filesystems can provide synced storage, or media
gallery access, or encrypted storage, or anything else we dream up.
Ian
Re: Transitioning to a better File API implementation
Posted by Andrew Grieve <ag...@chromium.org>.
Plan sounds well thought-out!
On Fri, Nov 15, 2013 at 10:03 AM, Marcel Kinard <cm...@gmail.com> wrote:
> Ian, will there be changes that app developers will need to do? If so,
> those should be clearly documented in a migration guide. If not, it sounds
> like the [improved] tests should pass on both the old and new versions,
> which would be sweet.
>
> Is there work which needs to be done on the other platforms (BB, WP8,
> Win8, FFOS, etc) so that those platforms don't get left behind? If so,
> would it make sense to reach out to those platform owners and at least get
> Jira items created with some notes?
>
> Sounds like you've built some very significant improvements. Thanks!
>
> On Nov 15, 2013, at 9:29 AM, Ian Clelland <ic...@chromium.org> wrote:
>
> > After working with the File plugin for a week or so, I've gotten it more
> or
> > less where I want it to be on iOS, and am working through Android.
> Looking
> > at the end result, though, I expect that over 90% of the code has been
> > touched in some way. (It's a huge diff.)
>
Re: Transitioning to a better File API implementation
Posted by Shazron <sh...@gmail.com>.
Reminds me of the iOS LocalStorage (CDVLocalStorage plugin) bs that needed
to be handled - we definitely need a migration path (automatic hopefully)
if not users are in for a world of hurt...
On Wed, Dec 11, 2013 at 11:33 AM, Tommy Williams <to...@devgeeks.org> wrote:
> +1
> On 12/12/2013 6:26 am, "Ian Clelland" <ic...@chromium.org> wrote:
>
> > Yeah, it would definitely require some kind of migration support. Not
> > suggesting that this is something that we ever actually do, but we could
> > give developers a bit of code that automatically looks in both places,
> and
> > moves files to the new location on open. Or we do it under a flag that is
> > off for existing apps (off-on-upgrade) and only enabled-by-default for
> new
> > apps.
> >
> > I agree with you: this could cause worlds of pain, with developers, and
> > worse with users. We shouldn't take any steps in that direction without a
> > lot of very careful thought. For now, /Documents, sub-optimal though it
> may
> > be, is where Cordova puts persistent files.
> >
> > Ian
> >
> > On Wed, Dec 11, 2013 at 2:18 PM, Tommy Williams <to...@devgeeks.org>
> > wrote:
> >
> > > Changing where PERSISTENT is located sounds like a very very bad idea.
> > >
> > > I know that would cause me personally a lot of grief with existing user
> > > data.
> > > On 11/12/2013 8:58 am, "Ian Clelland" <ic...@chromium.org> wrote:
> > >
> > > > On Tue, Dec 10, 2013 at 4:37 PM, Michael Gauthier <
> > mike@silverorange.com
> > > > >wrote:
> > > >
> > > > > On 10/12/13 05:02 PM, Ian Clelland wrote:
> > > > >
> > > > >> Yep.
> > > > >>
> > > > >> I think that Library is the more natural place for the HTML
> > persistent
> > > > >> filesystem, since it's used by an app for whatever persistent
> > storage
> > > it
> > > > >> needs, without exposing all of it's implementation details to the
> > > user.
> > > > >> (There's lots of room for debate on this, I'm sure)
> > > > >>
> > > > >> The trouble is that we've historically used /Documents for
> > persistent
> > > > >> storage, and changing that will break apps.
> > > > >>
> > > > >> I'm fine with a BC break, but I don't have to maintain any legacy
> > > > > applications. /Library does make more sense as the default for
> > > > PERSISTENT.
> > > > > The big problem with BC is for installed apps with existing data on
> > the
> > > > > filesystem, right?
> > > >
> > > >
> > > > Exactly. I'd hate for developers to update their plugins, and push a
> > new
> > > > version of their app that seems fine -- everything works great when
> you
> > > > start from scratch -- but users with existing data find themselves
> > > > completely locked out of it.
> > > >
> > > >
> > > > >
> > > > > One idea is to allow something like requestFilesystem(DOCUMENT),
> in
> > > > >> addition to PERSISTENT and TEMPORARY. Another suggestion has been
> to
> > > add
> > > > >> extra arguments -- hints -- such as "{sync: true}", or maybe in
> this
> > > > case
> > > > >> "{purpose: documents}" to specify the attributes of the filesystem
> > > that
> > > > is
> > > > >> returned.
> > > > >>
> > > > > >
> > > > > {sync: true} is a bit tricky because /Library/Cache is not synced
> but
> > > > > /Library/Application Data is synced. Having a DOCUMENT type would
> > match
> > > > > /tmp and /Library as the top-level dirs mapping to file-system
> > > constants.
> > > > >
> > > > > All my feedback is only from the iOS perspective though. Not sure
> if
> > > > these
> > > > > ideas make other platforms more or less complicated.
> > > > >
> > > > > Cheers,
> > > > > Mike
> > > > >
> > > > >
> > > > >>
> > > > >>
> > > > >> On Tue, Dec 10, 2013 at 2:39 PM, Michael Gauthier <
> > > > mike@silverorange.com
> > > > >> >wrote:
> > > > >>
> > > > >> Hmm.. The two directories have different defined roles on iOS and
> > > both
> > > > >>> are
> > > > >>> normal targets for applications to read/write files. Library is
> for
> > > > cache
> > > > >>> data or app-specific resources. Documents is for saving/loading
> > > actual
> > > > >>> things created by users within apps.
> > > > >>>
> > > > >>> See https://developer.apple.com/library/ios/documentation/
> > > > >>>
> > > >
> > FileManagement/Conceptual/FileSystemProgrammingGuide/FileSystemOverview/
> > > > >>> FileSystemOverview.html#//apple_ref/doc/uid/TP40010672-CH2-SW14
> > > > >>>
> > > > >>> Supporting both should be a goal.
> > > > >>>
> > > > >>> Cheers,
> > > > >>> Mike
> > > > >>>
> > > > >>>
> > > > >>>
> > > > >>> On 2013-12-10 14:39, Ian Clelland wrote:
> > > > >>>
> > > > >>> If you could do that before, it was probably a bug :) (You
> > shouldn't
> > > > be
> > > > >>>> able to use getParent on the filesystem root, and the native
> code
> > > was
> > > > >>>> supposed to be checking for the correct path prefix before
> > allowing
> > > > >>>> access
> > > > >>>> to any files on the device)
> > > > >>>>
> > > > >>>> At the moment, it's not possible to do that -- the filesystems
> > > should
> > > > be
> > > > >>>> properly isolated, and only allow access to correctly formed
> urls,
> > > > like
> > > > >>>> filesystem://localhost/persistent/some-file-here.txt.
> > > > >>>>
> > > > >>>> What we *can* do easily, though, is allow a new URL scheme for
> > > library
> > > > >>>> assets; something like filesystem://localhost/
> > > > >>>> library/some-file-here.txt,
> > > > >>>> and we can have a filesystem handler for those URLs. That'll
> work
> > if
> > > > >>>> your
> > > > >>>> use case is just "I need access to files in /Library", rather
> than
> > > "I
> > > > >>>> need
> > > > >>>> to get to them via string manipulation".
> > > > >>>>
> > > > >>>> I've also had some discussions about making /Library the default
> > > place
> > > > >>>> for
> > > > >>>> the persistent filesystem, and leaving /Documents either just
> for
> > > > legacy
> > > > >>>> apps, or making *that* the target of the special URL scheme.
> > That's
> > > a
> > > > >>>> proposal for a different day, though. There are some pretty big
> > > > >>>> backwards-compatibility issues there.
> > > > >>>>
> > > > >>>>
> > > > >>>>
> > > > >>>> On Tue, Dec 10, 2013 at 11:54 AM, Michael Gauthier <
> > > > >>>> mike@silverorange.com
> > > > >>>>
> > > > >>>>> wrote:
> > > > >>>>>
> > > > >>>>
> > > > >>>> Ian,
> > > > >>>>
> > > > >>>>>
> > > > >>>>> With the new URLs will it be possible to write to both
> /Documents
> > > and
> > > > >>>>> /Library for iOS apps? In the old version, the filesystem root
> > > > resolved
> > > > >>>>> as
> > > > >>>>> /Documents but it was possible to get to /Library by navigating
> > the
> > > > the
> > > > >>>>> parent dir.
> > > > >>>>>
> > > > >>>>> Cheers,
> > > > >>>>> Mike
> > > > >>>>>
> > > > >>>>>
> > > > >>>>>
> > > > >>>>> On 2013-11-15 15:19, Ian Clelland wrote:
> > > > >>>>>
> > > > >>>>> On Fri, Nov 15, 2013 at 1:40 PM, purplecabbage <
> > > > >>>>> purplecabbage@gmail.com
> > > > >>>>>
> > > > >>>>>>
> > > > >>>>>>> wrote:
> > > > >>>>>>
> > > > >>>>>> Considering the magnitude of the changes I would have
> > expected
> > > > that
> > > > >>>>>> this
> > > > >>>>>>
> > > > >>>>>> was just a new file plugin. The previous version was based
> on a
> > > > spec,
> > > > >>>>>>> and
> > > > >>>>>>> if we are deviating from it we should probably maintain both,
> > and
> > > > >>>>>>> possibly
> > > > >>>>>>> even make a recommendation to the w3c.
> > > > >>>>>>> I hope we at least do a major version update for this if APIs
> > > have
> > > > >>>>>>> changed.
> > > > >>>>>>>
> > > > >>>>>>>
> > > > >>>>>>> Yes, definitely a major version bump, at least so that devs
> > > > aren't
> > > > >>>>>>>
> > > > >>>>>> caught
> > > > >>>>>> by the URL-format-change.
> > > > >>>>>>
> > > > >>>>>> There aren't any changes to external APIs; I've been very
> > careful
> > > to
> > > > >>>>>> maintain conformance with the existing tests, except where
> those
> > > > tests
> > > > >>>>>> have
> > > > >>>>>> been for undocumented implementation details. The only
> > app-visible
> > > > >>>>>> change
> > > > >>>>>> is that URLs returned by the .toURL() method will look like
> > > > >>>>>>
> > > > >>>>>> filesystem://localhost/persistent/my/file.jpg
> > > > >>>>>>
> > > > >>>>>> rather than
> > > > >>>>>>
> > > > >>>>>> file:///var/mobile/Applications/<app id>/Documents/my/file.jpg
> > > > >>>>>>
> > > > >>>>>> We are still following the spec -- in fact, this brings us
> > closer
> > > in
> > > > >>>>>> line
> > > > >>>>>> with the W3C File API spec, on these points:
> > > > >>>>>>
> > > > >>>>>> http://www.w3.org/TR/file-system-api/#widl-Entry-fullPath --
> > The
> > > > >>>>>> fullPath
> > > > >>>>>> of an entry should be the path from the root (of the HTML
> > > > filesystem,
> > > > >>>>>> not
> > > > >>>>>> the underlying device file system).
> > > > >>>>>>
> > > > >>>>>>
> > http://www.w3.org/TR/file-system-api/#widl-Entry-toURL-DOMString--
> > > > >>>>>> This
> > > > >>>>>> is
> > > > >>>>>> a note, rather than a requirement, but a filesystem url scheme
> > is
> > > > >>>>>> mentioned
> > > > >>>>>> there.
> > > > >>>>>>
> > > > >>>>>> We're considering extending the spec in one way, which should
> be
> > > > >>>>>> compatible
> > > > >>>>>> with the spirit of the spec, and backwards-compatible with the
> > > > >>>>>> previous
> > > > >>>>>> API. That is the ability to declare new filesystem types,
> beyond
> > > > >>>>>> "temporary" and "persistent", so that we can access things
> like
> > > > device
> > > > >>>>>> media and app bundle assets using the same APIs that we use
> for
> > > > >>>>>> accessing
> > > > >>>>>> user-defined files. If we're happy with the way that works,
> then
> > > we
> > > > >>>>>> should
> > > > >>>>>> definitely propose it for inclusion in the standard (not the
> > > > specific
> > > > >>>>>> filesystems, perhaps, but the mechanism for requesting and
> > > > interacting
> > > > >>>>>> with
> > > > >>>>>> them)
> > > > >>>>>>
> > > > >>>>>> Ian
> > > > >>>>>>
> > > > >>>>>>
> > > > >>>>>>
> > > > >>>>>> Sent from my iPhone
> > > > >>>>>>
> > > > >>>>>>>
> > > > >>>>>>> On Nov 15, 2013, at 8:36 AM, Ian Clelland <
> > > > iclelland@chromium.org
> > > > >>>>>>> >
> > > > >>>>>>>
> > > > >>>>>>>
> > > > >>>>>>>> wrote:
> > > > >>>>>>>>
> > > > >>>>>>>
> > > > >>>>>>>
> > > > >>>>>>> I've created CB-5403 as the über-ticket for this; various
> bits
> > > of
> > > > it
> > > > >>>>>>>> are
> > > > >>>>>>>>
> > > > >>>>>>>> in
> > > > >>>>>>>>
> > > > >>>>>>>
> > > > >>>>>>> subtasks.
> > > > >>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>> Once I have the tests and the JS updated, then platform
> owners
> > > can
> > > > >>>>>>>> take
> > > > >>>>>>>> a
> > > > >>>>>>>> look and see whether they have anything to do to support
> their
> > > own
> > > > >>>>>>>>
> > > > >>>>>>>> schemes.
> > > > >>>>>>>>
> > > > >>>>>>>
> > > > >>>>>>>
> > > > >>>>>>> As general guidelines, I would say:
> > > > >>>>>>>>
> > > > >>>>>>>> * If you can intercept filesystem://* URLs in native code,
> and
> > > > >>>>>>>> return
> > > > >>>>>>>> arbitrary responses, then do it! It will let your platform
> > > support
> > > > >>>>>>>> new
> > > > >>>>>>>> filesystems in the future as we come up with them. Add a
> > couple
> > > of
> > > > >>>>>>>>
> > > > >>>>>>>> subtasks
> > > > >>>>>>>>
> > > > >>>>>>>
> > > > >>>>>>> for CB-5403 for your platform and go.
> > > > >>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>> * If you can't do that, but you *can* provide access to
> things
> > > > like
> > > > >>>>>>>> media
> > > > >>>>>>>> through special urls, then try that! You should be able to
> > > create
> > > > a
> > > > >>>>>>>> FileSystem object that uses that URL as its root, and
> > everything
> > > > >>>>>>>> should
> > > > >>>>>>>> work. Add a subtask for that, and see what you can do.
> > > > >>>>>>>>
> > > > >>>>>>>> * If you can't do that either, and you just want to stick
> with
> > > the
> > > > >>>>>>>>
> > > > >>>>>>>> file:///
> > > > >>>>>>>>
> > > > >>>>>>>
> > > > >>>>>>> urls that we are currently using, then don't do anything;
> > > nothing
> > > > >>>>>>>
> > > > >>>>>>>> should
> > > > >>>>>>>> change for you.
> > > > >>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>> On Fri, Nov 15, 2013 at 10:36 AM, Ian Clelland <
> > > > >>>>>>>> iclelland@chromium.org
> > > > >>>>>>>> wrote:
> > > > >>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>> On Fri, Nov 15, 2013 at 10:03 AM, Marcel Kinard <
> > > > >>>>>>>> cmarcelk@gmail.com
> > > > >>>>>>>>
> > > > >>>>>>>>>
> > > > >>>>>>>>> wrote:
> > > > >>>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>> Ian, will there be changes that app developers will need
> > to
> > > > do?
> > > > >>>>>>>>> If
> > > > >>>>>>>>> so,
> > > > >>>>>>>>>
> > > > >>>>>>>>> those should be clearly documented in a migration guide.
> If
> > > not,
> > > > >>>>>>>>>> it
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> sounds
> > > > >>>>>>>>>>
> > > > >>>>>>>>>
> > > > >>>>>>>>>
> > > > >>>>>>>> like the [improved] tests should pass on both the old and
> > new
> > > > >>>>>>>
> > > > >>>>>>>> versions,
> > > > >>>>>>>>
> > > > >>>>>>>> which would be sweet.
> > > > >>>>>>>>>
> > > > >>>>>>>>>>
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> The filesystem URLs themselves will change as a result of
> > > this.
> > > > >>>>>>>>> The
> > > > >>>>>>>>>
> > > > >>>>>>>>> tests
> > > > >>>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>> should pass on both old and new versions, but the tests
> mint
> > > > their
> > > > >>>>>>> own
> > > > >>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>>> URLs
> > > > >>>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>> for testing against -- each version is internally
> consistent,
> > > but
> > > > >>>>>>> if
> > > > >>>>>>>
> > > > >>>>>>>> an
> > > > >>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>>> app
> > > > >>>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>> is saving internal URLs and tries to dereference an old
> > > > (file:///)
> > > > >>>>>>> url
> > > > >>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>>> with
> > > > >>>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>> the new plugin, it will probably not resolve.
> > > > >>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>>> Maybe there's a transition path here -- it might be
> possible
> > to
> > > > >>>>>>>>> allow
> > > > >>>>>>>>> window.resolveLocalFileSystemURL to access files with the
> old
> > > > URL,
> > > > >>>>>>>>> but
> > > > >>>>>>>>> never actually generate URLs.
> > > > >>>>>>>>>
> > > > >>>>>>>>> That would mean that
> > > > >>>>>>>>>
> > > > >>>>>>>>> (resolveLocalFileSystemURL(a)).toURL() !== a,
> > > > >>>>>>>>>
> > > > >>>>>>>>> but I don't think that we depend on that as an identity.
> > > > >>>>>>>>>
> > > > >>>>>>>>> Is there work which needs to be done on the other platforms
> > > (BB,
> > > > >>>>>>>>> WP8,
> > > > >>>>>>>>>
> > > > >>>>>>>>> Win8, FFOS, etc) so that those platforms don't get left
> > > behind?
> > > > >>>>>>>>> If
> > > > >>>>>>>>>
> > > > >>>>>>>>>> so,
> > > > >>>>>>>>>> would it make sense to reach out to those platform owners
> > and
> > > at
> > > > >>>>>>>>>> least
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> get
> > > > >>>>>>>>>>
> > > > >>>>>>>>>
> > > > >>>>>>>>>
> > > > >>>>>>>> Jira items created with some notes?
> > > > >>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>>>
> > > > >>>>>>>>>> Yes. I'm going to create a ticket for this, and we can
> add
> > > > >>>>>>>>> platform-specific tasks to it.
> > > > >>>>>>>>>
> > > > >>>>>>>>> Other platforms shouldn't *have* to do anything, and some
> > > > platforms
> > > > >>>>>>>>> *can't* do anything, because they cannot access anything
> > other
> > > > than
> > > > >>>>>>>>> file:/// urls anyway.
> > > > >>>>>>>>>
> > > > >>>>>>>>> However, if the other platforms want to get in on the
> > goodness,
> > > > and
> > > > >>>>>>>>>
> > > > >>>>>>>>> start
> > > > >>>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>> supporting their own URL schemes (I think the BB folks can
> > use
> > > > >>>>>>>
> > > > >>>>>>>> something
> > > > >>>>>>>>
> > > > >>>>>>>> like local:// for their filesystem access), then that'll be
> > the
> > > > >>>>>>>>> place
> > > > >>>>>>>>> to
> > > > >>>>>>>>> keep everything organized.
> > > > >>>>>>>>>
> > > > >>>>>>>>> I'll post back once I have that issue set up.
> > > > >>>>>>>>>
> > > > >>>>>>>>> Ian
> > > > >>>>>>>>>
> > > > >>>>>>>>>
> > > > >>>>>>>>> Sounds like you've built some very significant
> > improvements.
> > > > >>>>>>>>> Thanks!
> > > > >>>>>>>>>
> > > > >>>>>>>>>
> > > > >>>>>>>>>> On Nov 15, 2013, at 9:29 AM, Ian Clelland <
> > > > >>>>>>>>>> iclelland@chromium.org
> > > > >>>>>>>>>>
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>
> > > > >>>>>>>>>>> wrote:
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>
> > > > >>>>>>>>>>
> > > > >>>>>>>>>
> > > > >>>>>>> After working with the File plugin for a week or so, I've
> > > gotten
> > > > >>>>>>>> it
> > > > >>>>>>>>
> > > > >>>>>>>>>
> > > > >>>>>>>>>>> more or
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> less where I want it to be on iOS, and am working
> through
> > > > >>>>>>>>>> Android.
> > > > >>>>>>>>>>
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>> Looking
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> at the end result, though, I expect that over 90% of the
> > > code
> > > > >>>>>>>>>> has
> > > > >>>>>>>>>>
> > > > >>>>>>>>>>> been
> > > > >>>>>>>>>>> touched in some way. (It's a huge diff.)
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>
> > > > >>>>>>>>>
> > > > >>>>>>>>>
> > > > >>>>>>>
> > > > >>>>>>
> > > > >>>>>
> > > > >>>>
> > > > >>>
> > > > >>
> > > > >
> > > >
> > >
> >
>
Re: Transitioning to a better File API implementation
Posted by Tommy Williams <to...@devgeeks.org>.
+1
On 12/12/2013 6:26 am, "Ian Clelland" <ic...@chromium.org> wrote:
> Yeah, it would definitely require some kind of migration support. Not
> suggesting that this is something that we ever actually do, but we could
> give developers a bit of code that automatically looks in both places, and
> moves files to the new location on open. Or we do it under a flag that is
> off for existing apps (off-on-upgrade) and only enabled-by-default for new
> apps.
>
> I agree with you: this could cause worlds of pain, with developers, and
> worse with users. We shouldn't take any steps in that direction without a
> lot of very careful thought. For now, /Documents, sub-optimal though it may
> be, is where Cordova puts persistent files.
>
> Ian
>
> On Wed, Dec 11, 2013 at 2:18 PM, Tommy Williams <to...@devgeeks.org>
> wrote:
>
> > Changing where PERSISTENT is located sounds like a very very bad idea.
> >
> > I know that would cause me personally a lot of grief with existing user
> > data.
> > On 11/12/2013 8:58 am, "Ian Clelland" <ic...@chromium.org> wrote:
> >
> > > On Tue, Dec 10, 2013 at 4:37 PM, Michael Gauthier <
> mike@silverorange.com
> > > >wrote:
> > >
> > > > On 10/12/13 05:02 PM, Ian Clelland wrote:
> > > >
> > > >> Yep.
> > > >>
> > > >> I think that Library is the more natural place for the HTML
> persistent
> > > >> filesystem, since it's used by an app for whatever persistent
> storage
> > it
> > > >> needs, without exposing all of it's implementation details to the
> > user.
> > > >> (There's lots of room for debate on this, I'm sure)
> > > >>
> > > >> The trouble is that we've historically used /Documents for
> persistent
> > > >> storage, and changing that will break apps.
> > > >>
> > > >> I'm fine with a BC break, but I don't have to maintain any legacy
> > > > applications. /Library does make more sense as the default for
> > > PERSISTENT.
> > > > The big problem with BC is for installed apps with existing data on
> the
> > > > filesystem, right?
> > >
> > >
> > > Exactly. I'd hate for developers to update their plugins, and push a
> new
> > > version of their app that seems fine -- everything works great when you
> > > start from scratch -- but users with existing data find themselves
> > > completely locked out of it.
> > >
> > >
> > > >
> > > > One idea is to allow something like requestFilesystem(DOCUMENT), in
> > > >> addition to PERSISTENT and TEMPORARY. Another suggestion has been to
> > add
> > > >> extra arguments -- hints -- such as "{sync: true}", or maybe in this
> > > case
> > > >> "{purpose: documents}" to specify the attributes of the filesystem
> > that
> > > is
> > > >> returned.
> > > >>
> > > > >
> > > > {sync: true} is a bit tricky because /Library/Cache is not synced but
> > > > /Library/Application Data is synced. Having a DOCUMENT type would
> match
> > > > /tmp and /Library as the top-level dirs mapping to file-system
> > constants.
> > > >
> > > > All my feedback is only from the iOS perspective though. Not sure if
> > > these
> > > > ideas make other platforms more or less complicated.
> > > >
> > > > Cheers,
> > > > Mike
> > > >
> > > >
> > > >>
> > > >>
> > > >> On Tue, Dec 10, 2013 at 2:39 PM, Michael Gauthier <
> > > mike@silverorange.com
> > > >> >wrote:
> > > >>
> > > >> Hmm.. The two directories have different defined roles on iOS and
> > both
> > > >>> are
> > > >>> normal targets for applications to read/write files. Library is for
> > > cache
> > > >>> data or app-specific resources. Documents is for saving/loading
> > actual
> > > >>> things created by users within apps.
> > > >>>
> > > >>> See https://developer.apple.com/library/ios/documentation/
> > > >>>
> > >
> FileManagement/Conceptual/FileSystemProgrammingGuide/FileSystemOverview/
> > > >>> FileSystemOverview.html#//apple_ref/doc/uid/TP40010672-CH2-SW14
> > > >>>
> > > >>> Supporting both should be a goal.
> > > >>>
> > > >>> Cheers,
> > > >>> Mike
> > > >>>
> > > >>>
> > > >>>
> > > >>> On 2013-12-10 14:39, Ian Clelland wrote:
> > > >>>
> > > >>> If you could do that before, it was probably a bug :) (You
> shouldn't
> > > be
> > > >>>> able to use getParent on the filesystem root, and the native code
> > was
> > > >>>> supposed to be checking for the correct path prefix before
> allowing
> > > >>>> access
> > > >>>> to any files on the device)
> > > >>>>
> > > >>>> At the moment, it's not possible to do that -- the filesystems
> > should
> > > be
> > > >>>> properly isolated, and only allow access to correctly formed urls,
> > > like
> > > >>>> filesystem://localhost/persistent/some-file-here.txt.
> > > >>>>
> > > >>>> What we *can* do easily, though, is allow a new URL scheme for
> > library
> > > >>>> assets; something like filesystem://localhost/
> > > >>>> library/some-file-here.txt,
> > > >>>> and we can have a filesystem handler for those URLs. That'll work
> if
> > > >>>> your
> > > >>>> use case is just "I need access to files in /Library", rather than
> > "I
> > > >>>> need
> > > >>>> to get to them via string manipulation".
> > > >>>>
> > > >>>> I've also had some discussions about making /Library the default
> > place
> > > >>>> for
> > > >>>> the persistent filesystem, and leaving /Documents either just for
> > > legacy
> > > >>>> apps, or making *that* the target of the special URL scheme.
> That's
> > a
> > > >>>> proposal for a different day, though. There are some pretty big
> > > >>>> backwards-compatibility issues there.
> > > >>>>
> > > >>>>
> > > >>>>
> > > >>>> On Tue, Dec 10, 2013 at 11:54 AM, Michael Gauthier <
> > > >>>> mike@silverorange.com
> > > >>>>
> > > >>>>> wrote:
> > > >>>>>
> > > >>>>
> > > >>>> Ian,
> > > >>>>
> > > >>>>>
> > > >>>>> With the new URLs will it be possible to write to both /Documents
> > and
> > > >>>>> /Library for iOS apps? In the old version, the filesystem root
> > > resolved
> > > >>>>> as
> > > >>>>> /Documents but it was possible to get to /Library by navigating
> the
> > > the
> > > >>>>> parent dir.
> > > >>>>>
> > > >>>>> Cheers,
> > > >>>>> Mike
> > > >>>>>
> > > >>>>>
> > > >>>>>
> > > >>>>> On 2013-11-15 15:19, Ian Clelland wrote:
> > > >>>>>
> > > >>>>> On Fri, Nov 15, 2013 at 1:40 PM, purplecabbage <
> > > >>>>> purplecabbage@gmail.com
> > > >>>>>
> > > >>>>>>
> > > >>>>>>> wrote:
> > > >>>>>>
> > > >>>>>> Considering the magnitude of the changes I would have
> expected
> > > that
> > > >>>>>> this
> > > >>>>>>
> > > >>>>>> was just a new file plugin. The previous version was based on a
> > > spec,
> > > >>>>>>> and
> > > >>>>>>> if we are deviating from it we should probably maintain both,
> and
> > > >>>>>>> possibly
> > > >>>>>>> even make a recommendation to the w3c.
> > > >>>>>>> I hope we at least do a major version update for this if APIs
> > have
> > > >>>>>>> changed.
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>> Yes, definitely a major version bump, at least so that devs
> > > aren't
> > > >>>>>>>
> > > >>>>>> caught
> > > >>>>>> by the URL-format-change.
> > > >>>>>>
> > > >>>>>> There aren't any changes to external APIs; I've been very
> careful
> > to
> > > >>>>>> maintain conformance with the existing tests, except where those
> > > tests
> > > >>>>>> have
> > > >>>>>> been for undocumented implementation details. The only
> app-visible
> > > >>>>>> change
> > > >>>>>> is that URLs returned by the .toURL() method will look like
> > > >>>>>>
> > > >>>>>> filesystem://localhost/persistent/my/file.jpg
> > > >>>>>>
> > > >>>>>> rather than
> > > >>>>>>
> > > >>>>>> file:///var/mobile/Applications/<app id>/Documents/my/file.jpg
> > > >>>>>>
> > > >>>>>> We are still following the spec -- in fact, this brings us
> closer
> > in
> > > >>>>>> line
> > > >>>>>> with the W3C File API spec, on these points:
> > > >>>>>>
> > > >>>>>> http://www.w3.org/TR/file-system-api/#widl-Entry-fullPath --
> The
> > > >>>>>> fullPath
> > > >>>>>> of an entry should be the path from the root (of the HTML
> > > filesystem,
> > > >>>>>> not
> > > >>>>>> the underlying device file system).
> > > >>>>>>
> > > >>>>>>
> http://www.w3.org/TR/file-system-api/#widl-Entry-toURL-DOMString--
> > > >>>>>> This
> > > >>>>>> is
> > > >>>>>> a note, rather than a requirement, but a filesystem url scheme
> is
> > > >>>>>> mentioned
> > > >>>>>> there.
> > > >>>>>>
> > > >>>>>> We're considering extending the spec in one way, which should be
> > > >>>>>> compatible
> > > >>>>>> with the spirit of the spec, and backwards-compatible with the
> > > >>>>>> previous
> > > >>>>>> API. That is the ability to declare new filesystem types, beyond
> > > >>>>>> "temporary" and "persistent", so that we can access things like
> > > device
> > > >>>>>> media and app bundle assets using the same APIs that we use for
> > > >>>>>> accessing
> > > >>>>>> user-defined files. If we're happy with the way that works, then
> > we
> > > >>>>>> should
> > > >>>>>> definitely propose it for inclusion in the standard (not the
> > > specific
> > > >>>>>> filesystems, perhaps, but the mechanism for requesting and
> > > interacting
> > > >>>>>> with
> > > >>>>>> them)
> > > >>>>>>
> > > >>>>>> Ian
> > > >>>>>>
> > > >>>>>>
> > > >>>>>>
> > > >>>>>> Sent from my iPhone
> > > >>>>>>
> > > >>>>>>>
> > > >>>>>>> On Nov 15, 2013, at 8:36 AM, Ian Clelland <
> > > iclelland@chromium.org
> > > >>>>>>> >
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>>> wrote:
> > > >>>>>>>>
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>> I've created CB-5403 as the über-ticket for this; various bits
> > of
> > > it
> > > >>>>>>>> are
> > > >>>>>>>>
> > > >>>>>>>> in
> > > >>>>>>>>
> > > >>>>>>>
> > > >>>>>>> subtasks.
> > > >>>>>>>
> > > >>>>>>>>
> > > >>>>>>>> Once I have the tests and the JS updated, then platform owners
> > can
> > > >>>>>>>> take
> > > >>>>>>>> a
> > > >>>>>>>> look and see whether they have anything to do to support their
> > own
> > > >>>>>>>>
> > > >>>>>>>> schemes.
> > > >>>>>>>>
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>> As general guidelines, I would say:
> > > >>>>>>>>
> > > >>>>>>>> * If you can intercept filesystem://* URLs in native code, and
> > > >>>>>>>> return
> > > >>>>>>>> arbitrary responses, then do it! It will let your platform
> > support
> > > >>>>>>>> new
> > > >>>>>>>> filesystems in the future as we come up with them. Add a
> couple
> > of
> > > >>>>>>>>
> > > >>>>>>>> subtasks
> > > >>>>>>>>
> > > >>>>>>>
> > > >>>>>>> for CB-5403 for your platform and go.
> > > >>>>>>>
> > > >>>>>>>>
> > > >>>>>>>> * If you can't do that, but you *can* provide access to things
> > > like
> > > >>>>>>>> media
> > > >>>>>>>> through special urls, then try that! You should be able to
> > create
> > > a
> > > >>>>>>>> FileSystem object that uses that URL as its root, and
> everything
> > > >>>>>>>> should
> > > >>>>>>>> work. Add a subtask for that, and see what you can do.
> > > >>>>>>>>
> > > >>>>>>>> * If you can't do that either, and you just want to stick with
> > the
> > > >>>>>>>>
> > > >>>>>>>> file:///
> > > >>>>>>>>
> > > >>>>>>>
> > > >>>>>>> urls that we are currently using, then don't do anything;
> > nothing
> > > >>>>>>>
> > > >>>>>>>> should
> > > >>>>>>>> change for you.
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>> On Fri, Nov 15, 2013 at 10:36 AM, Ian Clelland <
> > > >>>>>>>> iclelland@chromium.org
> > > >>>>>>>> wrote:
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>> On Fri, Nov 15, 2013 at 10:03 AM, Marcel Kinard <
> > > >>>>>>>> cmarcelk@gmail.com
> > > >>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>> wrote:
> > > >>>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>> Ian, will there be changes that app developers will need
> to
> > > do?
> > > >>>>>>>>> If
> > > >>>>>>>>> so,
> > > >>>>>>>>>
> > > >>>>>>>>> those should be clearly documented in a migration guide. If
> > not,
> > > >>>>>>>>>> it
> > > >>>>>>>>>>
> > > >>>>>>>>>> sounds
> > > >>>>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>> like the [improved] tests should pass on both the old and
> new
> > > >>>>>>>
> > > >>>>>>>> versions,
> > > >>>>>>>>
> > > >>>>>>>> which would be sweet.
> > > >>>>>>>>>
> > > >>>>>>>>>>
> > > >>>>>>>>>>
> > > >>>>>>>>>> The filesystem URLs themselves will change as a result of
> > this.
> > > >>>>>>>>> The
> > > >>>>>>>>>
> > > >>>>>>>>> tests
> > > >>>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>> should pass on both old and new versions, but the tests mint
> > > their
> > > >>>>>>> own
> > > >>>>>>>
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>>> URLs
> > > >>>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>> for testing against -- each version is internally consistent,
> > but
> > > >>>>>>> if
> > > >>>>>>>
> > > >>>>>>>> an
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>>> app
> > > >>>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>> is saving internal URLs and tries to dereference an old
> > > (file:///)
> > > >>>>>>> url
> > > >>>>>>>
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>>> with
> > > >>>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>> the new plugin, it will probably not resolve.
> > > >>>>>>>
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>>> Maybe there's a transition path here -- it might be possible
> to
> > > >>>>>>>>> allow
> > > >>>>>>>>> window.resolveLocalFileSystemURL to access files with the old
> > > URL,
> > > >>>>>>>>> but
> > > >>>>>>>>> never actually generate URLs.
> > > >>>>>>>>>
> > > >>>>>>>>> That would mean that
> > > >>>>>>>>>
> > > >>>>>>>>> (resolveLocalFileSystemURL(a)).toURL() !== a,
> > > >>>>>>>>>
> > > >>>>>>>>> but I don't think that we depend on that as an identity.
> > > >>>>>>>>>
> > > >>>>>>>>> Is there work which needs to be done on the other platforms
> > (BB,
> > > >>>>>>>>> WP8,
> > > >>>>>>>>>
> > > >>>>>>>>> Win8, FFOS, etc) so that those platforms don't get left
> > behind?
> > > >>>>>>>>> If
> > > >>>>>>>>>
> > > >>>>>>>>>> so,
> > > >>>>>>>>>> would it make sense to reach out to those platform owners
> and
> > at
> > > >>>>>>>>>> least
> > > >>>>>>>>>>
> > > >>>>>>>>>> get
> > > >>>>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>> Jira items created with some notes?
> > > >>>>>>>
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>>> Yes. I'm going to create a ticket for this, and we can add
> > > >>>>>>>>> platform-specific tasks to it.
> > > >>>>>>>>>
> > > >>>>>>>>> Other platforms shouldn't *have* to do anything, and some
> > > platforms
> > > >>>>>>>>> *can't* do anything, because they cannot access anything
> other
> > > than
> > > >>>>>>>>> file:/// urls anyway.
> > > >>>>>>>>>
> > > >>>>>>>>> However, if the other platforms want to get in on the
> goodness,
> > > and
> > > >>>>>>>>>
> > > >>>>>>>>> start
> > > >>>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>> supporting their own URL schemes (I think the BB folks can
> use
> > > >>>>>>>
> > > >>>>>>>> something
> > > >>>>>>>>
> > > >>>>>>>> like local:// for their filesystem access), then that'll be
> the
> > > >>>>>>>>> place
> > > >>>>>>>>> to
> > > >>>>>>>>> keep everything organized.
> > > >>>>>>>>>
> > > >>>>>>>>> I'll post back once I have that issue set up.
> > > >>>>>>>>>
> > > >>>>>>>>> Ian
> > > >>>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>> Sounds like you've built some very significant
> improvements.
> > > >>>>>>>>> Thanks!
> > > >>>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>>> On Nov 15, 2013, at 9:29 AM, Ian Clelland <
> > > >>>>>>>>>> iclelland@chromium.org
> > > >>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>>>>>>
> > > >>>>>>>>>>> wrote:
> > > >>>>>>>>>>>
> > > >>>>>>>>>>
> > > >>>>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>> After working with the File plugin for a week or so, I've
> > gotten
> > > >>>>>>>> it
> > > >>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>>>> more or
> > > >>>>>>>>>>>
> > > >>>>>>>>>>
> > > >>>>>>>>>> less where I want it to be on iOS, and am working through
> > > >>>>>>>>>> Android.
> > > >>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> Looking
> > > >>>>>>>>>>>
> > > >>>>>>>>>>
> > > >>>>>>>>>> at the end result, though, I expect that over 90% of the
> > code
> > > >>>>>>>>>> has
> > > >>>>>>>>>>
> > > >>>>>>>>>>> been
> > > >>>>>>>>>>> touched in some way. (It's a huge diff.)
> > > >>>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>
> > > >>>>>>
> > > >>>>>
> > > >>>>
> > > >>>
> > > >>
> > > >
> > >
> >
>
Re: Transitioning to a better File API implementation
Posted by Ian Clelland <ic...@chromium.org>.
Yeah, it would definitely require some kind of migration support. Not
suggesting that this is something that we ever actually do, but we could
give developers a bit of code that automatically looks in both places, and
moves files to the new location on open. Or we do it under a flag that is
off for existing apps (off-on-upgrade) and only enabled-by-default for new
apps.
I agree with you: this could cause worlds of pain, with developers, and
worse with users. We shouldn't take any steps in that direction without a
lot of very careful thought. For now, /Documents, sub-optimal though it may
be, is where Cordova puts persistent files.
Ian
On Wed, Dec 11, 2013 at 2:18 PM, Tommy Williams <to...@devgeeks.org> wrote:
> Changing where PERSISTENT is located sounds like a very very bad idea.
>
> I know that would cause me personally a lot of grief with existing user
> data.
> On 11/12/2013 8:58 am, "Ian Clelland" <ic...@chromium.org> wrote:
>
> > On Tue, Dec 10, 2013 at 4:37 PM, Michael Gauthier <mike@silverorange.com
> > >wrote:
> >
> > > On 10/12/13 05:02 PM, Ian Clelland wrote:
> > >
> > >> Yep.
> > >>
> > >> I think that Library is the more natural place for the HTML persistent
> > >> filesystem, since it's used by an app for whatever persistent storage
> it
> > >> needs, without exposing all of it's implementation details to the
> user.
> > >> (There's lots of room for debate on this, I'm sure)
> > >>
> > >> The trouble is that we've historically used /Documents for persistent
> > >> storage, and changing that will break apps.
> > >>
> > >> I'm fine with a BC break, but I don't have to maintain any legacy
> > > applications. /Library does make more sense as the default for
> > PERSISTENT.
> > > The big problem with BC is for installed apps with existing data on the
> > > filesystem, right?
> >
> >
> > Exactly. I'd hate for developers to update their plugins, and push a new
> > version of their app that seems fine -- everything works great when you
> > start from scratch -- but users with existing data find themselves
> > completely locked out of it.
> >
> >
> > >
> > > One idea is to allow something like requestFilesystem(DOCUMENT), in
> > >> addition to PERSISTENT and TEMPORARY. Another suggestion has been to
> add
> > >> extra arguments -- hints -- such as "{sync: true}", or maybe in this
> > case
> > >> "{purpose: documents}" to specify the attributes of the filesystem
> that
> > is
> > >> returned.
> > >>
> > > >
> > > {sync: true} is a bit tricky because /Library/Cache is not synced but
> > > /Library/Application Data is synced. Having a DOCUMENT type would match
> > > /tmp and /Library as the top-level dirs mapping to file-system
> constants.
> > >
> > > All my feedback is only from the iOS perspective though. Not sure if
> > these
> > > ideas make other platforms more or less complicated.
> > >
> > > Cheers,
> > > Mike
> > >
> > >
> > >>
> > >>
> > >> On Tue, Dec 10, 2013 at 2:39 PM, Michael Gauthier <
> > mike@silverorange.com
> > >> >wrote:
> > >>
> > >> Hmm.. The two directories have different defined roles on iOS and
> both
> > >>> are
> > >>> normal targets for applications to read/write files. Library is for
> > cache
> > >>> data or app-specific resources. Documents is for saving/loading
> actual
> > >>> things created by users within apps.
> > >>>
> > >>> See https://developer.apple.com/library/ios/documentation/
> > >>>
> > FileManagement/Conceptual/FileSystemProgrammingGuide/FileSystemOverview/
> > >>> FileSystemOverview.html#//apple_ref/doc/uid/TP40010672-CH2-SW14
> > >>>
> > >>> Supporting both should be a goal.
> > >>>
> > >>> Cheers,
> > >>> Mike
> > >>>
> > >>>
> > >>>
> > >>> On 2013-12-10 14:39, Ian Clelland wrote:
> > >>>
> > >>> If you could do that before, it was probably a bug :) (You shouldn't
> > be
> > >>>> able to use getParent on the filesystem root, and the native code
> was
> > >>>> supposed to be checking for the correct path prefix before allowing
> > >>>> access
> > >>>> to any files on the device)
> > >>>>
> > >>>> At the moment, it's not possible to do that -- the filesystems
> should
> > be
> > >>>> properly isolated, and only allow access to correctly formed urls,
> > like
> > >>>> filesystem://localhost/persistent/some-file-here.txt.
> > >>>>
> > >>>> What we *can* do easily, though, is allow a new URL scheme for
> library
> > >>>> assets; something like filesystem://localhost/
> > >>>> library/some-file-here.txt,
> > >>>> and we can have a filesystem handler for those URLs. That'll work if
> > >>>> your
> > >>>> use case is just "I need access to files in /Library", rather than
> "I
> > >>>> need
> > >>>> to get to them via string manipulation".
> > >>>>
> > >>>> I've also had some discussions about making /Library the default
> place
> > >>>> for
> > >>>> the persistent filesystem, and leaving /Documents either just for
> > legacy
> > >>>> apps, or making *that* the target of the special URL scheme. That's
> a
> > >>>> proposal for a different day, though. There are some pretty big
> > >>>> backwards-compatibility issues there.
> > >>>>
> > >>>>
> > >>>>
> > >>>> On Tue, Dec 10, 2013 at 11:54 AM, Michael Gauthier <
> > >>>> mike@silverorange.com
> > >>>>
> > >>>>> wrote:
> > >>>>>
> > >>>>
> > >>>> Ian,
> > >>>>
> > >>>>>
> > >>>>> With the new URLs will it be possible to write to both /Documents
> and
> > >>>>> /Library for iOS apps? In the old version, the filesystem root
> > resolved
> > >>>>> as
> > >>>>> /Documents but it was possible to get to /Library by navigating the
> > the
> > >>>>> parent dir.
> > >>>>>
> > >>>>> Cheers,
> > >>>>> Mike
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>> On 2013-11-15 15:19, Ian Clelland wrote:
> > >>>>>
> > >>>>> On Fri, Nov 15, 2013 at 1:40 PM, purplecabbage <
> > >>>>> purplecabbage@gmail.com
> > >>>>>
> > >>>>>>
> > >>>>>>> wrote:
> > >>>>>>
> > >>>>>> Considering the magnitude of the changes I would have expected
> > that
> > >>>>>> this
> > >>>>>>
> > >>>>>> was just a new file plugin. The previous version was based on a
> > spec,
> > >>>>>>> and
> > >>>>>>> if we are deviating from it we should probably maintain both, and
> > >>>>>>> possibly
> > >>>>>>> even make a recommendation to the w3c.
> > >>>>>>> I hope we at least do a major version update for this if APIs
> have
> > >>>>>>> changed.
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> Yes, definitely a major version bump, at least so that devs
> > aren't
> > >>>>>>>
> > >>>>>> caught
> > >>>>>> by the URL-format-change.
> > >>>>>>
> > >>>>>> There aren't any changes to external APIs; I've been very careful
> to
> > >>>>>> maintain conformance with the existing tests, except where those
> > tests
> > >>>>>> have
> > >>>>>> been for undocumented implementation details. The only app-visible
> > >>>>>> change
> > >>>>>> is that URLs returned by the .toURL() method will look like
> > >>>>>>
> > >>>>>> filesystem://localhost/persistent/my/file.jpg
> > >>>>>>
> > >>>>>> rather than
> > >>>>>>
> > >>>>>> file:///var/mobile/Applications/<app id>/Documents/my/file.jpg
> > >>>>>>
> > >>>>>> We are still following the spec -- in fact, this brings us closer
> in
> > >>>>>> line
> > >>>>>> with the W3C File API spec, on these points:
> > >>>>>>
> > >>>>>> http://www.w3.org/TR/file-system-api/#widl-Entry-fullPath -- The
> > >>>>>> fullPath
> > >>>>>> of an entry should be the path from the root (of the HTML
> > filesystem,
> > >>>>>> not
> > >>>>>> the underlying device file system).
> > >>>>>>
> > >>>>>> http://www.w3.org/TR/file-system-api/#widl-Entry-toURL-DOMString--
> > >>>>>> This
> > >>>>>> is
> > >>>>>> a note, rather than a requirement, but a filesystem url scheme is
> > >>>>>> mentioned
> > >>>>>> there.
> > >>>>>>
> > >>>>>> We're considering extending the spec in one way, which should be
> > >>>>>> compatible
> > >>>>>> with the spirit of the spec, and backwards-compatible with the
> > >>>>>> previous
> > >>>>>> API. That is the ability to declare new filesystem types, beyond
> > >>>>>> "temporary" and "persistent", so that we can access things like
> > device
> > >>>>>> media and app bundle assets using the same APIs that we use for
> > >>>>>> accessing
> > >>>>>> user-defined files. If we're happy with the way that works, then
> we
> > >>>>>> should
> > >>>>>> definitely propose it for inclusion in the standard (not the
> > specific
> > >>>>>> filesystems, perhaps, but the mechanism for requesting and
> > interacting
> > >>>>>> with
> > >>>>>> them)
> > >>>>>>
> > >>>>>> Ian
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>> Sent from my iPhone
> > >>>>>>
> > >>>>>>>
> > >>>>>>> On Nov 15, 2013, at 8:36 AM, Ian Clelland <
> > iclelland@chromium.org
> > >>>>>>> >
> > >>>>>>>
> > >>>>>>>
> > >>>>>>>> wrote:
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> I've created CB-5403 as the über-ticket for this; various bits
> of
> > it
> > >>>>>>>> are
> > >>>>>>>>
> > >>>>>>>> in
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>> subtasks.
> > >>>>>>>
> > >>>>>>>>
> > >>>>>>>> Once I have the tests and the JS updated, then platform owners
> can
> > >>>>>>>> take
> > >>>>>>>> a
> > >>>>>>>> look and see whether they have anything to do to support their
> own
> > >>>>>>>>
> > >>>>>>>> schemes.
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> As general guidelines, I would say:
> > >>>>>>>>
> > >>>>>>>> * If you can intercept filesystem://* URLs in native code, and
> > >>>>>>>> return
> > >>>>>>>> arbitrary responses, then do it! It will let your platform
> support
> > >>>>>>>> new
> > >>>>>>>> filesystems in the future as we come up with them. Add a couple
> of
> > >>>>>>>>
> > >>>>>>>> subtasks
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>> for CB-5403 for your platform and go.
> > >>>>>>>
> > >>>>>>>>
> > >>>>>>>> * If you can't do that, but you *can* provide access to things
> > like
> > >>>>>>>> media
> > >>>>>>>> through special urls, then try that! You should be able to
> create
> > a
> > >>>>>>>> FileSystem object that uses that URL as its root, and everything
> > >>>>>>>> should
> > >>>>>>>> work. Add a subtask for that, and see what you can do.
> > >>>>>>>>
> > >>>>>>>> * If you can't do that either, and you just want to stick with
> the
> > >>>>>>>>
> > >>>>>>>> file:///
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>> urls that we are currently using, then don't do anything;
> nothing
> > >>>>>>>
> > >>>>>>>> should
> > >>>>>>>> change for you.
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> On Fri, Nov 15, 2013 at 10:36 AM, Ian Clelland <
> > >>>>>>>> iclelland@chromium.org
> > >>>>>>>> wrote:
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> On Fri, Nov 15, 2013 at 10:03 AM, Marcel Kinard <
> > >>>>>>>> cmarcelk@gmail.com
> > >>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>> wrote:
> > >>>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> Ian, will there be changes that app developers will need to
> > do?
> > >>>>>>>>> If
> > >>>>>>>>> so,
> > >>>>>>>>>
> > >>>>>>>>> those should be clearly documented in a migration guide. If
> not,
> > >>>>>>>>>> it
> > >>>>>>>>>>
> > >>>>>>>>>> sounds
> > >>>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>> like the [improved] tests should pass on both the old and new
> > >>>>>>>
> > >>>>>>>> versions,
> > >>>>>>>>
> > >>>>>>>> which would be sweet.
> > >>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>> The filesystem URLs themselves will change as a result of
> this.
> > >>>>>>>>> The
> > >>>>>>>>>
> > >>>>>>>>> tests
> > >>>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>> should pass on both old and new versions, but the tests mint
> > their
> > >>>>>>> own
> > >>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>> URLs
> > >>>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>> for testing against -- each version is internally consistent,
> but
> > >>>>>>> if
> > >>>>>>>
> > >>>>>>>> an
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>> app
> > >>>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>> is saving internal URLs and tries to dereference an old
> > (file:///)
> > >>>>>>> url
> > >>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>> with
> > >>>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>> the new plugin, it will probably not resolve.
> > >>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>> Maybe there's a transition path here -- it might be possible to
> > >>>>>>>>> allow
> > >>>>>>>>> window.resolveLocalFileSystemURL to access files with the old
> > URL,
> > >>>>>>>>> but
> > >>>>>>>>> never actually generate URLs.
> > >>>>>>>>>
> > >>>>>>>>> That would mean that
> > >>>>>>>>>
> > >>>>>>>>> (resolveLocalFileSystemURL(a)).toURL() !== a,
> > >>>>>>>>>
> > >>>>>>>>> but I don't think that we depend on that as an identity.
> > >>>>>>>>>
> > >>>>>>>>> Is there work which needs to be done on the other platforms
> (BB,
> > >>>>>>>>> WP8,
> > >>>>>>>>>
> > >>>>>>>>> Win8, FFOS, etc) so that those platforms don't get left
> behind?
> > >>>>>>>>> If
> > >>>>>>>>>
> > >>>>>>>>>> so,
> > >>>>>>>>>> would it make sense to reach out to those platform owners and
> at
> > >>>>>>>>>> least
> > >>>>>>>>>>
> > >>>>>>>>>> get
> > >>>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>> Jira items created with some notes?
> > >>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>> Yes. I'm going to create a ticket for this, and we can add
> > >>>>>>>>> platform-specific tasks to it.
> > >>>>>>>>>
> > >>>>>>>>> Other platforms shouldn't *have* to do anything, and some
> > platforms
> > >>>>>>>>> *can't* do anything, because they cannot access anything other
> > than
> > >>>>>>>>> file:/// urls anyway.
> > >>>>>>>>>
> > >>>>>>>>> However, if the other platforms want to get in on the goodness,
> > and
> > >>>>>>>>>
> > >>>>>>>>> start
> > >>>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>> supporting their own URL schemes (I think the BB folks can use
> > >>>>>>>
> > >>>>>>>> something
> > >>>>>>>>
> > >>>>>>>> like local:// for their filesystem access), then that'll be the
> > >>>>>>>>> place
> > >>>>>>>>> to
> > >>>>>>>>> keep everything organized.
> > >>>>>>>>>
> > >>>>>>>>> I'll post back once I have that issue set up.
> > >>>>>>>>>
> > >>>>>>>>> Ian
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>> Sounds like you've built some very significant improvements.
> > >>>>>>>>> Thanks!
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>> On Nov 15, 2013, at 9:29 AM, Ian Clelland <
> > >>>>>>>>>> iclelland@chromium.org
> > >>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>> wrote:
> > >>>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>
> > >>>>>>> After working with the File plugin for a week or so, I've
> gotten
> > >>>>>>>> it
> > >>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>>> more or
> > >>>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>> less where I want it to be on iOS, and am working through
> > >>>>>>>>>> Android.
> > >>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>> Looking
> > >>>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>> at the end result, though, I expect that over 90% of the
> code
> > >>>>>>>>>> has
> > >>>>>>>>>>
> > >>>>>>>>>>> been
> > >>>>>>>>>>> touched in some way. (It's a huge diff.)
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>
> > >>>>>>
> > >>>>>
> > >>>>
> > >>>
> > >>
> > >
> >
>
Re: Transitioning to a better File API implementation
Posted by Tommy Williams <to...@devgeeks.org>.
Changing where PERSISTENT is located sounds like a very very bad idea.
I know that would cause me personally a lot of grief with existing user
data.
On 11/12/2013 8:58 am, "Ian Clelland" <ic...@chromium.org> wrote:
> On Tue, Dec 10, 2013 at 4:37 PM, Michael Gauthier <mike@silverorange.com
> >wrote:
>
> > On 10/12/13 05:02 PM, Ian Clelland wrote:
> >
> >> Yep.
> >>
> >> I think that Library is the more natural place for the HTML persistent
> >> filesystem, since it's used by an app for whatever persistent storage it
> >> needs, without exposing all of it's implementation details to the user.
> >> (There's lots of room for debate on this, I'm sure)
> >>
> >> The trouble is that we've historically used /Documents for persistent
> >> storage, and changing that will break apps.
> >>
> >> I'm fine with a BC break, but I don't have to maintain any legacy
> > applications. /Library does make more sense as the default for
> PERSISTENT.
> > The big problem with BC is for installed apps with existing data on the
> > filesystem, right?
>
>
> Exactly. I'd hate for developers to update their plugins, and push a new
> version of their app that seems fine -- everything works great when you
> start from scratch -- but users with existing data find themselves
> completely locked out of it.
>
>
> >
> > One idea is to allow something like requestFilesystem(DOCUMENT), in
> >> addition to PERSISTENT and TEMPORARY. Another suggestion has been to add
> >> extra arguments -- hints -- such as "{sync: true}", or maybe in this
> case
> >> "{purpose: documents}" to specify the attributes of the filesystem that
> is
> >> returned.
> >>
> > >
> > {sync: true} is a bit tricky because /Library/Cache is not synced but
> > /Library/Application Data is synced. Having a DOCUMENT type would match
> > /tmp and /Library as the top-level dirs mapping to file-system constants.
> >
> > All my feedback is only from the iOS perspective though. Not sure if
> these
> > ideas make other platforms more or less complicated.
> >
> > Cheers,
> > Mike
> >
> >
> >>
> >>
> >> On Tue, Dec 10, 2013 at 2:39 PM, Michael Gauthier <
> mike@silverorange.com
> >> >wrote:
> >>
> >> Hmm.. The two directories have different defined roles on iOS and both
> >>> are
> >>> normal targets for applications to read/write files. Library is for
> cache
> >>> data or app-specific resources. Documents is for saving/loading actual
> >>> things created by users within apps.
> >>>
> >>> See https://developer.apple.com/library/ios/documentation/
> >>>
> FileManagement/Conceptual/FileSystemProgrammingGuide/FileSystemOverview/
> >>> FileSystemOverview.html#//apple_ref/doc/uid/TP40010672-CH2-SW14
> >>>
> >>> Supporting both should be a goal.
> >>>
> >>> Cheers,
> >>> Mike
> >>>
> >>>
> >>>
> >>> On 2013-12-10 14:39, Ian Clelland wrote:
> >>>
> >>> If you could do that before, it was probably a bug :) (You shouldn't
> be
> >>>> able to use getParent on the filesystem root, and the native code was
> >>>> supposed to be checking for the correct path prefix before allowing
> >>>> access
> >>>> to any files on the device)
> >>>>
> >>>> At the moment, it's not possible to do that -- the filesystems should
> be
> >>>> properly isolated, and only allow access to correctly formed urls,
> like
> >>>> filesystem://localhost/persistent/some-file-here.txt.
> >>>>
> >>>> What we *can* do easily, though, is allow a new URL scheme for library
> >>>> assets; something like filesystem://localhost/
> >>>> library/some-file-here.txt,
> >>>> and we can have a filesystem handler for those URLs. That'll work if
> >>>> your
> >>>> use case is just "I need access to files in /Library", rather than "I
> >>>> need
> >>>> to get to them via string manipulation".
> >>>>
> >>>> I've also had some discussions about making /Library the default place
> >>>> for
> >>>> the persistent filesystem, and leaving /Documents either just for
> legacy
> >>>> apps, or making *that* the target of the special URL scheme. That's a
> >>>> proposal for a different day, though. There are some pretty big
> >>>> backwards-compatibility issues there.
> >>>>
> >>>>
> >>>>
> >>>> On Tue, Dec 10, 2013 at 11:54 AM, Michael Gauthier <
> >>>> mike@silverorange.com
> >>>>
> >>>>> wrote:
> >>>>>
> >>>>
> >>>> Ian,
> >>>>
> >>>>>
> >>>>> With the new URLs will it be possible to write to both /Documents and
> >>>>> /Library for iOS apps? In the old version, the filesystem root
> resolved
> >>>>> as
> >>>>> /Documents but it was possible to get to /Library by navigating the
> the
> >>>>> parent dir.
> >>>>>
> >>>>> Cheers,
> >>>>> Mike
> >>>>>
> >>>>>
> >>>>>
> >>>>> On 2013-11-15 15:19, Ian Clelland wrote:
> >>>>>
> >>>>> On Fri, Nov 15, 2013 at 1:40 PM, purplecabbage <
> >>>>> purplecabbage@gmail.com
> >>>>>
> >>>>>>
> >>>>>>> wrote:
> >>>>>>
> >>>>>> Considering the magnitude of the changes I would have expected
> that
> >>>>>> this
> >>>>>>
> >>>>>> was just a new file plugin. The previous version was based on a
> spec,
> >>>>>>> and
> >>>>>>> if we are deviating from it we should probably maintain both, and
> >>>>>>> possibly
> >>>>>>> even make a recommendation to the w3c.
> >>>>>>> I hope we at least do a major version update for this if APIs have
> >>>>>>> changed.
> >>>>>>>
> >>>>>>>
> >>>>>>> Yes, definitely a major version bump, at least so that devs
> aren't
> >>>>>>>
> >>>>>> caught
> >>>>>> by the URL-format-change.
> >>>>>>
> >>>>>> There aren't any changes to external APIs; I've been very careful to
> >>>>>> maintain conformance with the existing tests, except where those
> tests
> >>>>>> have
> >>>>>> been for undocumented implementation details. The only app-visible
> >>>>>> change
> >>>>>> is that URLs returned by the .toURL() method will look like
> >>>>>>
> >>>>>> filesystem://localhost/persistent/my/file.jpg
> >>>>>>
> >>>>>> rather than
> >>>>>>
> >>>>>> file:///var/mobile/Applications/<app id>/Documents/my/file.jpg
> >>>>>>
> >>>>>> We are still following the spec -- in fact, this brings us closer in
> >>>>>> line
> >>>>>> with the W3C File API spec, on these points:
> >>>>>>
> >>>>>> http://www.w3.org/TR/file-system-api/#widl-Entry-fullPath -- The
> >>>>>> fullPath
> >>>>>> of an entry should be the path from the root (of the HTML
> filesystem,
> >>>>>> not
> >>>>>> the underlying device file system).
> >>>>>>
> >>>>>> http://www.w3.org/TR/file-system-api/#widl-Entry-toURL-DOMString --
> >>>>>> This
> >>>>>> is
> >>>>>> a note, rather than a requirement, but a filesystem url scheme is
> >>>>>> mentioned
> >>>>>> there.
> >>>>>>
> >>>>>> We're considering extending the spec in one way, which should be
> >>>>>> compatible
> >>>>>> with the spirit of the spec, and backwards-compatible with the
> >>>>>> previous
> >>>>>> API. That is the ability to declare new filesystem types, beyond
> >>>>>> "temporary" and "persistent", so that we can access things like
> device
> >>>>>> media and app bundle assets using the same APIs that we use for
> >>>>>> accessing
> >>>>>> user-defined files. If we're happy with the way that works, then we
> >>>>>> should
> >>>>>> definitely propose it for inclusion in the standard (not the
> specific
> >>>>>> filesystems, perhaps, but the mechanism for requesting and
> interacting
> >>>>>> with
> >>>>>> them)
> >>>>>>
> >>>>>> Ian
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> Sent from my iPhone
> >>>>>>
> >>>>>>>
> >>>>>>> On Nov 15, 2013, at 8:36 AM, Ian Clelland <
> iclelland@chromium.org
> >>>>>>> >
> >>>>>>>
> >>>>>>>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> I've created CB-5403 as the über-ticket for this; various bits of
> it
> >>>>>>>> are
> >>>>>>>>
> >>>>>>>> in
> >>>>>>>>
> >>>>>>>
> >>>>>>> subtasks.
> >>>>>>>
> >>>>>>>>
> >>>>>>>> Once I have the tests and the JS updated, then platform owners can
> >>>>>>>> take
> >>>>>>>> a
> >>>>>>>> look and see whether they have anything to do to support their own
> >>>>>>>>
> >>>>>>>> schemes.
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> As general guidelines, I would say:
> >>>>>>>>
> >>>>>>>> * If you can intercept filesystem://* URLs in native code, and
> >>>>>>>> return
> >>>>>>>> arbitrary responses, then do it! It will let your platform support
> >>>>>>>> new
> >>>>>>>> filesystems in the future as we come up with them. Add a couple of
> >>>>>>>>
> >>>>>>>> subtasks
> >>>>>>>>
> >>>>>>>
> >>>>>>> for CB-5403 for your platform and go.
> >>>>>>>
> >>>>>>>>
> >>>>>>>> * If you can't do that, but you *can* provide access to things
> like
> >>>>>>>> media
> >>>>>>>> through special urls, then try that! You should be able to create
> a
> >>>>>>>> FileSystem object that uses that URL as its root, and everything
> >>>>>>>> should
> >>>>>>>> work. Add a subtask for that, and see what you can do.
> >>>>>>>>
> >>>>>>>> * If you can't do that either, and you just want to stick with the
> >>>>>>>>
> >>>>>>>> file:///
> >>>>>>>>
> >>>>>>>
> >>>>>>> urls that we are currently using, then don't do anything; nothing
> >>>>>>>
> >>>>>>>> should
> >>>>>>>> change for you.
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Fri, Nov 15, 2013 at 10:36 AM, Ian Clelland <
> >>>>>>>> iclelland@chromium.org
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Fri, Nov 15, 2013 at 10:03 AM, Marcel Kinard <
> >>>>>>>> cmarcelk@gmail.com
> >>>>>>>>
> >>>>>>>>>
> >>>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> Ian, will there be changes that app developers will need to
> do?
> >>>>>>>>> If
> >>>>>>>>> so,
> >>>>>>>>>
> >>>>>>>>> those should be clearly documented in a migration guide. If not,
> >>>>>>>>>> it
> >>>>>>>>>>
> >>>>>>>>>> sounds
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>> like the [improved] tests should pass on both the old and new
> >>>>>>>
> >>>>>>>> versions,
> >>>>>>>>
> >>>>>>>> which would be sweet.
> >>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> The filesystem URLs themselves will change as a result of this.
> >>>>>>>>> The
> >>>>>>>>>
> >>>>>>>>> tests
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>> should pass on both old and new versions, but the tests mint
> their
> >>>>>>> own
> >>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>> URLs
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>> for testing against -- each version is internally consistent, but
> >>>>>>> if
> >>>>>>>
> >>>>>>>> an
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>> app
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>> is saving internal URLs and tries to dereference an old
> (file:///)
> >>>>>>> url
> >>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>> with
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>> the new plugin, it will probably not resolve.
> >>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>> Maybe there's a transition path here -- it might be possible to
> >>>>>>>>> allow
> >>>>>>>>> window.resolveLocalFileSystemURL to access files with the old
> URL,
> >>>>>>>>> but
> >>>>>>>>> never actually generate URLs.
> >>>>>>>>>
> >>>>>>>>> That would mean that
> >>>>>>>>>
> >>>>>>>>> (resolveLocalFileSystemURL(a)).toURL() !== a,
> >>>>>>>>>
> >>>>>>>>> but I don't think that we depend on that as an identity.
> >>>>>>>>>
> >>>>>>>>> Is there work which needs to be done on the other platforms (BB,
> >>>>>>>>> WP8,
> >>>>>>>>>
> >>>>>>>>> Win8, FFOS, etc) so that those platforms don't get left behind?
> >>>>>>>>> If
> >>>>>>>>>
> >>>>>>>>>> so,
> >>>>>>>>>> would it make sense to reach out to those platform owners and at
> >>>>>>>>>> least
> >>>>>>>>>>
> >>>>>>>>>> get
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>> Jira items created with some notes?
> >>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>>
> >>>>>>>>>> Yes. I'm going to create a ticket for this, and we can add
> >>>>>>>>> platform-specific tasks to it.
> >>>>>>>>>
> >>>>>>>>> Other platforms shouldn't *have* to do anything, and some
> platforms
> >>>>>>>>> *can't* do anything, because they cannot access anything other
> than
> >>>>>>>>> file:/// urls anyway.
> >>>>>>>>>
> >>>>>>>>> However, if the other platforms want to get in on the goodness,
> and
> >>>>>>>>>
> >>>>>>>>> start
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>> supporting their own URL schemes (I think the BB folks can use
> >>>>>>>
> >>>>>>>> something
> >>>>>>>>
> >>>>>>>> like local:// for their filesystem access), then that'll be the
> >>>>>>>>> place
> >>>>>>>>> to
> >>>>>>>>> keep everything organized.
> >>>>>>>>>
> >>>>>>>>> I'll post back once I have that issue set up.
> >>>>>>>>>
> >>>>>>>>> Ian
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> Sounds like you've built some very significant improvements.
> >>>>>>>>> Thanks!
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>> On Nov 15, 2013, at 9:29 AM, Ian Clelland <
> >>>>>>>>>> iclelland@chromium.org
> >>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>> After working with the File plugin for a week or so, I've gotten
> >>>>>>>> it
> >>>>>>>>
> >>>>>>>>>
> >>>>>>>>>>> more or
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> less where I want it to be on iOS, and am working through
> >>>>>>>>>> Android.
> >>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> Looking
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> at the end result, though, I expect that over 90% of the code
> >>>>>>>>>> has
> >>>>>>>>>>
> >>>>>>>>>>> been
> >>>>>>>>>>> touched in some way. (It's a huge diff.)
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>>
> >>
> >
>
Re: Transitioning to a better File API implementation
Posted by Ian Clelland <ic...@chromium.org>.
On Tue, Dec 10, 2013 at 4:37 PM, Michael Gauthier <mi...@silverorange.com>wrote:
> On 10/12/13 05:02 PM, Ian Clelland wrote:
>
>> Yep.
>>
>> I think that Library is the more natural place for the HTML persistent
>> filesystem, since it's used by an app for whatever persistent storage it
>> needs, without exposing all of it's implementation details to the user.
>> (There's lots of room for debate on this, I'm sure)
>>
>> The trouble is that we've historically used /Documents for persistent
>> storage, and changing that will break apps.
>>
>> I'm fine with a BC break, but I don't have to maintain any legacy
> applications. /Library does make more sense as the default for PERSISTENT.
> The big problem with BC is for installed apps with existing data on the
> filesystem, right?
Exactly. I'd hate for developers to update their plugins, and push a new
version of their app that seems fine -- everything works great when you
start from scratch -- but users with existing data find themselves
completely locked out of it.
>
> One idea is to allow something like requestFilesystem(DOCUMENT), in
>> addition to PERSISTENT and TEMPORARY. Another suggestion has been to add
>> extra arguments -- hints -- such as "{sync: true}", or maybe in this case
>> "{purpose: documents}" to specify the attributes of the filesystem that is
>> returned.
>>
> >
> {sync: true} is a bit tricky because /Library/Cache is not synced but
> /Library/Application Data is synced. Having a DOCUMENT type would match
> /tmp and /Library as the top-level dirs mapping to file-system constants.
>
> All my feedback is only from the iOS perspective though. Not sure if these
> ideas make other platforms more or less complicated.
>
> Cheers,
> Mike
>
>
>>
>>
>> On Tue, Dec 10, 2013 at 2:39 PM, Michael Gauthier <mike@silverorange.com
>> >wrote:
>>
>> Hmm.. The two directories have different defined roles on iOS and both
>>> are
>>> normal targets for applications to read/write files. Library is for cache
>>> data or app-specific resources. Documents is for saving/loading actual
>>> things created by users within apps.
>>>
>>> See https://developer.apple.com/library/ios/documentation/
>>> FileManagement/Conceptual/FileSystemProgrammingGuide/FileSystemOverview/
>>> FileSystemOverview.html#//apple_ref/doc/uid/TP40010672-CH2-SW14
>>>
>>> Supporting both should be a goal.
>>>
>>> Cheers,
>>> Mike
>>>
>>>
>>>
>>> On 2013-12-10 14:39, Ian Clelland wrote:
>>>
>>> If you could do that before, it was probably a bug :) (You shouldn't be
>>>> able to use getParent on the filesystem root, and the native code was
>>>> supposed to be checking for the correct path prefix before allowing
>>>> access
>>>> to any files on the device)
>>>>
>>>> At the moment, it's not possible to do that -- the filesystems should be
>>>> properly isolated, and only allow access to correctly formed urls, like
>>>> filesystem://localhost/persistent/some-file-here.txt.
>>>>
>>>> What we *can* do easily, though, is allow a new URL scheme for library
>>>> assets; something like filesystem://localhost/
>>>> library/some-file-here.txt,
>>>> and we can have a filesystem handler for those URLs. That'll work if
>>>> your
>>>> use case is just "I need access to files in /Library", rather than "I
>>>> need
>>>> to get to them via string manipulation".
>>>>
>>>> I've also had some discussions about making /Library the default place
>>>> for
>>>> the persistent filesystem, and leaving /Documents either just for legacy
>>>> apps, or making *that* the target of the special URL scheme. That's a
>>>> proposal for a different day, though. There are some pretty big
>>>> backwards-compatibility issues there.
>>>>
>>>>
>>>>
>>>> On Tue, Dec 10, 2013 at 11:54 AM, Michael Gauthier <
>>>> mike@silverorange.com
>>>>
>>>>> wrote:
>>>>>
>>>>
>>>> Ian,
>>>>
>>>>>
>>>>> With the new URLs will it be possible to write to both /Documents and
>>>>> /Library for iOS apps? In the old version, the filesystem root resolved
>>>>> as
>>>>> /Documents but it was possible to get to /Library by navigating the the
>>>>> parent dir.
>>>>>
>>>>> Cheers,
>>>>> Mike
>>>>>
>>>>>
>>>>>
>>>>> On 2013-11-15 15:19, Ian Clelland wrote:
>>>>>
>>>>> On Fri, Nov 15, 2013 at 1:40 PM, purplecabbage <
>>>>> purplecabbage@gmail.com
>>>>>
>>>>>>
>>>>>>> wrote:
>>>>>>
>>>>>> Considering the magnitude of the changes I would have expected that
>>>>>> this
>>>>>>
>>>>>> was just a new file plugin. The previous version was based on a spec,
>>>>>>> and
>>>>>>> if we are deviating from it we should probably maintain both, and
>>>>>>> possibly
>>>>>>> even make a recommendation to the w3c.
>>>>>>> I hope we at least do a major version update for this if APIs have
>>>>>>> changed.
>>>>>>>
>>>>>>>
>>>>>>> Yes, definitely a major version bump, at least so that devs aren't
>>>>>>>
>>>>>> caught
>>>>>> by the URL-format-change.
>>>>>>
>>>>>> There aren't any changes to external APIs; I've been very careful to
>>>>>> maintain conformance with the existing tests, except where those tests
>>>>>> have
>>>>>> been for undocumented implementation details. The only app-visible
>>>>>> change
>>>>>> is that URLs returned by the .toURL() method will look like
>>>>>>
>>>>>> filesystem://localhost/persistent/my/file.jpg
>>>>>>
>>>>>> rather than
>>>>>>
>>>>>> file:///var/mobile/Applications/<app id>/Documents/my/file.jpg
>>>>>>
>>>>>> We are still following the spec -- in fact, this brings us closer in
>>>>>> line
>>>>>> with the W3C File API spec, on these points:
>>>>>>
>>>>>> http://www.w3.org/TR/file-system-api/#widl-Entry-fullPath -- The
>>>>>> fullPath
>>>>>> of an entry should be the path from the root (of the HTML filesystem,
>>>>>> not
>>>>>> the underlying device file system).
>>>>>>
>>>>>> http://www.w3.org/TR/file-system-api/#widl-Entry-toURL-DOMString --
>>>>>> This
>>>>>> is
>>>>>> a note, rather than a requirement, but a filesystem url scheme is
>>>>>> mentioned
>>>>>> there.
>>>>>>
>>>>>> We're considering extending the spec in one way, which should be
>>>>>> compatible
>>>>>> with the spirit of the spec, and backwards-compatible with the
>>>>>> previous
>>>>>> API. That is the ability to declare new filesystem types, beyond
>>>>>> "temporary" and "persistent", so that we can access things like device
>>>>>> media and app bundle assets using the same APIs that we use for
>>>>>> accessing
>>>>>> user-defined files. If we're happy with the way that works, then we
>>>>>> should
>>>>>> definitely propose it for inclusion in the standard (not the specific
>>>>>> filesystems, perhaps, but the mechanism for requesting and interacting
>>>>>> with
>>>>>> them)
>>>>>>
>>>>>> Ian
>>>>>>
>>>>>>
>>>>>>
>>>>>> Sent from my iPhone
>>>>>>
>>>>>>>
>>>>>>> On Nov 15, 2013, at 8:36 AM, Ian Clelland <iclelland@chromium.org
>>>>>>> >
>>>>>>>
>>>>>>>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> I've created CB-5403 as the über-ticket for this; various bits of it
>>>>>>>> are
>>>>>>>>
>>>>>>>> in
>>>>>>>>
>>>>>>>
>>>>>>> subtasks.
>>>>>>>
>>>>>>>>
>>>>>>>> Once I have the tests and the JS updated, then platform owners can
>>>>>>>> take
>>>>>>>> a
>>>>>>>> look and see whether they have anything to do to support their own
>>>>>>>>
>>>>>>>> schemes.
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> As general guidelines, I would say:
>>>>>>>>
>>>>>>>> * If you can intercept filesystem://* URLs in native code, and
>>>>>>>> return
>>>>>>>> arbitrary responses, then do it! It will let your platform support
>>>>>>>> new
>>>>>>>> filesystems in the future as we come up with them. Add a couple of
>>>>>>>>
>>>>>>>> subtasks
>>>>>>>>
>>>>>>>
>>>>>>> for CB-5403 for your platform and go.
>>>>>>>
>>>>>>>>
>>>>>>>> * If you can't do that, but you *can* provide access to things like
>>>>>>>> media
>>>>>>>> through special urls, then try that! You should be able to create a
>>>>>>>> FileSystem object that uses that URL as its root, and everything
>>>>>>>> should
>>>>>>>> work. Add a subtask for that, and see what you can do.
>>>>>>>>
>>>>>>>> * If you can't do that either, and you just want to stick with the
>>>>>>>>
>>>>>>>> file:///
>>>>>>>>
>>>>>>>
>>>>>>> urls that we are currently using, then don't do anything; nothing
>>>>>>>
>>>>>>>> should
>>>>>>>> change for you.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Fri, Nov 15, 2013 at 10:36 AM, Ian Clelland <
>>>>>>>> iclelland@chromium.org
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>> On Fri, Nov 15, 2013 at 10:03 AM, Marcel Kinard <
>>>>>>>> cmarcelk@gmail.com
>>>>>>>>
>>>>>>>>>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Ian, will there be changes that app developers will need to do?
>>>>>>>>> If
>>>>>>>>> so,
>>>>>>>>>
>>>>>>>>> those should be clearly documented in a migration guide. If not,
>>>>>>>>>> it
>>>>>>>>>>
>>>>>>>>>> sounds
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>> like the [improved] tests should pass on both the old and new
>>>>>>>
>>>>>>>> versions,
>>>>>>>>
>>>>>>>> which would be sweet.
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> The filesystem URLs themselves will change as a result of this.
>>>>>>>>> The
>>>>>>>>>
>>>>>>>>> tests
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>> should pass on both old and new versions, but the tests mint their
>>>>>>> own
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> URLs
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>> for testing against -- each version is internally consistent, but
>>>>>>> if
>>>>>>>
>>>>>>>> an
>>>>>>>>
>>>>>>>>
>>>>>>>>> app
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>> is saving internal URLs and tries to dereference an old (file:///)
>>>>>>> url
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> with
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>> the new plugin, it will probably not resolve.
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> Maybe there's a transition path here -- it might be possible to
>>>>>>>>> allow
>>>>>>>>> window.resolveLocalFileSystemURL to access files with the old URL,
>>>>>>>>> but
>>>>>>>>> never actually generate URLs.
>>>>>>>>>
>>>>>>>>> That would mean that
>>>>>>>>>
>>>>>>>>> (resolveLocalFileSystemURL(a)).toURL() !== a,
>>>>>>>>>
>>>>>>>>> but I don't think that we depend on that as an identity.
>>>>>>>>>
>>>>>>>>> Is there work which needs to be done on the other platforms (BB,
>>>>>>>>> WP8,
>>>>>>>>>
>>>>>>>>> Win8, FFOS, etc) so that those platforms don't get left behind?
>>>>>>>>> If
>>>>>>>>>
>>>>>>>>>> so,
>>>>>>>>>> would it make sense to reach out to those platform owners and at
>>>>>>>>>> least
>>>>>>>>>>
>>>>>>>>>> get
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>> Jira items created with some notes?
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>>> Yes. I'm going to create a ticket for this, and we can add
>>>>>>>>> platform-specific tasks to it.
>>>>>>>>>
>>>>>>>>> Other platforms shouldn't *have* to do anything, and some platforms
>>>>>>>>> *can't* do anything, because they cannot access anything other than
>>>>>>>>> file:/// urls anyway.
>>>>>>>>>
>>>>>>>>> However, if the other platforms want to get in on the goodness, and
>>>>>>>>>
>>>>>>>>> start
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>> supporting their own URL schemes (I think the BB folks can use
>>>>>>>
>>>>>>>> something
>>>>>>>>
>>>>>>>> like local:// for their filesystem access), then that'll be the
>>>>>>>>> place
>>>>>>>>> to
>>>>>>>>> keep everything organized.
>>>>>>>>>
>>>>>>>>> I'll post back once I have that issue set up.
>>>>>>>>>
>>>>>>>>> Ian
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Sounds like you've built some very significant improvements.
>>>>>>>>> Thanks!
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> On Nov 15, 2013, at 9:29 AM, Ian Clelland <
>>>>>>>>>> iclelland@chromium.org
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>> After working with the File plugin for a week or so, I've gotten
>>>>>>>> it
>>>>>>>>
>>>>>>>>>
>>>>>>>>>>> more or
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> less where I want it to be on iOS, and am working through
>>>>>>>>>> Android.
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Looking
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> at the end result, though, I expect that over 90% of the code
>>>>>>>>>> has
>>>>>>>>>>
>>>>>>>>>>> been
>>>>>>>>>>> touched in some way. (It's a huge diff.)
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: Transitioning to a better File API implementation
Posted by Michael Gauthier <mi...@silverorange.com>.
On 10/12/13 05:02 PM, Ian Clelland wrote:
> Yep.
>
> I think that Library is the more natural place for the HTML persistent
> filesystem, since it's used by an app for whatever persistent storage it
> needs, without exposing all of it's implementation details to the user.
> (There's lots of room for debate on this, I'm sure)
>
> The trouble is that we've historically used /Documents for persistent
> storage, and changing that will break apps.
>
I'm fine with a BC break, but I don't have to maintain any legacy
applications. /Library does make more sense as the default for
PERSISTENT. The big problem with BC is for installed apps with existing
data on the filesystem, right?
> One idea is to allow something like requestFilesystem(DOCUMENT), in
> addition to PERSISTENT and TEMPORARY. Another suggestion has been to add
> extra arguments -- hints -- such as "{sync: true}", or maybe in this case
> "{purpose: documents}" to specify the attributes of the filesystem that is
> returned.
>
{sync: true} is a bit tricky because /Library/Cache is not synced but
/Library/Application Data is synced. Having a DOCUMENT type would match
/tmp and /Library as the top-level dirs mapping to file-system constants.
All my feedback is only from the iOS perspective though. Not sure if
these ideas make other platforms more or less complicated.
Cheers,
Mike
>
>
>
> On Tue, Dec 10, 2013 at 2:39 PM, Michael Gauthier <mi...@silverorange.com>wrote:
>
>> Hmm.. The two directories have different defined roles on iOS and both are
>> normal targets for applications to read/write files. Library is for cache
>> data or app-specific resources. Documents is for saving/loading actual
>> things created by users within apps.
>>
>> See https://developer.apple.com/library/ios/documentation/
>> FileManagement/Conceptual/FileSystemProgrammingGuide/FileSystemOverview/
>> FileSystemOverview.html#//apple_ref/doc/uid/TP40010672-CH2-SW14
>>
>> Supporting both should be a goal.
>>
>> Cheers,
>> Mike
>>
>>
>>
>> On 2013-12-10 14:39, Ian Clelland wrote:
>>
>>> If you could do that before, it was probably a bug :) (You shouldn't be
>>> able to use getParent on the filesystem root, and the native code was
>>> supposed to be checking for the correct path prefix before allowing access
>>> to any files on the device)
>>>
>>> At the moment, it's not possible to do that -- the filesystems should be
>>> properly isolated, and only allow access to correctly formed urls, like
>>> filesystem://localhost/persistent/some-file-here.txt.
>>>
>>> What we *can* do easily, though, is allow a new URL scheme for library
>>> assets; something like filesystem://localhost/library/some-file-here.txt,
>>> and we can have a filesystem handler for those URLs. That'll work if your
>>> use case is just "I need access to files in /Library", rather than "I need
>>> to get to them via string manipulation".
>>>
>>> I've also had some discussions about making /Library the default place for
>>> the persistent filesystem, and leaving /Documents either just for legacy
>>> apps, or making *that* the target of the special URL scheme. That's a
>>> proposal for a different day, though. There are some pretty big
>>> backwards-compatibility issues there.
>>>
>>>
>>>
>>> On Tue, Dec 10, 2013 at 11:54 AM, Michael Gauthier <mike@silverorange.com
>>>> wrote:
>>>
>>> Ian,
>>>>
>>>> With the new URLs will it be possible to write to both /Documents and
>>>> /Library for iOS apps? In the old version, the filesystem root resolved
>>>> as
>>>> /Documents but it was possible to get to /Library by navigating the the
>>>> parent dir.
>>>>
>>>> Cheers,
>>>> Mike
>>>>
>>>>
>>>>
>>>> On 2013-11-15 15:19, Ian Clelland wrote:
>>>>
>>>> On Fri, Nov 15, 2013 at 1:40 PM, purplecabbage <purplecabbage@gmail.com
>>>>>>
>>>>> wrote:
>>>>>
>>>>> Considering the magnitude of the changes I would have expected that
>>>>> this
>>>>>
>>>>>> was just a new file plugin. The previous version was based on a spec,
>>>>>> and
>>>>>> if we are deviating from it we should probably maintain both, and
>>>>>> possibly
>>>>>> even make a recommendation to the w3c.
>>>>>> I hope we at least do a major version update for this if APIs have
>>>>>> changed.
>>>>>>
>>>>>>
>>>>>> Yes, definitely a major version bump, at least so that devs aren't
>>>>> caught
>>>>> by the URL-format-change.
>>>>>
>>>>> There aren't any changes to external APIs; I've been very careful to
>>>>> maintain conformance with the existing tests, except where those tests
>>>>> have
>>>>> been for undocumented implementation details. The only app-visible
>>>>> change
>>>>> is that URLs returned by the .toURL() method will look like
>>>>>
>>>>> filesystem://localhost/persistent/my/file.jpg
>>>>>
>>>>> rather than
>>>>>
>>>>> file:///var/mobile/Applications/<app id>/Documents/my/file.jpg
>>>>>
>>>>> We are still following the spec -- in fact, this brings us closer in
>>>>> line
>>>>> with the W3C File API spec, on these points:
>>>>>
>>>>> http://www.w3.org/TR/file-system-api/#widl-Entry-fullPath -- The
>>>>> fullPath
>>>>> of an entry should be the path from the root (of the HTML filesystem,
>>>>> not
>>>>> the underlying device file system).
>>>>>
>>>>> http://www.w3.org/TR/file-system-api/#widl-Entry-toURL-DOMString --
>>>>> This
>>>>> is
>>>>> a note, rather than a requirement, but a filesystem url scheme is
>>>>> mentioned
>>>>> there.
>>>>>
>>>>> We're considering extending the spec in one way, which should be
>>>>> compatible
>>>>> with the spirit of the spec, and backwards-compatible with the previous
>>>>> API. That is the ability to declare new filesystem types, beyond
>>>>> "temporary" and "persistent", so that we can access things like device
>>>>> media and app bundle assets using the same APIs that we use for
>>>>> accessing
>>>>> user-defined files. If we're happy with the way that works, then we
>>>>> should
>>>>> definitely propose it for inclusion in the standard (not the specific
>>>>> filesystems, perhaps, but the mechanism for requesting and interacting
>>>>> with
>>>>> them)
>>>>>
>>>>> Ian
>>>>>
>>>>>
>>>>>
>>>>> Sent from my iPhone
>>>>>>
>>>>>> On Nov 15, 2013, at 8:36 AM, Ian Clelland <ic...@chromium.org>
>>>>>>
>>>>>>>
>>>>>>> wrote:
>>>>>>
>>>>>>
>>>>>>> I've created CB-5403 as the über-ticket for this; various bits of it
>>>>>>> are
>>>>>>>
>>>>>>> in
>>>>>>
>>>>>> subtasks.
>>>>>>>
>>>>>>> Once I have the tests and the JS updated, then platform owners can
>>>>>>> take
>>>>>>> a
>>>>>>> look and see whether they have anything to do to support their own
>>>>>>>
>>>>>>> schemes.
>>>>>>
>>>>>>
>>>>>>> As general guidelines, I would say:
>>>>>>>
>>>>>>> * If you can intercept filesystem://* URLs in native code, and return
>>>>>>> arbitrary responses, then do it! It will let your platform support new
>>>>>>> filesystems in the future as we come up with them. Add a couple of
>>>>>>>
>>>>>>> subtasks
>>>>>>
>>>>>> for CB-5403 for your platform and go.
>>>>>>>
>>>>>>> * If you can't do that, but you *can* provide access to things like
>>>>>>> media
>>>>>>> through special urls, then try that! You should be able to create a
>>>>>>> FileSystem object that uses that URL as its root, and everything
>>>>>>> should
>>>>>>> work. Add a subtask for that, and see what you can do.
>>>>>>>
>>>>>>> * If you can't do that either, and you just want to stick with the
>>>>>>>
>>>>>>> file:///
>>>>>>
>>>>>> urls that we are currently using, then don't do anything; nothing
>>>>>>> should
>>>>>>> change for you.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Fri, Nov 15, 2013 at 10:36 AM, Ian Clelland <
>>>>>>> iclelland@chromium.org
>>>>>>> wrote:
>>>>>>>
>>>>>>>
>>>>>>> On Fri, Nov 15, 2013 at 10:03 AM, Marcel Kinard <cmarcelk@gmail.com
>>>>>>>>
>>>>>>>> wrote:
>>>>>>>
>>>>>>>
>>>>>>>> Ian, will there be changes that app developers will need to do? If
>>>>>>>> so,
>>>>>>>>
>>>>>>>>> those should be clearly documented in a migration guide. If not, it
>>>>>>>>>
>>>>>>>>> sounds
>>>>>>>>
>>>>>>>
>>>>>> like the [improved] tests should pass on both the old and new
>>>>>>> versions,
>>>>>>>
>>>>>>>> which would be sweet.
>>>>>>>>>
>>>>>>>>>
>>>>>>>> The filesystem URLs themselves will change as a result of this. The
>>>>>>>>
>>>>>>>> tests
>>>>>>>
>>>>>>
>>>>>> should pass on both old and new versions, but the tests mint their own
>>>>>>>
>>>>>>>>
>>>>>>>> URLs
>>>>>>>
>>>>>>
>>>>>> for testing against -- each version is internally consistent, but if
>>>>>>> an
>>>>>>>
>>>>>>>>
>>>>>>>> app
>>>>>>>
>>>>>>
>>>>>> is saving internal URLs and tries to dereference an old (file:///) url
>>>>>>>
>>>>>>>>
>>>>>>>> with
>>>>>>>
>>>>>>
>>>>>> the new plugin, it will probably not resolve.
>>>>>>>
>>>>>>>>
>>>>>>>> Maybe there's a transition path here -- it might be possible to allow
>>>>>>>> window.resolveLocalFileSystemURL to access files with the old URL,
>>>>>>>> but
>>>>>>>> never actually generate URLs.
>>>>>>>>
>>>>>>>> That would mean that
>>>>>>>>
>>>>>>>> (resolveLocalFileSystemURL(a)).toURL() !== a,
>>>>>>>>
>>>>>>>> but I don't think that we depend on that as an identity.
>>>>>>>>
>>>>>>>> Is there work which needs to be done on the other platforms (BB, WP8,
>>>>>>>>
>>>>>>>> Win8, FFOS, etc) so that those platforms don't get left behind? If
>>>>>>>>> so,
>>>>>>>>> would it make sense to reach out to those platform owners and at
>>>>>>>>> least
>>>>>>>>>
>>>>>>>>> get
>>>>>>>>
>>>>>>>
>>>>>> Jira items created with some notes?
>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>> Yes. I'm going to create a ticket for this, and we can add
>>>>>>>> platform-specific tasks to it.
>>>>>>>>
>>>>>>>> Other platforms shouldn't *have* to do anything, and some platforms
>>>>>>>> *can't* do anything, because they cannot access anything other than
>>>>>>>> file:/// urls anyway.
>>>>>>>>
>>>>>>>> However, if the other platforms want to get in on the goodness, and
>>>>>>>>
>>>>>>>> start
>>>>>>>
>>>>>>
>>>>>> supporting their own URL schemes (I think the BB folks can use
>>>>>>> something
>>>>>>>
>>>>>>>> like local:// for their filesystem access), then that'll be the place
>>>>>>>> to
>>>>>>>> keep everything organized.
>>>>>>>>
>>>>>>>> I'll post back once I have that issue set up.
>>>>>>>>
>>>>>>>> Ian
>>>>>>>>
>>>>>>>>
>>>>>>>> Sounds like you've built some very significant improvements.
>>>>>>>> Thanks!
>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Nov 15, 2013, at 9:29 AM, Ian Clelland <iclelland@chromium.org
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>>> After working with the File plugin for a week or so, I've gotten it
>>>>>>>>>>
>>>>>>>>>> more or
>>>>>>>>>
>>>>>>>>> less where I want it to be on iOS, and am working through Android.
>>>>>>>>>>
>>>>>>>>>> Looking
>>>>>>>>>
>>>>>>>>> at the end result, though, I expect that over 90% of the code has
>>>>>>>>>> been
>>>>>>>>>> touched in some way. (It's a huge diff.)
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: Transitioning to a better File API implementation
Posted by Ian Clelland <ic...@chromium.org>.
Yep.
I think that Library is the more natural place for the HTML persistent
filesystem, since it's used by an app for whatever persistent storage it
needs, without exposing all of it's implementation details to the user.
(There's lots of room for debate on this, I'm sure)
The trouble is that we've historically used /Documents for persistent
storage, and changing that will break apps.
One idea is to allow something like requestFilesystem(DOCUMENT), in
addition to PERSISTENT and TEMPORARY. Another suggestion has been to add
extra arguments -- hints -- such as "{sync: true}", or maybe in this case
"{purpose: documents}" to specify the attributes of the filesystem that is
returned.
On Tue, Dec 10, 2013 at 2:39 PM, Michael Gauthier <mi...@silverorange.com>wrote:
> Hmm.. The two directories have different defined roles on iOS and both are
> normal targets for applications to read/write files. Library is for cache
> data or app-specific resources. Documents is for saving/loading actual
> things created by users within apps.
>
> See https://developer.apple.com/library/ios/documentation/
> FileManagement/Conceptual/FileSystemProgrammingGuide/FileSystemOverview/
> FileSystemOverview.html#//apple_ref/doc/uid/TP40010672-CH2-SW14
>
> Supporting both should be a goal.
>
> Cheers,
> Mike
>
>
>
> On 2013-12-10 14:39, Ian Clelland wrote:
>
>> If you could do that before, it was probably a bug :) (You shouldn't be
>> able to use getParent on the filesystem root, and the native code was
>> supposed to be checking for the correct path prefix before allowing access
>> to any files on the device)
>>
>> At the moment, it's not possible to do that -- the filesystems should be
>> properly isolated, and only allow access to correctly formed urls, like
>> filesystem://localhost/persistent/some-file-here.txt.
>>
>> What we *can* do easily, though, is allow a new URL scheme for library
>> assets; something like filesystem://localhost/library/some-file-here.txt,
>> and we can have a filesystem handler for those URLs. That'll work if your
>> use case is just "I need access to files in /Library", rather than "I need
>> to get to them via string manipulation".
>>
>> I've also had some discussions about making /Library the default place for
>> the persistent filesystem, and leaving /Documents either just for legacy
>> apps, or making *that* the target of the special URL scheme. That's a
>> proposal for a different day, though. There are some pretty big
>> backwards-compatibility issues there.
>>
>>
>>
>> On Tue, Dec 10, 2013 at 11:54 AM, Michael Gauthier <mike@silverorange.com
>> >wrote:
>>
>> Ian,
>>>
>>> With the new URLs will it be possible to write to both /Documents and
>>> /Library for iOS apps? In the old version, the filesystem root resolved
>>> as
>>> /Documents but it was possible to get to /Library by navigating the the
>>> parent dir.
>>>
>>> Cheers,
>>> Mike
>>>
>>>
>>>
>>> On 2013-11-15 15:19, Ian Clelland wrote:
>>>
>>> On Fri, Nov 15, 2013 at 1:40 PM, purplecabbage <purplecabbage@gmail.com
>>>> >
>>>> wrote:
>>>>
>>>> Considering the magnitude of the changes I would have expected that
>>>> this
>>>>
>>>>> was just a new file plugin. The previous version was based on a spec,
>>>>> and
>>>>> if we are deviating from it we should probably maintain both, and
>>>>> possibly
>>>>> even make a recommendation to the w3c.
>>>>> I hope we at least do a major version update for this if APIs have
>>>>> changed.
>>>>>
>>>>>
>>>>> Yes, definitely a major version bump, at least so that devs aren't
>>>> caught
>>>> by the URL-format-change.
>>>>
>>>> There aren't any changes to external APIs; I've been very careful to
>>>> maintain conformance with the existing tests, except where those tests
>>>> have
>>>> been for undocumented implementation details. The only app-visible
>>>> change
>>>> is that URLs returned by the .toURL() method will look like
>>>>
>>>> filesystem://localhost/persistent/my/file.jpg
>>>>
>>>> rather than
>>>>
>>>> file:///var/mobile/Applications/<app id>/Documents/my/file.jpg
>>>>
>>>> We are still following the spec -- in fact, this brings us closer in
>>>> line
>>>> with the W3C File API spec, on these points:
>>>>
>>>> http://www.w3.org/TR/file-system-api/#widl-Entry-fullPath -- The
>>>> fullPath
>>>> of an entry should be the path from the root (of the HTML filesystem,
>>>> not
>>>> the underlying device file system).
>>>>
>>>> http://www.w3.org/TR/file-system-api/#widl-Entry-toURL-DOMString --
>>>> This
>>>> is
>>>> a note, rather than a requirement, but a filesystem url scheme is
>>>> mentioned
>>>> there.
>>>>
>>>> We're considering extending the spec in one way, which should be
>>>> compatible
>>>> with the spirit of the spec, and backwards-compatible with the previous
>>>> API. That is the ability to declare new filesystem types, beyond
>>>> "temporary" and "persistent", so that we can access things like device
>>>> media and app bundle assets using the same APIs that we use for
>>>> accessing
>>>> user-defined files. If we're happy with the way that works, then we
>>>> should
>>>> definitely propose it for inclusion in the standard (not the specific
>>>> filesystems, perhaps, but the mechanism for requesting and interacting
>>>> with
>>>> them)
>>>>
>>>> Ian
>>>>
>>>>
>>>>
>>>> Sent from my iPhone
>>>>>
>>>>> On Nov 15, 2013, at 8:36 AM, Ian Clelland <ic...@chromium.org>
>>>>>
>>>>>>
>>>>>> wrote:
>>>>>
>>>>>
>>>>>> I've created CB-5403 as the über-ticket for this; various bits of it
>>>>>> are
>>>>>>
>>>>>> in
>>>>>
>>>>> subtasks.
>>>>>>
>>>>>> Once I have the tests and the JS updated, then platform owners can
>>>>>> take
>>>>>> a
>>>>>> look and see whether they have anything to do to support their own
>>>>>>
>>>>>> schemes.
>>>>>
>>>>>
>>>>>> As general guidelines, I would say:
>>>>>>
>>>>>> * If you can intercept filesystem://* URLs in native code, and return
>>>>>> arbitrary responses, then do it! It will let your platform support new
>>>>>> filesystems in the future as we come up with them. Add a couple of
>>>>>>
>>>>>> subtasks
>>>>>
>>>>> for CB-5403 for your platform and go.
>>>>>>
>>>>>> * If you can't do that, but you *can* provide access to things like
>>>>>> media
>>>>>> through special urls, then try that! You should be able to create a
>>>>>> FileSystem object that uses that URL as its root, and everything
>>>>>> should
>>>>>> work. Add a subtask for that, and see what you can do.
>>>>>>
>>>>>> * If you can't do that either, and you just want to stick with the
>>>>>>
>>>>>> file:///
>>>>>
>>>>> urls that we are currently using, then don't do anything; nothing
>>>>>> should
>>>>>> change for you.
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Fri, Nov 15, 2013 at 10:36 AM, Ian Clelland <
>>>>>> iclelland@chromium.org
>>>>>> wrote:
>>>>>>
>>>>>>
>>>>>> On Fri, Nov 15, 2013 at 10:03 AM, Marcel Kinard <cmarcelk@gmail.com
>>>>>>>
>>>>>>> wrote:
>>>>>>
>>>>>>
>>>>>>> Ian, will there be changes that app developers will need to do? If
>>>>>>> so,
>>>>>>>
>>>>>>>> those should be clearly documented in a migration guide. If not, it
>>>>>>>>
>>>>>>>> sounds
>>>>>>>
>>>>>>
>>>>> like the [improved] tests should pass on both the old and new
>>>>>> versions,
>>>>>>
>>>>>>> which would be sweet.
>>>>>>>>
>>>>>>>>
>>>>>>> The filesystem URLs themselves will change as a result of this. The
>>>>>>>
>>>>>>> tests
>>>>>>
>>>>>
>>>>> should pass on both old and new versions, but the tests mint their own
>>>>>>
>>>>>>>
>>>>>>> URLs
>>>>>>
>>>>>
>>>>> for testing against -- each version is internally consistent, but if
>>>>>> an
>>>>>>
>>>>>>>
>>>>>>> app
>>>>>>
>>>>>
>>>>> is saving internal URLs and tries to dereference an old (file:///) url
>>>>>>
>>>>>>>
>>>>>>> with
>>>>>>
>>>>>
>>>>> the new plugin, it will probably not resolve.
>>>>>>
>>>>>>>
>>>>>>> Maybe there's a transition path here -- it might be possible to allow
>>>>>>> window.resolveLocalFileSystemURL to access files with the old URL,
>>>>>>> but
>>>>>>> never actually generate URLs.
>>>>>>>
>>>>>>> That would mean that
>>>>>>>
>>>>>>> (resolveLocalFileSystemURL(a)).toURL() !== a,
>>>>>>>
>>>>>>> but I don't think that we depend on that as an identity.
>>>>>>>
>>>>>>> Is there work which needs to be done on the other platforms (BB, WP8,
>>>>>>>
>>>>>>> Win8, FFOS, etc) so that those platforms don't get left behind? If
>>>>>>>> so,
>>>>>>>> would it make sense to reach out to those platform owners and at
>>>>>>>> least
>>>>>>>>
>>>>>>>> get
>>>>>>>
>>>>>>
>>>>> Jira items created with some notes?
>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>> Yes. I'm going to create a ticket for this, and we can add
>>>>>>> platform-specific tasks to it.
>>>>>>>
>>>>>>> Other platforms shouldn't *have* to do anything, and some platforms
>>>>>>> *can't* do anything, because they cannot access anything other than
>>>>>>> file:/// urls anyway.
>>>>>>>
>>>>>>> However, if the other platforms want to get in on the goodness, and
>>>>>>>
>>>>>>> start
>>>>>>
>>>>>
>>>>> supporting their own URL schemes (I think the BB folks can use
>>>>>> something
>>>>>>
>>>>>>> like local:// for their filesystem access), then that'll be the place
>>>>>>> to
>>>>>>> keep everything organized.
>>>>>>>
>>>>>>> I'll post back once I have that issue set up.
>>>>>>>
>>>>>>> Ian
>>>>>>>
>>>>>>>
>>>>>>> Sounds like you've built some very significant improvements.
>>>>>>> Thanks!
>>>>>>>
>>>>>>>>
>>>>>>>> On Nov 15, 2013, at 9:29 AM, Ian Clelland <iclelland@chromium.org
>>>>>>>> >
>>>>>>>>
>>>>>>>>>
>>>>>>>>> wrote:
>>>>>>>>
>>>>>>>
>>>>>
>>>>>> After working with the File plugin for a week or so, I've gotten it
>>>>>>>>>
>>>>>>>>> more or
>>>>>>>>
>>>>>>>> less where I want it to be on iOS, and am working through Android.
>>>>>>>>>
>>>>>>>>> Looking
>>>>>>>>
>>>>>>>> at the end result, though, I expect that over 90% of the code has
>>>>>>>>> been
>>>>>>>>> touched in some way. (It's a huge diff.)
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: Transitioning to a better File API implementation
Posted by Michael Gauthier <mi...@silverorange.com>.
Hmm.. The two directories have different defined roles on iOS and both
are normal targets for applications to read/write files. Library is for
cache data or app-specific resources. Documents is for saving/loading
actual things created by users within apps.
See
https://developer.apple.com/library/ios/documentation/FileManagement/Conceptual/FileSystemProgrammingGuide/FileSystemOverview/FileSystemOverview.html#//apple_ref/doc/uid/TP40010672-CH2-SW14
Supporting both should be a goal.
Cheers,
Mike
On 2013-12-10 14:39, Ian Clelland wrote:
> If you could do that before, it was probably a bug :) (You shouldn't be
> able to use getParent on the filesystem root, and the native code was
> supposed to be checking for the correct path prefix before allowing access
> to any files on the device)
>
> At the moment, it's not possible to do that -- the filesystems should be
> properly isolated, and only allow access to correctly formed urls, like
> filesystem://localhost/persistent/some-file-here.txt.
>
> What we *can* do easily, though, is allow a new URL scheme for library
> assets; something like filesystem://localhost/library/some-file-here.txt,
> and we can have a filesystem handler for those URLs. That'll work if your
> use case is just "I need access to files in /Library", rather than "I need
> to get to them via string manipulation".
>
> I've also had some discussions about making /Library the default place for
> the persistent filesystem, and leaving /Documents either just for legacy
> apps, or making *that* the target of the special URL scheme. That's a
> proposal for a different day, though. There are some pretty big
> backwards-compatibility issues there.
>
>
>
> On Tue, Dec 10, 2013 at 11:54 AM, Michael Gauthier <mi...@silverorange.com>wrote:
>
>> Ian,
>>
>> With the new URLs will it be possible to write to both /Documents and
>> /Library for iOS apps? In the old version, the filesystem root resolved as
>> /Documents but it was possible to get to /Library by navigating the the
>> parent dir.
>>
>> Cheers,
>> Mike
>>
>>
>>
>> On 2013-11-15 15:19, Ian Clelland wrote:
>>
>>> On Fri, Nov 15, 2013 at 1:40 PM, purplecabbage <pu...@gmail.com>
>>> wrote:
>>>
>>> Considering the magnitude of the changes I would have expected that this
>>>> was just a new file plugin. The previous version was based on a spec, and
>>>> if we are deviating from it we should probably maintain both, and
>>>> possibly
>>>> even make a recommendation to the w3c.
>>>> I hope we at least do a major version update for this if APIs have
>>>> changed.
>>>>
>>>>
>>> Yes, definitely a major version bump, at least so that devs aren't caught
>>> by the URL-format-change.
>>>
>>> There aren't any changes to external APIs; I've been very careful to
>>> maintain conformance with the existing tests, except where those tests
>>> have
>>> been for undocumented implementation details. The only app-visible change
>>> is that URLs returned by the .toURL() method will look like
>>>
>>> filesystem://localhost/persistent/my/file.jpg
>>>
>>> rather than
>>>
>>> file:///var/mobile/Applications/<app id>/Documents/my/file.jpg
>>>
>>> We are still following the spec -- in fact, this brings us closer in line
>>> with the W3C File API spec, on these points:
>>>
>>> http://www.w3.org/TR/file-system-api/#widl-Entry-fullPath -- The fullPath
>>> of an entry should be the path from the root (of the HTML filesystem, not
>>> the underlying device file system).
>>>
>>> http://www.w3.org/TR/file-system-api/#widl-Entry-toURL-DOMString -- This
>>> is
>>> a note, rather than a requirement, but a filesystem url scheme is
>>> mentioned
>>> there.
>>>
>>> We're considering extending the spec in one way, which should be
>>> compatible
>>> with the spirit of the spec, and backwards-compatible with the previous
>>> API. That is the ability to declare new filesystem types, beyond
>>> "temporary" and "persistent", so that we can access things like device
>>> media and app bundle assets using the same APIs that we use for accessing
>>> user-defined files. If we're happy with the way that works, then we should
>>> definitely propose it for inclusion in the standard (not the specific
>>> filesystems, perhaps, but the mechanism for requesting and interacting
>>> with
>>> them)
>>>
>>> Ian
>>>
>>>
>>>
>>>> Sent from my iPhone
>>>>
>>>> On Nov 15, 2013, at 8:36 AM, Ian Clelland <ic...@chromium.org>
>>>>>
>>>> wrote:
>>>>
>>>>>
>>>>> I've created CB-5403 as the über-ticket for this; various bits of it are
>>>>>
>>>> in
>>>>
>>>>> subtasks.
>>>>>
>>>>> Once I have the tests and the JS updated, then platform owners can take
>>>>> a
>>>>> look and see whether they have anything to do to support their own
>>>>>
>>>> schemes.
>>>>
>>>>>
>>>>> As general guidelines, I would say:
>>>>>
>>>>> * If you can intercept filesystem://* URLs in native code, and return
>>>>> arbitrary responses, then do it! It will let your platform support new
>>>>> filesystems in the future as we come up with them. Add a couple of
>>>>>
>>>> subtasks
>>>>
>>>>> for CB-5403 for your platform and go.
>>>>>
>>>>> * If you can't do that, but you *can* provide access to things like
>>>>> media
>>>>> through special urls, then try that! You should be able to create a
>>>>> FileSystem object that uses that URL as its root, and everything should
>>>>> work. Add a subtask for that, and see what you can do.
>>>>>
>>>>> * If you can't do that either, and you just want to stick with the
>>>>>
>>>> file:///
>>>>
>>>>> urls that we are currently using, then don't do anything; nothing should
>>>>> change for you.
>>>>>
>>>>>
>>>>>
>>>>> On Fri, Nov 15, 2013 at 10:36 AM, Ian Clelland <iclelland@chromium.org
>>>>> wrote:
>>>>>
>>>>>
>>>>>> On Fri, Nov 15, 2013 at 10:03 AM, Marcel Kinard <cmarcelk@gmail.com
>>>>>>
>>>>> wrote:
>>>>>
>>>>>>
>>>>>> Ian, will there be changes that app developers will need to do? If so,
>>>>>>> those should be clearly documented in a migration guide. If not, it
>>>>>>>
>>>>>> sounds
>>>>
>>>>> like the [improved] tests should pass on both the old and new versions,
>>>>>>> which would be sweet.
>>>>>>>
>>>>>>
>>>>>> The filesystem URLs themselves will change as a result of this. The
>>>>>>
>>>>> tests
>>>>
>>>>> should pass on both old and new versions, but the tests mint their own
>>>>>>
>>>>> URLs
>>>>
>>>>> for testing against -- each version is internally consistent, but if an
>>>>>>
>>>>> app
>>>>
>>>>> is saving internal URLs and tries to dereference an old (file:///) url
>>>>>>
>>>>> with
>>>>
>>>>> the new plugin, it will probably not resolve.
>>>>>>
>>>>>> Maybe there's a transition path here -- it might be possible to allow
>>>>>> window.resolveLocalFileSystemURL to access files with the old URL, but
>>>>>> never actually generate URLs.
>>>>>>
>>>>>> That would mean that
>>>>>>
>>>>>> (resolveLocalFileSystemURL(a)).toURL() !== a,
>>>>>>
>>>>>> but I don't think that we depend on that as an identity.
>>>>>>
>>>>>> Is there work which needs to be done on the other platforms (BB, WP8,
>>>>>>
>>>>>>> Win8, FFOS, etc) so that those platforms don't get left behind? If so,
>>>>>>> would it make sense to reach out to those platform owners and at least
>>>>>>>
>>>>>> get
>>>>
>>>>> Jira items created with some notes?
>>>>>>>
>>>>>>
>>>>>> Yes. I'm going to create a ticket for this, and we can add
>>>>>> platform-specific tasks to it.
>>>>>>
>>>>>> Other platforms shouldn't *have* to do anything, and some platforms
>>>>>> *can't* do anything, because they cannot access anything other than
>>>>>> file:/// urls anyway.
>>>>>>
>>>>>> However, if the other platforms want to get in on the goodness, and
>>>>>>
>>>>> start
>>>>
>>>>> supporting their own URL schemes (I think the BB folks can use something
>>>>>> like local:// for their filesystem access), then that'll be the place
>>>>>> to
>>>>>> keep everything organized.
>>>>>>
>>>>>> I'll post back once I have that issue set up.
>>>>>>
>>>>>> Ian
>>>>>>
>>>>>>
>>>>>> Sounds like you've built some very significant improvements. Thanks!
>>>>>>>
>>>>>>> On Nov 15, 2013, at 9:29 AM, Ian Clelland <ic...@chromium.org>
>>>>>>>>
>>>>>>> wrote:
>>>>
>>>>>
>>>>>>>> After working with the File plugin for a week or so, I've gotten it
>>>>>>>>
>>>>>>> more or
>>>>>>>
>>>>>>>> less where I want it to be on iOS, and am working through Android.
>>>>>>>>
>>>>>>> Looking
>>>>>>>
>>>>>>>> at the end result, though, I expect that over 90% of the code has
>>>>>>>> been
>>>>>>>> touched in some way. (It's a huge diff.)
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>
>>>
>>
>
Re: Transitioning to a better File API implementation
Posted by Ian Clelland <ic...@chromium.org>.
If you could do that before, it was probably a bug :) (You shouldn't be
able to use getParent on the filesystem root, and the native code was
supposed to be checking for the correct path prefix before allowing access
to any files on the device)
At the moment, it's not possible to do that -- the filesystems should be
properly isolated, and only allow access to correctly formed urls, like
filesystem://localhost/persistent/some-file-here.txt.
What we *can* do easily, though, is allow a new URL scheme for library
assets; something like filesystem://localhost/library/some-file-here.txt,
and we can have a filesystem handler for those URLs. That'll work if your
use case is just "I need access to files in /Library", rather than "I need
to get to them via string manipulation".
I've also had some discussions about making /Library the default place for
the persistent filesystem, and leaving /Documents either just for legacy
apps, or making *that* the target of the special URL scheme. That's a
proposal for a different day, though. There are some pretty big
backwards-compatibility issues there.
On Tue, Dec 10, 2013 at 11:54 AM, Michael Gauthier <mi...@silverorange.com>wrote:
> Ian,
>
> With the new URLs will it be possible to write to both /Documents and
> /Library for iOS apps? In the old version, the filesystem root resolved as
> /Documents but it was possible to get to /Library by navigating the the
> parent dir.
>
> Cheers,
> Mike
>
>
>
> On 2013-11-15 15:19, Ian Clelland wrote:
>
>> On Fri, Nov 15, 2013 at 1:40 PM, purplecabbage <pu...@gmail.com>
>> wrote:
>>
>> Considering the magnitude of the changes I would have expected that this
>>> was just a new file plugin. The previous version was based on a spec, and
>>> if we are deviating from it we should probably maintain both, and
>>> possibly
>>> even make a recommendation to the w3c.
>>> I hope we at least do a major version update for this if APIs have
>>> changed.
>>>
>>>
>> Yes, definitely a major version bump, at least so that devs aren't caught
>> by the URL-format-change.
>>
>> There aren't any changes to external APIs; I've been very careful to
>> maintain conformance with the existing tests, except where those tests
>> have
>> been for undocumented implementation details. The only app-visible change
>> is that URLs returned by the .toURL() method will look like
>>
>> filesystem://localhost/persistent/my/file.jpg
>>
>> rather than
>>
>> file:///var/mobile/Applications/<app id>/Documents/my/file.jpg
>>
>> We are still following the spec -- in fact, this brings us closer in line
>> with the W3C File API spec, on these points:
>>
>> http://www.w3.org/TR/file-system-api/#widl-Entry-fullPath -- The fullPath
>> of an entry should be the path from the root (of the HTML filesystem, not
>> the underlying device file system).
>>
>> http://www.w3.org/TR/file-system-api/#widl-Entry-toURL-DOMString -- This
>> is
>> a note, rather than a requirement, but a filesystem url scheme is
>> mentioned
>> there.
>>
>> We're considering extending the spec in one way, which should be
>> compatible
>> with the spirit of the spec, and backwards-compatible with the previous
>> API. That is the ability to declare new filesystem types, beyond
>> "temporary" and "persistent", so that we can access things like device
>> media and app bundle assets using the same APIs that we use for accessing
>> user-defined files. If we're happy with the way that works, then we should
>> definitely propose it for inclusion in the standard (not the specific
>> filesystems, perhaps, but the mechanism for requesting and interacting
>> with
>> them)
>>
>> Ian
>>
>>
>>
>>> Sent from my iPhone
>>>
>>> On Nov 15, 2013, at 8:36 AM, Ian Clelland <ic...@chromium.org>
>>>>
>>> wrote:
>>>
>>>>
>>>> I've created CB-5403 as the über-ticket for this; various bits of it are
>>>>
>>> in
>>>
>>>> subtasks.
>>>>
>>>> Once I have the tests and the JS updated, then platform owners can take
>>>> a
>>>> look and see whether they have anything to do to support their own
>>>>
>>> schemes.
>>>
>>>>
>>>> As general guidelines, I would say:
>>>>
>>>> * If you can intercept filesystem://* URLs in native code, and return
>>>> arbitrary responses, then do it! It will let your platform support new
>>>> filesystems in the future as we come up with them. Add a couple of
>>>>
>>> subtasks
>>>
>>>> for CB-5403 for your platform and go.
>>>>
>>>> * If you can't do that, but you *can* provide access to things like
>>>> media
>>>> through special urls, then try that! You should be able to create a
>>>> FileSystem object that uses that URL as its root, and everything should
>>>> work. Add a subtask for that, and see what you can do.
>>>>
>>>> * If you can't do that either, and you just want to stick with the
>>>>
>>> file:///
>>>
>>>> urls that we are currently using, then don't do anything; nothing should
>>>> change for you.
>>>>
>>>>
>>>>
>>>> On Fri, Nov 15, 2013 at 10:36 AM, Ian Clelland <iclelland@chromium.org
>>>> wrote:
>>>>
>>>>
>>>>> On Fri, Nov 15, 2013 at 10:03 AM, Marcel Kinard <cmarcelk@gmail.com
>>>>>
>>>> wrote:
>>>>
>>>>>
>>>>> Ian, will there be changes that app developers will need to do? If so,
>>>>>> those should be clearly documented in a migration guide. If not, it
>>>>>>
>>>>> sounds
>>>
>>>> like the [improved] tests should pass on both the old and new versions,
>>>>>> which would be sweet.
>>>>>>
>>>>>
>>>>> The filesystem URLs themselves will change as a result of this. The
>>>>>
>>>> tests
>>>
>>>> should pass on both old and new versions, but the tests mint their own
>>>>>
>>>> URLs
>>>
>>>> for testing against -- each version is internally consistent, but if an
>>>>>
>>>> app
>>>
>>>> is saving internal URLs and tries to dereference an old (file:///) url
>>>>>
>>>> with
>>>
>>>> the new plugin, it will probably not resolve.
>>>>>
>>>>> Maybe there's a transition path here -- it might be possible to allow
>>>>> window.resolveLocalFileSystemURL to access files with the old URL, but
>>>>> never actually generate URLs.
>>>>>
>>>>> That would mean that
>>>>>
>>>>> (resolveLocalFileSystemURL(a)).toURL() !== a,
>>>>>
>>>>> but I don't think that we depend on that as an identity.
>>>>>
>>>>> Is there work which needs to be done on the other platforms (BB, WP8,
>>>>>
>>>>>> Win8, FFOS, etc) so that those platforms don't get left behind? If so,
>>>>>> would it make sense to reach out to those platform owners and at least
>>>>>>
>>>>> get
>>>
>>>> Jira items created with some notes?
>>>>>>
>>>>>
>>>>> Yes. I'm going to create a ticket for this, and we can add
>>>>> platform-specific tasks to it.
>>>>>
>>>>> Other platforms shouldn't *have* to do anything, and some platforms
>>>>> *can't* do anything, because they cannot access anything other than
>>>>> file:/// urls anyway.
>>>>>
>>>>> However, if the other platforms want to get in on the goodness, and
>>>>>
>>>> start
>>>
>>>> supporting their own URL schemes (I think the BB folks can use something
>>>>> like local:// for their filesystem access), then that'll be the place
>>>>> to
>>>>> keep everything organized.
>>>>>
>>>>> I'll post back once I have that issue set up.
>>>>>
>>>>> Ian
>>>>>
>>>>>
>>>>> Sounds like you've built some very significant improvements. Thanks!
>>>>>>
>>>>>> On Nov 15, 2013, at 9:29 AM, Ian Clelland <ic...@chromium.org>
>>>>>>>
>>>>>> wrote:
>>>
>>>>
>>>>>>> After working with the File plugin for a week or so, I've gotten it
>>>>>>>
>>>>>> more or
>>>>>>
>>>>>>> less where I want it to be on iOS, and am working through Android.
>>>>>>>
>>>>>> Looking
>>>>>>
>>>>>>> at the end result, though, I expect that over 90% of the code has
>>>>>>> been
>>>>>>> touched in some way. (It's a huge diff.)
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>
>>
>
Re: Transitioning to a better File API implementation
Posted by Michael Gauthier <mi...@silverorange.com>.
Ian,
With the new URLs will it be possible to write to both /Documents and
/Library for iOS apps? In the old version, the filesystem root resolved
as /Documents but it was possible to get to /Library by navigating the
the parent dir.
Cheers,
Mike
On 2013-11-15 15:19, Ian Clelland wrote:
> On Fri, Nov 15, 2013 at 1:40 PM, purplecabbage <pu...@gmail.com>wrote:
>
>> Considering the magnitude of the changes I would have expected that this
>> was just a new file plugin. The previous version was based on a spec, and
>> if we are deviating from it we should probably maintain both, and possibly
>> even make a recommendation to the w3c.
>> I hope we at least do a major version update for this if APIs have changed.
>>
>
> Yes, definitely a major version bump, at least so that devs aren't caught
> by the URL-format-change.
>
> There aren't any changes to external APIs; I've been very careful to
> maintain conformance with the existing tests, except where those tests have
> been for undocumented implementation details. The only app-visible change
> is that URLs returned by the .toURL() method will look like
>
> filesystem://localhost/persistent/my/file.jpg
>
> rather than
>
> file:///var/mobile/Applications/<app id>/Documents/my/file.jpg
>
> We are still following the spec -- in fact, this brings us closer in line
> with the W3C File API spec, on these points:
>
> http://www.w3.org/TR/file-system-api/#widl-Entry-fullPath -- The fullPath
> of an entry should be the path from the root (of the HTML filesystem, not
> the underlying device file system).
>
> http://www.w3.org/TR/file-system-api/#widl-Entry-toURL-DOMString -- This is
> a note, rather than a requirement, but a filesystem url scheme is mentioned
> there.
>
> We're considering extending the spec in one way, which should be compatible
> with the spirit of the spec, and backwards-compatible with the previous
> API. That is the ability to declare new filesystem types, beyond
> "temporary" and "persistent", so that we can access things like device
> media and app bundle assets using the same APIs that we use for accessing
> user-defined files. If we're happy with the way that works, then we should
> definitely propose it for inclusion in the standard (not the specific
> filesystems, perhaps, but the mechanism for requesting and interacting with
> them)
>
> Ian
>
>
>>
>> Sent from my iPhone
>>
>>> On Nov 15, 2013, at 8:36 AM, Ian Clelland <ic...@chromium.org>
>> wrote:
>>>
>>> I've created CB-5403 as the über-ticket for this; various bits of it are
>> in
>>> subtasks.
>>>
>>> Once I have the tests and the JS updated, then platform owners can take a
>>> look and see whether they have anything to do to support their own
>> schemes.
>>>
>>> As general guidelines, I would say:
>>>
>>> * If you can intercept filesystem://* URLs in native code, and return
>>> arbitrary responses, then do it! It will let your platform support new
>>> filesystems in the future as we come up with them. Add a couple of
>> subtasks
>>> for CB-5403 for your platform and go.
>>>
>>> * If you can't do that, but you *can* provide access to things like media
>>> through special urls, then try that! You should be able to create a
>>> FileSystem object that uses that URL as its root, and everything should
>>> work. Add a subtask for that, and see what you can do.
>>>
>>> * If you can't do that either, and you just want to stick with the
>> file:///
>>> urls that we are currently using, then don't do anything; nothing should
>>> change for you.
>>>
>>>
>>>
>>> On Fri, Nov 15, 2013 at 10:36 AM, Ian Clelland <iclelland@chromium.org
>>> wrote:
>>>
>>>>
>>>> On Fri, Nov 15, 2013 at 10:03 AM, Marcel Kinard <cmarcelk@gmail.com
>>> wrote:
>>>>
>>>>> Ian, will there be changes that app developers will need to do? If so,
>>>>> those should be clearly documented in a migration guide. If not, it
>> sounds
>>>>> like the [improved] tests should pass on both the old and new versions,
>>>>> which would be sweet.
>>>>
>>>> The filesystem URLs themselves will change as a result of this. The
>> tests
>>>> should pass on both old and new versions, but the tests mint their own
>> URLs
>>>> for testing against -- each version is internally consistent, but if an
>> app
>>>> is saving internal URLs and tries to dereference an old (file:///) url
>> with
>>>> the new plugin, it will probably not resolve.
>>>>
>>>> Maybe there's a transition path here -- it might be possible to allow
>>>> window.resolveLocalFileSystemURL to access files with the old URL, but
>>>> never actually generate URLs.
>>>>
>>>> That would mean that
>>>>
>>>> (resolveLocalFileSystemURL(a)).toURL() !== a,
>>>>
>>>> but I don't think that we depend on that as an identity.
>>>>
>>>> Is there work which needs to be done on the other platforms (BB, WP8,
>>>>> Win8, FFOS, etc) so that those platforms don't get left behind? If so,
>>>>> would it make sense to reach out to those platform owners and at least
>> get
>>>>> Jira items created with some notes?
>>>>
>>>> Yes. I'm going to create a ticket for this, and we can add
>>>> platform-specific tasks to it.
>>>>
>>>> Other platforms shouldn't *have* to do anything, and some platforms
>>>> *can't* do anything, because they cannot access anything other than
>>>> file:/// urls anyway.
>>>>
>>>> However, if the other platforms want to get in on the goodness, and
>> start
>>>> supporting their own URL schemes (I think the BB folks can use something
>>>> like local:// for their filesystem access), then that'll be the place to
>>>> keep everything organized.
>>>>
>>>> I'll post back once I have that issue set up.
>>>>
>>>> Ian
>>>>
>>>>
>>>>> Sounds like you've built some very significant improvements. Thanks!
>>>>>
>>>>>> On Nov 15, 2013, at 9:29 AM, Ian Clelland <ic...@chromium.org>
>> wrote:
>>>>>>
>>>>>> After working with the File plugin for a week or so, I've gotten it
>>>>> more or
>>>>>> less where I want it to be on iOS, and am working through Android.
>>>>> Looking
>>>>>> at the end result, though, I expect that over 90% of the code has been
>>>>>> touched in some way. (It's a huge diff.)
>>>>
>>>>
>>
>
Re: Transitioning to a better File API implementation
Posted by Brian LeRoux <b...@brian.io>.
Awesome stuff Ian. Our community is going to love this.
On Fri, Nov 15, 2013 at 11:19 AM, Ian Clelland <ic...@chromium.org>wrote:
> On Fri, Nov 15, 2013 at 1:40 PM, purplecabbage <purplecabbage@gmail.com
> >wrote:
>
> > Considering the magnitude of the changes I would have expected that this
> > was just a new file plugin. The previous version was based on a spec, and
> > if we are deviating from it we should probably maintain both, and
> possibly
> > even make a recommendation to the w3c.
> > I hope we at least do a major version update for this if APIs have
> changed.
> >
>
> Yes, definitely a major version bump, at least so that devs aren't caught
> by the URL-format-change.
>
> There aren't any changes to external APIs; I've been very careful to
> maintain conformance with the existing tests, except where those tests have
> been for undocumented implementation details. The only app-visible change
> is that URLs returned by the .toURL() method will look like
>
> filesystem://localhost/persistent/my/file.jpg
>
> rather than
>
> file:///var/mobile/Applications/<app id>/Documents/my/file.jpg
>
> We are still following the spec -- in fact, this brings us closer in line
> with the W3C File API spec, on these points:
>
> http://www.w3.org/TR/file-system-api/#widl-Entry-fullPath -- The fullPath
> of an entry should be the path from the root (of the HTML filesystem, not
> the underlying device file system).
>
> http://www.w3.org/TR/file-system-api/#widl-Entry-toURL-DOMString -- This
> is
> a note, rather than a requirement, but a filesystem url scheme is mentioned
> there.
>
> We're considering extending the spec in one way, which should be compatible
> with the spirit of the spec, and backwards-compatible with the previous
> API. That is the ability to declare new filesystem types, beyond
> "temporary" and "persistent", so that we can access things like device
> media and app bundle assets using the same APIs that we use for accessing
> user-defined files. If we're happy with the way that works, then we should
> definitely propose it for inclusion in the standard (not the specific
> filesystems, perhaps, but the mechanism for requesting and interacting with
> them)
>
> Ian
>
>
> >
> > Sent from my iPhone
> >
> > > On Nov 15, 2013, at 8:36 AM, Ian Clelland <ic...@chromium.org>
> > wrote:
> > >
> > > I've created CB-5403 as the über-ticket for this; various bits of it
> are
> > in
> > > subtasks.
> > >
> > > Once I have the tests and the JS updated, then platform owners can
> take a
> > > look and see whether they have anything to do to support their own
> > schemes.
> > >
> > > As general guidelines, I would say:
> > >
> > > * If you can intercept filesystem://* URLs in native code, and return
> > > arbitrary responses, then do it! It will let your platform support new
> > > filesystems in the future as we come up with them. Add a couple of
> > subtasks
> > > for CB-5403 for your platform and go.
> > >
> > > * If you can't do that, but you *can* provide access to things like
> media
> > > through special urls, then try that! You should be able to create a
> > > FileSystem object that uses that URL as its root, and everything should
> > > work. Add a subtask for that, and see what you can do.
> > >
> > > * If you can't do that either, and you just want to stick with the
> > file:///
> > > urls that we are currently using, then don't do anything; nothing
> should
> > > change for you.
> > >
> > >
> > >
> > > On Fri, Nov 15, 2013 at 10:36 AM, Ian Clelland <iclelland@chromium.org
> > >wrote:
> > >
> > >>
> > >> On Fri, Nov 15, 2013 at 10:03 AM, Marcel Kinard <cmarcelk@gmail.com
> > >wrote:
> > >>
> > >>> Ian, will there be changes that app developers will need to do? If
> so,
> > >>> those should be clearly documented in a migration guide. If not, it
> > sounds
> > >>> like the [improved] tests should pass on both the old and new
> versions,
> > >>> which would be sweet.
> > >>
> > >> The filesystem URLs themselves will change as a result of this. The
> > tests
> > >> should pass on both old and new versions, but the tests mint their own
> > URLs
> > >> for testing against -- each version is internally consistent, but if
> an
> > app
> > >> is saving internal URLs and tries to dereference an old (file:///) url
> > with
> > >> the new plugin, it will probably not resolve.
> > >>
> > >> Maybe there's a transition path here -- it might be possible to allow
> > >> window.resolveLocalFileSystemURL to access files with the old URL, but
> > >> never actually generate URLs.
> > >>
> > >> That would mean that
> > >>
> > >> (resolveLocalFileSystemURL(a)).toURL() !== a,
> > >>
> > >> but I don't think that we depend on that as an identity.
> > >>
> > >> Is there work which needs to be done on the other platforms (BB, WP8,
> > >>> Win8, FFOS, etc) so that those platforms don't get left behind? If
> so,
> > >>> would it make sense to reach out to those platform owners and at
> least
> > get
> > >>> Jira items created with some notes?
> > >>
> > >> Yes. I'm going to create a ticket for this, and we can add
> > >> platform-specific tasks to it.
> > >>
> > >> Other platforms shouldn't *have* to do anything, and some platforms
> > >> *can't* do anything, because they cannot access anything other than
> > >> file:/// urls anyway.
> > >>
> > >> However, if the other platforms want to get in on the goodness, and
> > start
> > >> supporting their own URL schemes (I think the BB folks can use
> something
> > >> like local:// for their filesystem access), then that'll be the place
> to
> > >> keep everything organized.
> > >>
> > >> I'll post back once I have that issue set up.
> > >>
> > >> Ian
> > >>
> > >>
> > >>> Sounds like you've built some very significant improvements. Thanks!
> > >>>
> > >>>> On Nov 15, 2013, at 9:29 AM, Ian Clelland <ic...@chromium.org>
> > wrote:
> > >>>>
> > >>>> After working with the File plugin for a week or so, I've gotten it
> > >>> more or
> > >>>> less where I want it to be on iOS, and am working through Android.
> > >>> Looking
> > >>>> at the end result, though, I expect that over 90% of the code has
> been
> > >>>> touched in some way. (It's a huge diff.)
> > >>
> > >>
> >
>
Re: Transitioning to a better File API implementation
Posted by Ian Clelland <ic...@chromium.org>.
On Fri, Nov 15, 2013 at 1:40 PM, purplecabbage <pu...@gmail.com>wrote:
> Considering the magnitude of the changes I would have expected that this
> was just a new file plugin. The previous version was based on a spec, and
> if we are deviating from it we should probably maintain both, and possibly
> even make a recommendation to the w3c.
> I hope we at least do a major version update for this if APIs have changed.
>
Yes, definitely a major version bump, at least so that devs aren't caught
by the URL-format-change.
There aren't any changes to external APIs; I've been very careful to
maintain conformance with the existing tests, except where those tests have
been for undocumented implementation details. The only app-visible change
is that URLs returned by the .toURL() method will look like
filesystem://localhost/persistent/my/file.jpg
rather than
file:///var/mobile/Applications/<app id>/Documents/my/file.jpg
We are still following the spec -- in fact, this brings us closer in line
with the W3C File API spec, on these points:
http://www.w3.org/TR/file-system-api/#widl-Entry-fullPath -- The fullPath
of an entry should be the path from the root (of the HTML filesystem, not
the underlying device file system).
http://www.w3.org/TR/file-system-api/#widl-Entry-toURL-DOMString -- This is
a note, rather than a requirement, but a filesystem url scheme is mentioned
there.
We're considering extending the spec in one way, which should be compatible
with the spirit of the spec, and backwards-compatible with the previous
API. That is the ability to declare new filesystem types, beyond
"temporary" and "persistent", so that we can access things like device
media and app bundle assets using the same APIs that we use for accessing
user-defined files. If we're happy with the way that works, then we should
definitely propose it for inclusion in the standard (not the specific
filesystems, perhaps, but the mechanism for requesting and interacting with
them)
Ian
>
> Sent from my iPhone
>
> > On Nov 15, 2013, at 8:36 AM, Ian Clelland <ic...@chromium.org>
> wrote:
> >
> > I've created CB-5403 as the über-ticket for this; various bits of it are
> in
> > subtasks.
> >
> > Once I have the tests and the JS updated, then platform owners can take a
> > look and see whether they have anything to do to support their own
> schemes.
> >
> > As general guidelines, I would say:
> >
> > * If you can intercept filesystem://* URLs in native code, and return
> > arbitrary responses, then do it! It will let your platform support new
> > filesystems in the future as we come up with them. Add a couple of
> subtasks
> > for CB-5403 for your platform and go.
> >
> > * If you can't do that, but you *can* provide access to things like media
> > through special urls, then try that! You should be able to create a
> > FileSystem object that uses that URL as its root, and everything should
> > work. Add a subtask for that, and see what you can do.
> >
> > * If you can't do that either, and you just want to stick with the
> file:///
> > urls that we are currently using, then don't do anything; nothing should
> > change for you.
> >
> >
> >
> > On Fri, Nov 15, 2013 at 10:36 AM, Ian Clelland <iclelland@chromium.org
> >wrote:
> >
> >>
> >> On Fri, Nov 15, 2013 at 10:03 AM, Marcel Kinard <cmarcelk@gmail.com
> >wrote:
> >>
> >>> Ian, will there be changes that app developers will need to do? If so,
> >>> those should be clearly documented in a migration guide. If not, it
> sounds
> >>> like the [improved] tests should pass on both the old and new versions,
> >>> which would be sweet.
> >>
> >> The filesystem URLs themselves will change as a result of this. The
> tests
> >> should pass on both old and new versions, but the tests mint their own
> URLs
> >> for testing against -- each version is internally consistent, but if an
> app
> >> is saving internal URLs and tries to dereference an old (file:///) url
> with
> >> the new plugin, it will probably not resolve.
> >>
> >> Maybe there's a transition path here -- it might be possible to allow
> >> window.resolveLocalFileSystemURL to access files with the old URL, but
> >> never actually generate URLs.
> >>
> >> That would mean that
> >>
> >> (resolveLocalFileSystemURL(a)).toURL() !== a,
> >>
> >> but I don't think that we depend on that as an identity.
> >>
> >> Is there work which needs to be done on the other platforms (BB, WP8,
> >>> Win8, FFOS, etc) so that those platforms don't get left behind? If so,
> >>> would it make sense to reach out to those platform owners and at least
> get
> >>> Jira items created with some notes?
> >>
> >> Yes. I'm going to create a ticket for this, and we can add
> >> platform-specific tasks to it.
> >>
> >> Other platforms shouldn't *have* to do anything, and some platforms
> >> *can't* do anything, because they cannot access anything other than
> >> file:/// urls anyway.
> >>
> >> However, if the other platforms want to get in on the goodness, and
> start
> >> supporting their own URL schemes (I think the BB folks can use something
> >> like local:// for their filesystem access), then that'll be the place to
> >> keep everything organized.
> >>
> >> I'll post back once I have that issue set up.
> >>
> >> Ian
> >>
> >>
> >>> Sounds like you've built some very significant improvements. Thanks!
> >>>
> >>>> On Nov 15, 2013, at 9:29 AM, Ian Clelland <ic...@chromium.org>
> wrote:
> >>>>
> >>>> After working with the File plugin for a week or so, I've gotten it
> >>> more or
> >>>> less where I want it to be on iOS, and am working through Android.
> >>> Looking
> >>>> at the end result, though, I expect that over 90% of the code has been
> >>>> touched in some way. (It's a huge diff.)
> >>
> >>
>
Re: Transitioning to a better File API implementation
Posted by purplecabbage <pu...@gmail.com>.
Considering the magnitude of the changes I would have expected that this was just a new file plugin. The previous version was based on a spec, and if we are deviating from it we should probably maintain both, and possibly even make a recommendation to the w3c.
I hope we at least do a major version update for this if APIs have changed.
Sent from my iPhone
> On Nov 15, 2013, at 8:36 AM, Ian Clelland <ic...@chromium.org> wrote:
>
> I've created CB-5403 as the über-ticket for this; various bits of it are in
> subtasks.
>
> Once I have the tests and the JS updated, then platform owners can take a
> look and see whether they have anything to do to support their own schemes.
>
> As general guidelines, I would say:
>
> * If you can intercept filesystem://* URLs in native code, and return
> arbitrary responses, then do it! It will let your platform support new
> filesystems in the future as we come up with them. Add a couple of subtasks
> for CB-5403 for your platform and go.
>
> * If you can't do that, but you *can* provide access to things like media
> through special urls, then try that! You should be able to create a
> FileSystem object that uses that URL as its root, and everything should
> work. Add a subtask for that, and see what you can do.
>
> * If you can't do that either, and you just want to stick with the file:///
> urls that we are currently using, then don't do anything; nothing should
> change for you.
>
>
>
> On Fri, Nov 15, 2013 at 10:36 AM, Ian Clelland <ic...@chromium.org>wrote:
>
>>
>> On Fri, Nov 15, 2013 at 10:03 AM, Marcel Kinard <cm...@gmail.com>wrote:
>>
>>> Ian, will there be changes that app developers will need to do? If so,
>>> those should be clearly documented in a migration guide. If not, it sounds
>>> like the [improved] tests should pass on both the old and new versions,
>>> which would be sweet.
>>
>> The filesystem URLs themselves will change as a result of this. The tests
>> should pass on both old and new versions, but the tests mint their own URLs
>> for testing against -- each version is internally consistent, but if an app
>> is saving internal URLs and tries to dereference an old (file:///) url with
>> the new plugin, it will probably not resolve.
>>
>> Maybe there's a transition path here -- it might be possible to allow
>> window.resolveLocalFileSystemURL to access files with the old URL, but
>> never actually generate URLs.
>>
>> That would mean that
>>
>> (resolveLocalFileSystemURL(a)).toURL() !== a,
>>
>> but I don't think that we depend on that as an identity.
>>
>> Is there work which needs to be done on the other platforms (BB, WP8,
>>> Win8, FFOS, etc) so that those platforms don't get left behind? If so,
>>> would it make sense to reach out to those platform owners and at least get
>>> Jira items created with some notes?
>>
>> Yes. I'm going to create a ticket for this, and we can add
>> platform-specific tasks to it.
>>
>> Other platforms shouldn't *have* to do anything, and some platforms
>> *can't* do anything, because they cannot access anything other than
>> file:/// urls anyway.
>>
>> However, if the other platforms want to get in on the goodness, and start
>> supporting their own URL schemes (I think the BB folks can use something
>> like local:// for their filesystem access), then that'll be the place to
>> keep everything organized.
>>
>> I'll post back once I have that issue set up.
>>
>> Ian
>>
>>
>>> Sounds like you've built some very significant improvements. Thanks!
>>>
>>>> On Nov 15, 2013, at 9:29 AM, Ian Clelland <ic...@chromium.org> wrote:
>>>>
>>>> After working with the File plugin for a week or so, I've gotten it
>>> more or
>>>> less where I want it to be on iOS, and am working through Android.
>>> Looking
>>>> at the end result, though, I expect that over 90% of the code has been
>>>> touched in some way. (It's a huge diff.)
>>
>>
Re: Transitioning to a better File API implementation
Posted by Ian Clelland <ic...@chromium.org>.
I've created CB-5403 as the über-ticket for this; various bits of it are in
subtasks.
Once I have the tests and the JS updated, then platform owners can take a
look and see whether they have anything to do to support their own schemes.
As general guidelines, I would say:
* If you can intercept filesystem://* URLs in native code, and return
arbitrary responses, then do it! It will let your platform support new
filesystems in the future as we come up with them. Add a couple of subtasks
for CB-5403 for your platform and go.
* If you can't do that, but you *can* provide access to things like media
through special urls, then try that! You should be able to create a
FileSystem object that uses that URL as its root, and everything should
work. Add a subtask for that, and see what you can do.
* If you can't do that either, and you just want to stick with the file:///
urls that we are currently using, then don't do anything; nothing should
change for you.
On Fri, Nov 15, 2013 at 10:36 AM, Ian Clelland <ic...@chromium.org>wrote:
>
> On Fri, Nov 15, 2013 at 10:03 AM, Marcel Kinard <cm...@gmail.com>wrote:
>
>> Ian, will there be changes that app developers will need to do? If so,
>> those should be clearly documented in a migration guide. If not, it sounds
>> like the [improved] tests should pass on both the old and new versions,
>> which would be sweet.
>>
>
> The filesystem URLs themselves will change as a result of this. The tests
> should pass on both old and new versions, but the tests mint their own URLs
> for testing against -- each version is internally consistent, but if an app
> is saving internal URLs and tries to dereference an old (file:///) url with
> the new plugin, it will probably not resolve.
>
> Maybe there's a transition path here -- it might be possible to allow
> window.resolveLocalFileSystemURL to access files with the old URL, but
> never actually generate URLs.
>
> That would mean that
>
> (resolveLocalFileSystemURL(a)).toURL() !== a,
>
> but I don't think that we depend on that as an identity.
>
> Is there work which needs to be done on the other platforms (BB, WP8,
>> Win8, FFOS, etc) so that those platforms don't get left behind? If so,
>> would it make sense to reach out to those platform owners and at least get
>> Jira items created with some notes?
>>
>
> Yes. I'm going to create a ticket for this, and we can add
> platform-specific tasks to it.
>
> Other platforms shouldn't *have* to do anything, and some platforms
> *can't* do anything, because they cannot access anything other than
> file:/// urls anyway.
>
> However, if the other platforms want to get in on the goodness, and start
> supporting their own URL schemes (I think the BB folks can use something
> like local:// for their filesystem access), then that'll be the place to
> keep everything organized.
>
> I'll post back once I have that issue set up.
>
> Ian
>
>
>> Sounds like you've built some very significant improvements. Thanks!
>>
>> On Nov 15, 2013, at 9:29 AM, Ian Clelland <ic...@chromium.org> wrote:
>>
>> > After working with the File plugin for a week or so, I've gotten it
>> more or
>> > less where I want it to be on iOS, and am working through Android.
>> Looking
>> > at the end result, though, I expect that over 90% of the code has been
>> > touched in some way. (It's a huge diff.)
>>
>
>
Re: Transitioning to a better File API implementation
Posted by Ian Clelland <ic...@chromium.org>.
On Fri, Nov 15, 2013 at 10:03 AM, Marcel Kinard <cm...@gmail.com> wrote:
> Ian, will there be changes that app developers will need to do? If so,
> those should be clearly documented in a migration guide. If not, it sounds
> like the [improved] tests should pass on both the old and new versions,
> which would be sweet.
>
The filesystem URLs themselves will change as a result of this. The tests
should pass on both old and new versions, but the tests mint their own URLs
for testing against -- each version is internally consistent, but if an app
is saving internal URLs and tries to dereference an old (file:///) url with
the new plugin, it will probably not resolve.
Maybe there's a transition path here -- it might be possible to allow
window.resolveLocalFileSystemURL to access files with the old URL, but
never actually generate URLs.
That would mean that
(resolveLocalFileSystemURL(a)).toURL() !== a,
but I don't think that we depend on that as an identity.
Is there work which needs to be done on the other platforms (BB, WP8, Win8,
> FFOS, etc) so that those platforms don't get left behind? If so, would it
> make sense to reach out to those platform owners and at least get Jira
> items created with some notes?
>
Yes. I'm going to create a ticket for this, and we can add
platform-specific tasks to it.
Other platforms shouldn't *have* to do anything, and some platforms *can't*
do anything, because they cannot access anything other than file:/// urls
anyway.
However, if the other platforms want to get in on the goodness, and start
supporting their own URL schemes (I think the BB folks can use something
like local:// for their filesystem access), then that'll be the place to
keep everything organized.
I'll post back once I have that issue set up.
Ian
> Sounds like you've built some very significant improvements. Thanks!
>
> On Nov 15, 2013, at 9:29 AM, Ian Clelland <ic...@chromium.org> wrote:
>
> > After working with the File plugin for a week or so, I've gotten it more
> or
> > less where I want it to be on iOS, and am working through Android.
> Looking
> > at the end result, though, I expect that over 90% of the code has been
> > touched in some way. (It's a huge diff.)
>
Re: Transitioning to a better File API implementation
Posted by Marcel Kinard <cm...@gmail.com>.
Ian, will there be changes that app developers will need to do? If so, those should be clearly documented in a migration guide. If not, it sounds like the [improved] tests should pass on both the old and new versions, which would be sweet.
Is there work which needs to be done on the other platforms (BB, WP8, Win8, FFOS, etc) so that those platforms don't get left behind? If so, would it make sense to reach out to those platform owners and at least get Jira items created with some notes?
Sounds like you've built some very significant improvements. Thanks!
On Nov 15, 2013, at 9:29 AM, Ian Clelland <ic...@chromium.org> wrote:
> After working with the File plugin for a week or so, I've gotten it more or
> less where I want it to be on iOS, and am working through Android. Looking
> at the end result, though, I expect that over 90% of the code has been
> touched in some way. (It's a huge diff.)