Looking at the new File API pragmatically

[Ian Clelland <ic...@chromium.org>]

The new File plugin has been in the wild now for almost two months now, and
it seems to be seeing a lot of real-world use (bug reports are coming in to
JIRA almost daily :) )

There have been a lot of bugs fixed, and a fair number of people guided
towards new ways of doing things, but there are definitely some recurring
themes, and I think, looking at things pragmatically, that there are some
things that we can do to make the whole file experience better for
developers.


---- ONE ----

The first thing has to do with .toURL(), and it's new baby brother,
.toNativeURL(). Originally, the ideal was that .toURL() would always return
a URL with the prefix "cdvfile://", but this has caused problems with a
number of use cases, all involving passing the URL to the webview as a
"src" attribute. The Android code for resolving "src" attributes generally
doesn't pass the URLs back to the application, and so a real file:/// URL
is required.

Because of this, I introduced .toNativeURL, which will try to return a
file:/// URL, or a content:// URL, and will only return a cdvfile:// URL if
it has no other alternative. I've been providing guidance to developers to
try to use that if they want to use File-plugin-files in video or audio
tags, or even for img tags on Android <= 3.0

This is ugly, though, and is going to get worse over time, and become a
division between Cordova and any platforms which actually implement the
File API correctly. I'd like to propose switching the behaviour of
.toURL(), to match .toNativeURL -- returning a webview-usable URL by
default -- and implementing some other method or property to get the CDV
URL when it's necessary.

---- TWO ----

The second issue has to do with the filesystem locations which are made
available to Cordova apps. The File plugin provides two sandboxed
filesystem locations, one for persistent files, and one for temporary
files. Files which reside outside of those two locations are not accessible
to the plugin, by default.

(For comparison, the old version of File would happily access files
anywhere on the device, even well outside of the claimed 'roots')

This generally works, doesn't cause problems with applications using the
File plugin, and is (I think) still a good idea. It breaks down, however,
when other plugins come in to the mix. Before File was released, I made
changes to FileTransfer, and since, I have changed Camera, and been working
on MediaCapture -- all to ensure that they play within File's sandboxes.
MediaCapture, though, is more difficult, because it's the OS that is
creating the files, and it will create them where it wants to, not where we
think it should.

I revived the file-system-roots plugin to take care of this problem -- when
installed, it registers several additional file systems, covering other
areas of the device, allowing access to places like "SD Card storage",
"external cache", "internal cache", etc (see the plugin README for all of
the details). It's even possible to register the "root" filesystem, which
re-enables access to the *entire* device. Adding this plugin has worked for
basically everyone who's had a problem with file locations, but it seems
wrong to have to recommend an outside-of-core plugin to fix a problem with
core plugins.

The question here, I guess -- because a number of people have required this
feature, and there may be additional third-party plugins out there that
will require it as well -- is whether it makes sense to include this
functionality in the core File plugin, or whether to just continue to
recommend it to people whose apps write files in locations outside of the
standard persistent/temporary file systems.

Thoughts, opinions, flames -- all welcomed. Thanks for reading this far :)

Ian

Re: Looking at the new File API pragmatically

[Michal Mocny <mm...@chromium.org>]

On Mon, Mar 31, 2014 at 4:33 PM, Ian Clelland <ic...@chromium.org>wrote:

> On Mon, Mar 31, 2014 at 4:20 PM, Michal Mocny <mm...@chromium.org> wrote:
>
> > On Mon, Mar 31, 2014 at 3:39 PM, Ian Clelland <iclelland@chromium.org
> > >wrote:
> > > This is ugly, though, and is going to get worse over time, and become a
> > > division between Cordova and any platforms which actually implement the
> > > File API correctly. I'd like to propose switching the behaviour of
> > > .toURL(), to match .toNativeURL -- returning a webview-usable URL by
> > > default -- and implementing some other method or property to get the
> CDV
> > > URL when it's necessary.
> > >
> >
> > Everything you've said sounds like its all upside to make the switch.  So
> > I'm curious, when would CDV URL be necessary/useful over file/content
> urls?
> >
>
> cdvfile:// URLs would still be necessary when dealing with files that just
> don't *have* an alternate representation. There currently aren't any of
> those, but we could implement virtual file systems entirely inside of a
> plugin, and those would require a cdvfile:// URL to be read.


I'm assuming the toURL change you propose wouldn't prohibit the use case
you mention, since we could still return cdvfile:// if file:// isn't
available.  toNativeURL should still exist as a call and *only* return
native urls, and toURL would allow native or virtual.

So, I'm curious, do we also need a toCdvURL()?  Would you ever want to use
cdvfile:// when a file:// *is* available?


> I think we'd recommend them when saving URLs to persistent storage, if
> there is any chance that the actual files could be moved / migrated, and we
> could hide that from the user by giving them a more abstract identifier
> than one which specifies a physical location.
> cdvfile://localhost/persistent/my/file.txt might be more durable over time
> than file:///data/data/com.company.package/files/my/file.txt, perhaps
> across OS upgrades.
>

That was my suspicion.  I think its not a powerful enough reason to not
make the change you propose.

One option is to support toCdvURL()/toPersistantURL() for the old toURL()
behaviour.  Another is to acknowledge that the next migration will never be
pretty and just address it when it comes (my vote).


>
> >
> >
> > >
> > > ---- TWO ----
> > >
> > > The second issue has to do with the filesystem locations which are made
> > > available to Cordova apps. The File plugin provides two sandboxed
> > > filesystem locations, one for persistent files, and one for temporary
> > > files. Files which reside outside of those two locations are not
> > accessible
> > > to the plugin, by default.
> > >
> > > (For comparison, the old version of File would happily access files
> > > anywhere on the device, even well outside of the claimed 'roots')
> > >
> > > This generally works, doesn't cause problems with applications using
> the
> > > File plugin, and is (I think) still a good idea. It breaks down,
> however,
> > > when other plugins come in to the mix. Before File was released, I made
> > > changes to FileTransfer, and since, I have changed Camera, and been
> > working
> > > on MediaCapture -- all to ensure that they play within File's
> sandboxes.
> > > MediaCapture, though, is more difficult, because it's the OS that is
> > > creating the files, and it will create them where it wants to, not
> where
> > we
> > > think it should.
> > >
> > > I revived the file-system-roots plugin to take care of this problem --
> > when
> > > installed, it registers several additional file systems, covering other
> > > areas of the device, allowing access to places like "SD Card storage",
> > > "external cache", "internal cache", etc (see the plugin README for all
> of
> > >
> >
> > FYI:
> > https://github.com/apache/cordova-plugins/tree/master/file-system-roots
> >
> >
> > > the details). It's even possible to register the "root" filesystem,
> which
> > > re-enables access to the *entire* device. Adding this plugin has worked
> > for
> > > basically everyone who's had a problem with file locations, but it
> seems
> > > wrong to have to recommend an outside-of-core plugin to fix a problem
> > with
> > > core plugins.
> > >
> > > The question here, I guess -- because a number of people have required
> > this
> > > feature, and there may be additional third-party plugins out there that
> > > will require it as well -- is whether it makes sense to include this
> > > functionality in the core File plugin, or whether to just continue to
> > > recommend it to people whose apps write files in locations outside of
> the
> > > standard persistent/temporary file systems.
> >
> >
> > > Thoughts, opinions, flames -- all welcomed. Thanks for reading this far
> > :)
> > >
> >
> > I do like adding these by default.
> >
> > Most devs will follow your guidance anyway by use of PERSISTENT and
> > TEMPORARY fs type, and will never "accidently" reach into these other fs
> > roots without meaning to.  I like when hybrid apps can do just as much as
> > native equivalents without having to try too hard.
> >
> >
> > > Ian
> > >
> >
>

Re: Looking at the new File API pragmatically

[David Kemp <dr...@chromium.org>]

So in summary....
 make toURL() return the same thing that toNativeURL() does now.
 do nothing to toNativeURL()
 maybe later add a toCdvURL()

If thats correct, then +1




On Tue, Apr 1, 2014 at 4:50 AM, Anis KADRI <an...@gmail.com> wrote:

> On Tue, Apr 1, 2014 at 3:28 AM, Ian Clelland <ic...@chromium.org>
> wrote:
>
> > On Mon, Mar 31, 2014 at 4:33 PM, Ian Clelland <iclelland@chromium.org
> > >wrote:
> >
> > > On Mon, Mar 31, 2014 at 4:20 PM, Michal Mocny <mm...@chromium.org>
> > wrote:
> > >
> > >> On Mon, Mar 31, 2014 at 3:39 PM, Ian Clelland <iclelland@chromium.org
> > >> >wrote:
> > >> > This is ugly, though, and is going to get worse over time, and
> become
> > a
> > >> > division between Cordova and any platforms which actually implement
> > the
> > >> > File API correctly. I'd like to propose switching the behaviour of
> > >> > .toURL(), to match .toNativeURL -- returning a webview-usable URL by
> > >> > default -- and implementing some other method or property to get the
> > CDV
> > >> > URL when it's necessary.
> > >> >
> > >>
> > >> Everything you've said sounds like its all upside to make the switch.
> >  So
> > >> I'm curious, when would CDV URL be necessary/useful over file/content
> > >> urls?
> > >>
> > >
> > > cdvfile:// URLs would still be necessary when dealing with files that
> > just
> > > don't *have* an alternate representation. There currently aren't any of
> > > those, but we could implement virtual file systems entirely inside of a
> > > plugin, and those would require a cdvfile:// URL to be read.
> > >
> > > I think we'd recommend them when saving URLs to persistent storage, if
> > > there is any chance that the actual files could be moved / migrated,
> and
> > we
> > > could hide that from the user by giving them a more abstract identifier
> > > than one which specifies a physical location.
> > > cdvfile://localhost/persistent/my/file.txt might be more durable over
> > time
> > > than file:///data/data/com.company.package/files/my/file.txt, perhaps
> > > across OS upgrades.
> > >
> >
> > Actually, forget all of that.
> >
> > Your question had me looking for reasons to advocate users using
> cdvfile://
> > URLs, when perhaps none exist. The truth of the matter is this: The
> cdvfile
> > URL has two parts: the filesystem name, and the full path. Those two
> parts
> > form a consistent internal representation for all of the types of file
> that
> > the plugin can handle, and so all of the internal / native bits of the
> file
> > plugin use them almost exclusively. We make sure that every FileEntry and
> > DirectoryEntry has those parts, and we only need to turn them into a URL
> > for passing them across the bridge.
> >
> > One day someone may discover a great reason to use deliberately use
> cdvfile
> > URLs at the application level; until then, they're available, and we can
> > continue to use them internally to simplify the plugin code, enforce the
> > sandboxing, and make everything generally more consistent and efficient,
> > and users shouldn't need to know or care what the URLs in use actually
> are.
> >
>
> I agree with this as long as the URLs are useable in the WebView (as src
> attributes for example). If they're not, I also suggest that we return URLs
> that are useable (file:///, content:/// or whatever) by default.
>
> As for filesystems (temp or persistent), I think most developers will use
> whatever the default is. BUT they should be able to specify where they want
> to store their data if they feel like it without using a third-party
> plugin.
>
>
> >
> > Ian
> >
>

Re: Looking at the new File API pragmatically

[Andrew Grieve <ag...@chromium.org>]

All sounds great Ian. Awesome work with all of this.

That's awesome Bryan!
I think file-system-roots fills a giant need, and would be great if it were
supported everywhere.

With merging file-system-roots in, it'll be important to have file's docs
updated to have links to the iOS / Android guidance pages on where to store
things (as linked to in CB-285)


On Tue, Apr 1, 2014 at 7:54 AM, Bryan Higgins <br...@bryanhiggins.net>wrote:

> I'm in the process of converting the bb10 file plugin to use the exec proxy
> rather than clobbering common js. Once that's done, I'll take a look at
> adding some additional file system roots.
>
>
> On Tue, Apr 1, 2014 at 10:24 AM, Ian Clelland <iclelland@chromium.org
> >wrote:
>
> > On Tue, Apr 1, 2014 at 10:14 AM, Bryan Higgins <bryan@bryanhiggins.net
> > >wrote:
> >
> > > +1 to replacing toURL with toNativeURL behaviour
> > >
> >
> > I think that's what I'll do. I'm also adding a "toInternalURL" method --
> > not because it's necessarily useful for end users, but because we can use
> > it as the single method to construct URLs for transport across the
> bridge.
> >
> >
> > > For accessing the root file system, what I'd personally like is a ROOT
> > > filesystem type for requestFileSystem. If the file-system-roots plugin
> > > provides that I think we should definitely pull it in.
> > >
> >
> > It does provide that; although I haven't enabled it by default. I've been
> > pretty wary of allowing JavaScript to arbitrarily access *everything* on
> > the entire device that is permitted by the OS.
> >
> >
> > >
> > > For the camera and media plugins on BB10 we return a full path relative
> > to
> > > the file system root as the native apps save into default locations. I
> > > wonder if there is an opportunity to add additional roots which map to
> > > default locations for pictures, music and video (maybe also
> documents?).
> > It
> > > seems like that would be super useful for media player apps or
> > downloaders.
> > >
> >
> > That's the whole point of the file-system-roots plugin -- on Android, it
> > adds things like sdcard, external cache, external persistent files,
> > documetns, etc. On iOS, it gives library, documents, cache, bundle, and a
> > couple of non-icloud-synced versions of them. I'd love to get a set of
> > useful BlackBerry file system roots as well.
> >
> > Ian
> >
> > >
> > > On Tue, Apr 1, 2014 at 4:50 AM, Anis KADRI <an...@gmail.com>
> wrote:
> > >
> > > > On Tue, Apr 1, 2014 at 3:28 AM, Ian Clelland <iclelland@chromium.org
> >
> > > > wrote:
> > > >
> > > > > On Mon, Mar 31, 2014 at 4:33 PM, Ian Clelland <
> > iclelland@chromium.org
> > > > > >wrote:
> > > > >
> > > > > > On Mon, Mar 31, 2014 at 4:20 PM, Michal Mocny <
> mmocny@chromium.org
> > >
> > > > > wrote:
> > > > > >
> > > > > >> On Mon, Mar 31, 2014 at 3:39 PM, Ian Clelland <
> > > iclelland@chromium.org
> > > > > >> >wrote:
> > > > > >> > This is ugly, though, and is going to get worse over time, and
> > > > become
> > > > > a
> > > > > >> > division between Cordova and any platforms which actually
> > > implement
> > > > > the
> > > > > >> > File API correctly. I'd like to propose switching the
> behaviour
> > of
> > > > > >> > .toURL(), to match .toNativeURL -- returning a webview-usable
> > URL
> > > by
> > > > > >> > default -- and implementing some other method or property to
> get
> > > the
> > > > > CDV
> > > > > >> > URL when it's necessary.
> > > > > >> >
> > > > > >>
> > > > > >> Everything you've said sounds like its all upside to make the
> > > switch.
> > > > >  So
> > > > > >> I'm curious, when would CDV URL be necessary/useful over
> > > file/content
> > > > > >> urls?
> > > > > >>
> > > > > >
> > > > > > cdvfile:// URLs would still be necessary when dealing with files
> > that
> > > > > just
> > > > > > don't *have* an alternate representation. There currently aren't
> > any
> > > of
> > > > > > those, but we could implement virtual file systems entirely
> inside
> > > of a
> > > > > > plugin, and those would require a cdvfile:// URL to be read.
> > > > > >
> > > > > > I think we'd recommend them when saving URLs to persistent
> storage,
> > > if
> > > > > > there is any chance that the actual files could be moved /
> > migrated,
> > > > and
> > > > > we
> > > > > > could hide that from the user by giving them a more abstract
> > > identifier
> > > > > > than one which specifies a physical location.
> > > > > > cdvfile://localhost/persistent/my/file.txt might be more durable
> > over
> > > > > time
> > > > > > than file:///data/data/com.company.package/files/my/file.txt,
> > perhaps
> > > > > > across OS upgrades.
> > > > > >
> > > > >
> > > > > Actually, forget all of that.
> > > > >
> > > > > Your question had me looking for reasons to advocate users using
> > > > cdvfile://
> > > > > URLs, when perhaps none exist. The truth of the matter is this: The
> > > > cdvfile
> > > > > URL has two parts: the filesystem name, and the full path. Those
> two
> > > > parts
> > > > > form a consistent internal representation for all of the types of
> > file
> > > > that
> > > > > the plugin can handle, and so all of the internal / native bits of
> > the
> > > > file
> > > > > plugin use them almost exclusively. We make sure that every
> FileEntry
> > > and
> > > > > DirectoryEntry has those parts, and we only need to turn them into
> a
> > > URL
> > > > > for passing them across the bridge.
> > > > >
> > > > > One day someone may discover a great reason to use deliberately use
> > > > cdvfile
> > > > > URLs at the application level; until then, they're available, and
> we
> > > can
> > > > > continue to use them internally to simplify the plugin code,
> enforce
> > > the
> > > > > sandboxing, and make everything generally more consistent and
> > > efficient,
> > > > > and users shouldn't need to know or care what the URLs in use
> > actually
> > > > are.
> > > > >
> > > >
> > > > I agree with this as long as the URLs are useable in the WebView (as
> > src
> > > > attributes for example). If they're not, I also suggest that we
> return
> > > URLs
> > > > that are useable (file:///, content:/// or whatever) by default.
> > > >
> > > > As for filesystems (temp or persistent), I think most developers will
> > use
> > > > whatever the default is. BUT they should be able to specify where
> they
> > > want
> > > > to store their data if they feel like it without using a third-party
> > > > plugin.
> > > >
> > > >
> > > > >
> > > > > Ian
> > > > >
> > > >
> > >
> >
>

Re: Looking at the new File API pragmatically

[Bryan Higgins <br...@bryanhiggins.net>]

I'm in the process of converting the bb10 file plugin to use the exec proxy
rather than clobbering common js. Once that's done, I'll take a look at
adding some additional file system roots.


On Tue, Apr 1, 2014 at 10:24 AM, Ian Clelland <ic...@chromium.org>wrote:

> On Tue, Apr 1, 2014 at 10:14 AM, Bryan Higgins <bryan@bryanhiggins.net
> >wrote:
>
> > +1 to replacing toURL with toNativeURL behaviour
> >
>
> I think that's what I'll do. I'm also adding a "toInternalURL" method --
> not because it's necessarily useful for end users, but because we can use
> it as the single method to construct URLs for transport across the bridge.
>
>
> > For accessing the root file system, what I'd personally like is a ROOT
> > filesystem type for requestFileSystem. If the file-system-roots plugin
> > provides that I think we should definitely pull it in.
> >
>
> It does provide that; although I haven't enabled it by default. I've been
> pretty wary of allowing JavaScript to arbitrarily access *everything* on
> the entire device that is permitted by the OS.
>
>
> >
> > For the camera and media plugins on BB10 we return a full path relative
> to
> > the file system root as the native apps save into default locations. I
> > wonder if there is an opportunity to add additional roots which map to
> > default locations for pictures, music and video (maybe also documents?).
> It
> > seems like that would be super useful for media player apps or
> downloaders.
> >
>
> That's the whole point of the file-system-roots plugin -- on Android, it
> adds things like sdcard, external cache, external persistent files,
> documetns, etc. On iOS, it gives library, documents, cache, bundle, and a
> couple of non-icloud-synced versions of them. I'd love to get a set of
> useful BlackBerry file system roots as well.
>
> Ian
>
> >
> > On Tue, Apr 1, 2014 at 4:50 AM, Anis KADRI <an...@gmail.com> wrote:
> >
> > > On Tue, Apr 1, 2014 at 3:28 AM, Ian Clelland <ic...@chromium.org>
> > > wrote:
> > >
> > > > On Mon, Mar 31, 2014 at 4:33 PM, Ian Clelland <
> iclelland@chromium.org
> > > > >wrote:
> > > >
> > > > > On Mon, Mar 31, 2014 at 4:20 PM, Michal Mocny <mmocny@chromium.org
> >
> > > > wrote:
> > > > >
> > > > >> On Mon, Mar 31, 2014 at 3:39 PM, Ian Clelland <
> > iclelland@chromium.org
> > > > >> >wrote:
> > > > >> > This is ugly, though, and is going to get worse over time, and
> > > become
> > > > a
> > > > >> > division between Cordova and any platforms which actually
> > implement
> > > > the
> > > > >> > File API correctly. I'd like to propose switching the behaviour
> of
> > > > >> > .toURL(), to match .toNativeURL -- returning a webview-usable
> URL
> > by
> > > > >> > default -- and implementing some other method or property to get
> > the
> > > > CDV
> > > > >> > URL when it's necessary.
> > > > >> >
> > > > >>
> > > > >> Everything you've said sounds like its all upside to make the
> > switch.
> > > >  So
> > > > >> I'm curious, when would CDV URL be necessary/useful over
> > file/content
> > > > >> urls?
> > > > >>
> > > > >
> > > > > cdvfile:// URLs would still be necessary when dealing with files
> that
> > > > just
> > > > > don't *have* an alternate representation. There currently aren't
> any
> > of
> > > > > those, but we could implement virtual file systems entirely inside
> > of a
> > > > > plugin, and those would require a cdvfile:// URL to be read.
> > > > >
> > > > > I think we'd recommend them when saving URLs to persistent storage,
> > if
> > > > > there is any chance that the actual files could be moved /
> migrated,
> > > and
> > > > we
> > > > > could hide that from the user by giving them a more abstract
> > identifier
> > > > > than one which specifies a physical location.
> > > > > cdvfile://localhost/persistent/my/file.txt might be more durable
> over
> > > > time
> > > > > than file:///data/data/com.company.package/files/my/file.txt,
> perhaps
> > > > > across OS upgrades.
> > > > >
> > > >
> > > > Actually, forget all of that.
> > > >
> > > > Your question had me looking for reasons to advocate users using
> > > cdvfile://
> > > > URLs, when perhaps none exist. The truth of the matter is this: The
> > > cdvfile
> > > > URL has two parts: the filesystem name, and the full path. Those two
> > > parts
> > > > form a consistent internal representation for all of the types of
> file
> > > that
> > > > the plugin can handle, and so all of the internal / native bits of
> the
> > > file
> > > > plugin use them almost exclusively. We make sure that every FileEntry
> > and
> > > > DirectoryEntry has those parts, and we only need to turn them into a
> > URL
> > > > for passing them across the bridge.
> > > >
> > > > One day someone may discover a great reason to use deliberately use
> > > cdvfile
> > > > URLs at the application level; until then, they're available, and we
> > can
> > > > continue to use them internally to simplify the plugin code, enforce
> > the
> > > > sandboxing, and make everything generally more consistent and
> > efficient,
> > > > and users shouldn't need to know or care what the URLs in use
> actually
> > > are.
> > > >
> > >
> > > I agree with this as long as the URLs are useable in the WebView (as
> src
> > > attributes for example). If they're not, I also suggest that we return
> > URLs
> > > that are useable (file:///, content:/// or whatever) by default.
> > >
> > > As for filesystems (temp or persistent), I think most developers will
> use
> > > whatever the default is. BUT they should be able to specify where they
> > want
> > > to store their data if they feel like it without using a third-party
> > > plugin.
> > >
> > >
> > > >
> > > > Ian
> > > >
> > >
> >
>

Re: Looking at the new File API pragmatically

[Ian Clelland <ic...@chromium.org>]

On Tue, Apr 1, 2014 at 10:14 AM, Bryan Higgins <br...@bryanhiggins.net>wrote:

> +1 to replacing toURL with toNativeURL behaviour
>

I think that's what I'll do. I'm also adding a "toInternalURL" method --
not because it's necessarily useful for end users, but because we can use
it as the single method to construct URLs for transport across the bridge.


> For accessing the root file system, what I'd personally like is a ROOT
> filesystem type for requestFileSystem. If the file-system-roots plugin
> provides that I think we should definitely pull it in.
>

It does provide that; although I haven't enabled it by default. I've been
pretty wary of allowing JavaScript to arbitrarily access *everything* on
the entire device that is permitted by the OS.


>
> For the camera and media plugins on BB10 we return a full path relative to
> the file system root as the native apps save into default locations. I
> wonder if there is an opportunity to add additional roots which map to
> default locations for pictures, music and video (maybe also documents?). It
> seems like that would be super useful for media player apps or downloaders.
>

That's the whole point of the file-system-roots plugin -- on Android, it
adds things like sdcard, external cache, external persistent files,
documetns, etc. On iOS, it gives library, documents, cache, bundle, and a
couple of non-icloud-synced versions of them. I'd love to get a set of
useful BlackBerry file system roots as well.

Ian

>
> On Tue, Apr 1, 2014 at 4:50 AM, Anis KADRI <an...@gmail.com> wrote:
>
> > On Tue, Apr 1, 2014 at 3:28 AM, Ian Clelland <ic...@chromium.org>
> > wrote:
> >
> > > On Mon, Mar 31, 2014 at 4:33 PM, Ian Clelland <iclelland@chromium.org
> > > >wrote:
> > >
> > > > On Mon, Mar 31, 2014 at 4:20 PM, Michal Mocny <mm...@chromium.org>
> > > wrote:
> > > >
> > > >> On Mon, Mar 31, 2014 at 3:39 PM, Ian Clelland <
> iclelland@chromium.org
> > > >> >wrote:
> > > >> > This is ugly, though, and is going to get worse over time, and
> > become
> > > a
> > > >> > division between Cordova and any platforms which actually
> implement
> > > the
> > > >> > File API correctly. I'd like to propose switching the behaviour of
> > > >> > .toURL(), to match .toNativeURL -- returning a webview-usable URL
> by
> > > >> > default -- and implementing some other method or property to get
> the
> > > CDV
> > > >> > URL when it's necessary.
> > > >> >
> > > >>
> > > >> Everything you've said sounds like its all upside to make the
> switch.
> > >  So
> > > >> I'm curious, when would CDV URL be necessary/useful over
> file/content
> > > >> urls?
> > > >>
> > > >
> > > > cdvfile:// URLs would still be necessary when dealing with files that
> > > just
> > > > don't *have* an alternate representation. There currently aren't any
> of
> > > > those, but we could implement virtual file systems entirely inside
> of a
> > > > plugin, and those would require a cdvfile:// URL to be read.
> > > >
> > > > I think we'd recommend them when saving URLs to persistent storage,
> if
> > > > there is any chance that the actual files could be moved / migrated,
> > and
> > > we
> > > > could hide that from the user by giving them a more abstract
> identifier
> > > > than one which specifies a physical location.
> > > > cdvfile://localhost/persistent/my/file.txt might be more durable over
> > > time
> > > > than file:///data/data/com.company.package/files/my/file.txt, perhaps
> > > > across OS upgrades.
> > > >
> > >
> > > Actually, forget all of that.
> > >
> > > Your question had me looking for reasons to advocate users using
> > cdvfile://
> > > URLs, when perhaps none exist. The truth of the matter is this: The
> > cdvfile
> > > URL has two parts: the filesystem name, and the full path. Those two
> > parts
> > > form a consistent internal representation for all of the types of file
> > that
> > > the plugin can handle, and so all of the internal / native bits of the
> > file
> > > plugin use them almost exclusively. We make sure that every FileEntry
> and
> > > DirectoryEntry has those parts, and we only need to turn them into a
> URL
> > > for passing them across the bridge.
> > >
> > > One day someone may discover a great reason to use deliberately use
> > cdvfile
> > > URLs at the application level; until then, they're available, and we
> can
> > > continue to use them internally to simplify the plugin code, enforce
> the
> > > sandboxing, and make everything generally more consistent and
> efficient,
> > > and users shouldn't need to know or care what the URLs in use actually
> > are.
> > >
> >
> > I agree with this as long as the URLs are useable in the WebView (as src
> > attributes for example). If they're not, I also suggest that we return
> URLs
> > that are useable (file:///, content:/// or whatever) by default.
> >
> > As for filesystems (temp or persistent), I think most developers will use
> > whatever the default is. BUT they should be able to specify where they
> want
> > to store their data if they feel like it without using a third-party
> > plugin.
> >
> >
> > >
> > > Ian
> > >
> >
>

Re: Looking at the new File API pragmatically

[Bryan Higgins <br...@bryanhiggins.net>]

+1 to replacing toURL with toNativeURL behaviour

For accessing the root file system, what I'd personally like is a ROOT
filesystem type for requestFileSystem. If the file-system-roots plugin
provides that I think we should definitely pull it in.

For the camera and media plugins on BB10 we return a full path relative to
the file system root as the native apps save into default locations. I
wonder if there is an opportunity to add additional roots which map to
default locations for pictures, music and video (maybe also documents?). It
seems like that would be super useful for media player apps or downloaders.

On Tue, Apr 1, 2014 at 4:50 AM, Anis KADRI <an...@gmail.com> wrote:

> On Tue, Apr 1, 2014 at 3:28 AM, Ian Clelland <ic...@chromium.org>
> wrote:
>
> > On Mon, Mar 31, 2014 at 4:33 PM, Ian Clelland <iclelland@chromium.org
> > >wrote:
> >
> > > On Mon, Mar 31, 2014 at 4:20 PM, Michal Mocny <mm...@chromium.org>
> > wrote:
> > >
> > >> On Mon, Mar 31, 2014 at 3:39 PM, Ian Clelland <iclelland@chromium.org
> > >> >wrote:
> > >> > This is ugly, though, and is going to get worse over time, and
> become
> > a
> > >> > division between Cordova and any platforms which actually implement
> > the
> > >> > File API correctly. I'd like to propose switching the behaviour of
> > >> > .toURL(), to match .toNativeURL -- returning a webview-usable URL by
> > >> > default -- and implementing some other method or property to get the
> > CDV
> > >> > URL when it's necessary.
> > >> >
> > >>
> > >> Everything you've said sounds like its all upside to make the switch.
> >  So
> > >> I'm curious, when would CDV URL be necessary/useful over file/content
> > >> urls?
> > >>
> > >
> > > cdvfile:// URLs would still be necessary when dealing with files that
> > just
> > > don't *have* an alternate representation. There currently aren't any of
> > > those, but we could implement virtual file systems entirely inside of a
> > > plugin, and those would require a cdvfile:// URL to be read.
> > >
> > > I think we'd recommend them when saving URLs to persistent storage, if
> > > there is any chance that the actual files could be moved / migrated,
> and
> > we
> > > could hide that from the user by giving them a more abstract identifier
> > > than one which specifies a physical location.
> > > cdvfile://localhost/persistent/my/file.txt might be more durable over
> > time
> > > than file:///data/data/com.company.package/files/my/file.txt, perhaps
> > > across OS upgrades.
> > >
> >
> > Actually, forget all of that.
> >
> > Your question had me looking for reasons to advocate users using
> cdvfile://
> > URLs, when perhaps none exist. The truth of the matter is this: The
> cdvfile
> > URL has two parts: the filesystem name, and the full path. Those two
> parts
> > form a consistent internal representation for all of the types of file
> that
> > the plugin can handle, and so all of the internal / native bits of the
> file
> > plugin use them almost exclusively. We make sure that every FileEntry and
> > DirectoryEntry has those parts, and we only need to turn them into a URL
> > for passing them across the bridge.
> >
> > One day someone may discover a great reason to use deliberately use
> cdvfile
> > URLs at the application level; until then, they're available, and we can
> > continue to use them internally to simplify the plugin code, enforce the
> > sandboxing, and make everything generally more consistent and efficient,
> > and users shouldn't need to know or care what the URLs in use actually
> are.
> >
>
> I agree with this as long as the URLs are useable in the WebView (as src
> attributes for example). If they're not, I also suggest that we return URLs
> that are useable (file:///, content:/// or whatever) by default.
>
> As for filesystems (temp or persistent), I think most developers will use
> whatever the default is. BUT they should be able to specify where they want
> to store their data if they feel like it without using a third-party
> plugin.
>
>
> >
> > Ian
> >
>

Re: Looking at the new File API pragmatically

[Anis KADRI <an...@gmail.com>]

On Tue, Apr 1, 2014 at 3:28 AM, Ian Clelland <ic...@chromium.org> wrote:

> On Mon, Mar 31, 2014 at 4:33 PM, Ian Clelland <iclelland@chromium.org
> >wrote:
>
> > On Mon, Mar 31, 2014 at 4:20 PM, Michal Mocny <mm...@chromium.org>
> wrote:
> >
> >> On Mon, Mar 31, 2014 at 3:39 PM, Ian Clelland <iclelland@chromium.org
> >> >wrote:
> >> > This is ugly, though, and is going to get worse over time, and become
> a
> >> > division between Cordova and any platforms which actually implement
> the
> >> > File API correctly. I'd like to propose switching the behaviour of
> >> > .toURL(), to match .toNativeURL -- returning a webview-usable URL by
> >> > default -- and implementing some other method or property to get the
> CDV
> >> > URL when it's necessary.
> >> >
> >>
> >> Everything you've said sounds like its all upside to make the switch.
>  So
> >> I'm curious, when would CDV URL be necessary/useful over file/content
> >> urls?
> >>
> >
> > cdvfile:// URLs would still be necessary when dealing with files that
> just
> > don't *have* an alternate representation. There currently aren't any of
> > those, but we could implement virtual file systems entirely inside of a
> > plugin, and those would require a cdvfile:// URL to be read.
> >
> > I think we'd recommend them when saving URLs to persistent storage, if
> > there is any chance that the actual files could be moved / migrated, and
> we
> > could hide that from the user by giving them a more abstract identifier
> > than one which specifies a physical location.
> > cdvfile://localhost/persistent/my/file.txt might be more durable over
> time
> > than file:///data/data/com.company.package/files/my/file.txt, perhaps
> > across OS upgrades.
> >
>
> Actually, forget all of that.
>
> Your question had me looking for reasons to advocate users using cdvfile://
> URLs, when perhaps none exist. The truth of the matter is this: The cdvfile
> URL has two parts: the filesystem name, and the full path. Those two parts
> form a consistent internal representation for all of the types of file that
> the plugin can handle, and so all of the internal / native bits of the file
> plugin use them almost exclusively. We make sure that every FileEntry and
> DirectoryEntry has those parts, and we only need to turn them into a URL
> for passing them across the bridge.
>
> One day someone may discover a great reason to use deliberately use cdvfile
> URLs at the application level; until then, they're available, and we can
> continue to use them internally to simplify the plugin code, enforce the
> sandboxing, and make everything generally more consistent and efficient,
> and users shouldn't need to know or care what the URLs in use actually are.
>

I agree with this as long as the URLs are useable in the WebView (as src
attributes for example). If they're not, I also suggest that we return URLs
that are useable (file:///, content:/// or whatever) by default.

As for filesystems (temp or persistent), I think most developers will use
whatever the default is. BUT they should be able to specify where they want
to store their data if they feel like it without using a third-party plugin.


>
> Ian
>

Re: Looking at the new File API pragmatically

[Ian Clelland <ic...@chromium.org>]

On Mon, Mar 31, 2014 at 4:33 PM, Ian Clelland <ic...@chromium.org>wrote:

> On Mon, Mar 31, 2014 at 4:20 PM, Michal Mocny <mm...@chromium.org> wrote:
>
>> On Mon, Mar 31, 2014 at 3:39 PM, Ian Clelland <iclelland@chromium.org
>> >wrote:
>> > This is ugly, though, and is going to get worse over time, and become a
>> > division between Cordova and any platforms which actually implement the
>> > File API correctly. I'd like to propose switching the behaviour of
>> > .toURL(), to match .toNativeURL -- returning a webview-usable URL by
>> > default -- and implementing some other method or property to get the CDV
>> > URL when it's necessary.
>> >
>>
>> Everything you've said sounds like its all upside to make the switch.  So
>> I'm curious, when would CDV URL be necessary/useful over file/content
>> urls?
>>
>
> cdvfile:// URLs would still be necessary when dealing with files that just
> don't *have* an alternate representation. There currently aren't any of
> those, but we could implement virtual file systems entirely inside of a
> plugin, and those would require a cdvfile:// URL to be read.
>
> I think we'd recommend them when saving URLs to persistent storage, if
> there is any chance that the actual files could be moved / migrated, and we
> could hide that from the user by giving them a more abstract identifier
> than one which specifies a physical location.
> cdvfile://localhost/persistent/my/file.txt might be more durable over time
> than file:///data/data/com.company.package/files/my/file.txt, perhaps
> across OS upgrades.
>

Actually, forget all of that.

Your question had me looking for reasons to advocate users using cdvfile://
URLs, when perhaps none exist. The truth of the matter is this: The cdvfile
URL has two parts: the filesystem name, and the full path. Those two parts
form a consistent internal representation for all of the types of file that
the plugin can handle, and so all of the internal / native bits of the file
plugin use them almost exclusively. We make sure that every FileEntry and
DirectoryEntry has those parts, and we only need to turn them into a URL
for passing them across the bridge.

One day someone may discover a great reason to use deliberately use cdvfile
URLs at the application level; until then, they're available, and we can
continue to use them internally to simplify the plugin code, enforce the
sandboxing, and make everything generally more consistent and efficient,
and users shouldn't need to know or care what the URLs in use actually are.

Ian

Re: Looking at the new File API pragmatically

[Ian Clelland <ic...@chromium.org>]

On Mon, Mar 31, 2014 at 4:20 PM, Michal Mocny <mm...@chromium.org> wrote:

> On Mon, Mar 31, 2014 at 3:39 PM, Ian Clelland <iclelland@chromium.org
> >wrote:
> > This is ugly, though, and is going to get worse over time, and become a
> > division between Cordova and any platforms which actually implement the
> > File API correctly. I'd like to propose switching the behaviour of
> > .toURL(), to match .toNativeURL -- returning a webview-usable URL by
> > default -- and implementing some other method or property to get the CDV
> > URL when it's necessary.
> >
>
> Everything you've said sounds like its all upside to make the switch.  So
> I'm curious, when would CDV URL be necessary/useful over file/content urls?
>

cdvfile:// URLs would still be necessary when dealing with files that just
don't *have* an alternate representation. There currently aren't any of
those, but we could implement virtual file systems entirely inside of a
plugin, and those would require a cdvfile:// URL to be read.

I think we'd recommend them when saving URLs to persistent storage, if
there is any chance that the actual files could be moved / migrated, and we
could hide that from the user by giving them a more abstract identifier
than one which specifies a physical location.
cdvfile://localhost/persistent/my/file.txt might be more durable over time
than file:///data/data/com.company.package/files/my/file.txt, perhaps
across OS upgrades.


>
>
> >
> > ---- TWO ----
> >
> > The second issue has to do with the filesystem locations which are made
> > available to Cordova apps. The File plugin provides two sandboxed
> > filesystem locations, one for persistent files, and one for temporary
> > files. Files which reside outside of those two locations are not
> accessible
> > to the plugin, by default.
> >
> > (For comparison, the old version of File would happily access files
> > anywhere on the device, even well outside of the claimed 'roots')
> >
> > This generally works, doesn't cause problems with applications using the
> > File plugin, and is (I think) still a good idea. It breaks down, however,
> > when other plugins come in to the mix. Before File was released, I made
> > changes to FileTransfer, and since, I have changed Camera, and been
> working
> > on MediaCapture -- all to ensure that they play within File's sandboxes.
> > MediaCapture, though, is more difficult, because it's the OS that is
> > creating the files, and it will create them where it wants to, not where
> we
> > think it should.
> >
> > I revived the file-system-roots plugin to take care of this problem --
> when
> > installed, it registers several additional file systems, covering other
> > areas of the device, allowing access to places like "SD Card storage",
> > "external cache", "internal cache", etc (see the plugin README for all of
> >
>
> FYI:
> https://github.com/apache/cordova-plugins/tree/master/file-system-roots
>
>
> > the details). It's even possible to register the "root" filesystem, which
> > re-enables access to the *entire* device. Adding this plugin has worked
> for
> > basically everyone who's had a problem with file locations, but it seems
> > wrong to have to recommend an outside-of-core plugin to fix a problem
> with
> > core plugins.
> >
> > The question here, I guess -- because a number of people have required
> this
> > feature, and there may be additional third-party plugins out there that
> > will require it as well -- is whether it makes sense to include this
> > functionality in the core File plugin, or whether to just continue to
> > recommend it to people whose apps write files in locations outside of the
> > standard persistent/temporary file systems.
>
>
> > Thoughts, opinions, flames -- all welcomed. Thanks for reading this far
> :)
> >
>
> I do like adding these by default.
>
> Most devs will follow your guidance anyway by use of PERSISTENT and
> TEMPORARY fs type, and will never "accidently" reach into these other fs
> roots without meaning to.  I like when hybrid apps can do just as much as
> native equivalents without having to try too hard.
>
>
> > Ian
> >
>

Re: Looking at the new File API pragmatically

[Michal Mocny <mm...@chromium.org>]

On Mon, Mar 31, 2014 at 3:39 PM, Ian Clelland <ic...@chromium.org>wrote:

> The new File plugin has been in the wild now for almost two months now, and
> it seems to be seeing a lot of real-world use (bug reports are coming in to
> JIRA almost daily :) )
>
> There have been a lot of bugs fixed, and a fair number of people guided
> towards new ways of doing things, but there are definitely some recurring
> themes, and I think, looking at things pragmatically, that there are some
> things that we can do to make the whole file experience better for
> developers.
>
>
> ---- ONE ----
>
> The first thing has to do with .toURL(), and it's new baby brother,
> .toNativeURL(). Originally, the ideal was that .toURL() would always return
> a URL with the prefix "cdvfile://", but this has caused problems with a
> number of use cases, all involving passing the URL to the webview as a
> "src" attribute. The Android code for resolving "src" attributes generally
> doesn't pass the URLs back to the application, and so a real file:/// URL
> is required.
>
> Because of this, I introduced .toNativeURL, which will try to return a
> file:/// URL, or a content:// URL, and will only return a cdvfile:// URL if
> it has no other alternative. I've been providing guidance to developers to
> try to use that if they want to use File-plugin-files in video or audio
> tags, or even for img tags on Android <= 3.0
>
> This is ugly, though, and is going to get worse over time, and become a
> division between Cordova and any platforms which actually implement the
> File API correctly. I'd like to propose switching the behaviour of
> .toURL(), to match .toNativeURL -- returning a webview-usable URL by
> default -- and implementing some other method or property to get the CDV
> URL when it's necessary.
>

Everything you've said sounds like its all upside to make the switch.  So
I'm curious, when would CDV URL be necessary/useful over file/content urls?


>
> ---- TWO ----
>
> The second issue has to do with the filesystem locations which are made
> available to Cordova apps. The File plugin provides two sandboxed
> filesystem locations, one for persistent files, and one for temporary
> files. Files which reside outside of those two locations are not accessible
> to the plugin, by default.
>
> (For comparison, the old version of File would happily access files
> anywhere on the device, even well outside of the claimed 'roots')
>
> This generally works, doesn't cause problems with applications using the
> File plugin, and is (I think) still a good idea. It breaks down, however,
> when other plugins come in to the mix. Before File was released, I made
> changes to FileTransfer, and since, I have changed Camera, and been working
> on MediaCapture -- all to ensure that they play within File's sandboxes.
> MediaCapture, though, is more difficult, because it's the OS that is
> creating the files, and it will create them where it wants to, not where we
> think it should.
>
> I revived the file-system-roots plugin to take care of this problem -- when
> installed, it registers several additional file systems, covering other
> areas of the device, allowing access to places like "SD Card storage",
> "external cache", "internal cache", etc (see the plugin README for all of
>

FYI: https://github.com/apache/cordova-plugins/tree/master/file-system-roots


> the details). It's even possible to register the "root" filesystem, which
> re-enables access to the *entire* device. Adding this plugin has worked for
> basically everyone who's had a problem with file locations, but it seems
> wrong to have to recommend an outside-of-core plugin to fix a problem with
> core plugins.
>
> The question here, I guess -- because a number of people have required this
> feature, and there may be additional third-party plugins out there that
> will require it as well -- is whether it makes sense to include this
> functionality in the core File plugin, or whether to just continue to
> recommend it to people whose apps write files in locations outside of the
> standard persistent/temporary file systems.


> Thoughts, opinions, flames -- all welcomed. Thanks for reading this far :)
>

I do like adding these by default.

Most devs will follow your guidance anyway by use of PERSISTENT and
TEMPORARY fs type, and will never "accidently" reach into these other fs
roots without meaning to.  I like when hybrid apps can do just as much as
native equivalents without having to try too hard.


> Ian
>