You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@felix.apache.org by Sahoo <Sa...@Sun.COM> on 2009/06/07 15:22:47 UTC
Why reference: scheme is supported only for file URLs?
Hi,
I see the following code in BundleArchive.java:
// Check if the location string represents a reference URL.
if ((location != null) &&
location.startsWith(REFERENCE_PROTOCOL))
{
// Reference URLs only support the file protocol.
location = location.substring(REFERENCE_PROTOCOL.length());
if (!location.startsWith(FILE_PROTOCOL))
{
throw new IOException("Reference URLs can only be
files: " + location);
}
...
Why does reference: scheme only work if the underlying URL represents a
file? Can't the code assume that for all other protocols, the underlying
stream is a JarInputStream and proceed from there?
Thanks,
Sahoo
Re: Why reference: scheme is supported only for file URLs?
Posted by Sahoo <Sa...@Sun.COM>.
Stuart McCulloch wrote:
> 2009/6/8 Sahoo <Sa...@sun.com>
>
>
>> Stuart McCulloch wrote:
>>
>>
>>> 2009/6/8 Sahoo <Sa...@sun.com>
>>>
>>> Stuart McCulloch wrote:
>>>
>>>>
>>>>> I guess I'm not seeing why you want to use the reference: scheme with
>>>>> non-file URLs?
>>>>>
>>>>> do you just want to avoid copying the remote content locally or is there
>>>>> another reason?
>>>>>
>>>>> Good question. I want to modify the content as it is read. e.g., I want
>>>>>
>>>> to
>>>> process the MANIFEST.MF by having my own URL handler for a custom
>>>> protocol.
>>>> All these work as long as I don't use reference scheme.
>>>>
>>>>
>>> Ah, ok - I think I understand now.
>>>
>>> You want to have "reference:" in the URL and have Felix ignore it for
>>> non-"file:" schemes.
>>>
>>>
>>>
>> Not really.
>>
>>
>
> hmm, well at the moment it throws an exception which I doubt you want ;)
>
> but seriously the only alternative is to log a message and treat it as a
> non-reference URL (see below)
>
>
>
I think exception is a better choice (as is the current case) when
reference semantics can't be supported. So, I am OK if there is no
change made. See below for a description of my use case.
>> ie. log a warning / info / debug and continue as if the URL was not a
>>
>>> "reference:" instead
>>> of throwing an exception as it currently does.
>>>
>>>
>>>
>> I just want Felix to read the content off the JarInputStream when it
>> encounters a non-file URL after reference:. Is there a reason why Felix must
>> copy? Is performance the reason?
>>
>
>
> yes - as Richard pointed out we need access to the file, remotely querying a
> jar over the network is very slow compared to copying it and querying
> locally
>
> What if the custom url handler is already backed by a JarFile object?
>
>
> problem is we have no way of knowing that - we'd have to repeatedly query
> through the handler, which could also be slow
>
>
>
>> Let's put it this way. How can we have directory deployment of bundles with
>> support for content filtering?
>>
>>
>
> isn't this what the various OPS4J URL handlers do? they let you install and
> auto-wrap jars into bundles, etc.
>
> as you say everything works when you don't have "reference:" in front of
> your custom URLs, so what's the benefit that you expect to gain from adding
> it?
>
>
>
What I wanted to do was directory deployment with MANIFEST.MF
replacement. So, it involved both reference: scheme (for directory
deployment) and my custom webbundle: scheme (for MANIFEST .MF
processing). The reason for directory deployment was to allow users to
change JSP pages in the source directory and see the effect immediately
next time when they accessed the JSP over the web. Without directory
deployment, they would have had to package the changed JSPs and update
the bundle. Since, this is all happening in the development
environment, I feel performance is not a big concern here. Plus, network
is actually not involved here.
I am fine if this is not considered a valid use case.
Thanks,
Sahoo
Re: Why reference: scheme is supported only for file URLs?
Posted by Stuart McCulloch <mc...@gmail.com>.
2009/6/8 Sahoo <Sa...@sun.com>
> Stuart McCulloch wrote:
>
>> 2009/6/8 Sahoo <Sa...@sun.com>
>>
>> Stuart McCulloch wrote:
>>>
>>>
>>>> I guess I'm not seeing why you want to use the reference: scheme with
>>>> non-file URLs?
>>>>
>>>> do you just want to avoid copying the remote content locally or is there
>>>> another reason?
>>>>
>>>> Good question. I want to modify the content as it is read. e.g., I want
>>> to
>>> process the MANIFEST.MF by having my own URL handler for a custom
>>> protocol.
>>> All these work as long as I don't use reference scheme.
>>>
>>
>> Ah, ok - I think I understand now.
>>
>> You want to have "reference:" in the URL and have Felix ignore it for
>> non-"file:" schemes.
>>
>>
> Not really.
>
hmm, well at the moment it throws an exception which I doubt you want ;)
but seriously the only alternative is to log a message and treat it as a
non-reference URL (see below)
>
> ie. log a warning / info / debug and continue as if the URL was not a
>> "reference:" instead
>> of throwing an exception as it currently does.
>>
>>
> I just want Felix to read the content off the JarInputStream when it
> encounters a non-file URL after reference:. Is there a reason why Felix must
> copy? Is performance the reason?
yes - as Richard pointed out we need access to the file, remotely querying a
jar over the network is very slow compared to copying it and querying
locally
What if the custom url handler is already backed by a JarFile object?
>
problem is we have no way of knowing that - we'd have to repeatedly query
through the handler, which could also be slow
> Let's put it this way. How can we have directory deployment of bundles with
> support for content filtering?
>
isn't this what the various OPS4J URL handlers do? they let you install and
auto-wrap jars into bundles, etc.
as you say everything works when you don't have "reference:" in front of
your custom URLs, so what's the benefit that you expect to gain from adding
it?
> Thanks,
> Sahoo
>
--
Cheers, Stuart
Re: Why reference: scheme is supported only for file URLs?
Posted by Sahoo <Sa...@Sun.COM>.
Stuart McCulloch wrote:
> 2009/6/8 Sahoo <Sa...@sun.com>
>
>
>> Stuart McCulloch wrote:
>>
>>
>>> I guess I'm not seeing why you want to use the reference: scheme with
>>> non-file URLs?
>>>
>>> do you just want to avoid copying the remote content locally or is there
>>> another reason?
>>>
>>>
>>>
>> Good question. I want to modify the content as it is read. e.g., I want to
>> process the MANIFEST.MF by having my own URL handler for a custom protocol.
>> All these work as long as I don't use reference scheme.
>>
>>
>
> Ah, ok - I think I understand now.
>
> You want to have "reference:" in the URL and have Felix ignore it for
> non-"file:" schemes.
>
Not really.
> ie. log a warning / info / debug and continue as if the URL was not a
> "reference:" instead
> of throwing an exception as it currently does.
>
I just want Felix to read the content off the JarInputStream when it
encounters a non-file URL after reference:. Is there a reason why Felix
must copy? Is performance the reason? What if the custom url handler is
already backed by a JarFile object?
Let's put it this way. How can we have directory deployment of bundles
with support for content filtering?
Thanks,
Sahoo
Re: Why reference: scheme is supported only for file URLs?
Posted by Stuart McCulloch <mc...@gmail.com>.
2009/6/8 Sahoo <Sa...@sun.com>
> Stuart McCulloch wrote:
>
>>
>> I guess I'm not seeing why you want to use the reference: scheme with
>> non-file URLs?
>>
>> do you just want to avoid copying the remote content locally or is there
>> another reason?
>>
>>
> Good question. I want to modify the content as it is read. e.g., I want to
> process the MANIFEST.MF by having my own URL handler for a custom protocol.
> All these work as long as I don't use reference scheme.
>
Ah, ok - I think I understand now.
You want to have "reference:" in the URL and have Felix ignore it for
non-"file:" schemes.
ie. log a warning / info / debug and continue as if the URL was not a
"reference:" instead
of throwing an exception as it currently does.
I was confused by the subject line, which suggested you wanted to apply the
"reference:"
semantics (ie. not copying the contents but accessing it directly) to
non-"file:" schemes.
Changing it from an exception to a message is OK with me - anyone else want
to chip in?
Thanks,
> Sahoo
>
--
Cheers, Stuart
Re: Why reference: scheme is supported only for file URLs?
Posted by "Richard S. Hall" <he...@ungoverned.org>.
On 6/7/09 9:31 AM, Sahoo wrote:
> Stuart McCulloch wrote:
>>
>> I guess I'm not seeing why you want to use the reference: scheme with
>> non-file URLs?
>>
>> do you just want to avoid copying the remote content locally or is there
>> another reason?
>>
> Good question. I want to modify the content as it is read. e.g., I
> want to process the MANIFEST.MF by having my own URL handler for a
> custom protocol. All these work as long as I don't use reference scheme.
We use JarFile internally, which needs a File. Of course, we could read
the JAR using a JarInputStream (I tried it this way a long time ago with
Oscar), but it is really slow. So, in the end, we need to copy it any
way, so it doesn't seem to make much of a difference. Last time I
checked, Equinox was the same.
It seems like you should just create your own stream handler, no?
-> richard
Re: Why reference: scheme is supported only for file URLs?
Posted by Sahoo <Sa...@Sun.COM>.
Stuart McCulloch wrote:
>
> I guess I'm not seeing why you want to use the reference: scheme with
> non-file URLs?
>
> do you just want to avoid copying the remote content locally or is there
> another reason?
>
>
Good question. I want to modify the content as it is read. e.g., I want
to process the MANIFEST.MF by having my own URL handler for a custom
protocol. All these work as long as I don't use reference scheme.
Thanks,
Sahoo
Re: Why reference: scheme is supported only for file URLs?
Posted by Stuart McCulloch <mc...@gmail.com>.
2009/6/8 Sahoo <Sa...@sun.com>
> Stuart McCulloch wrote:
>
>> 2009/6/7 Sahoo <Sa...@sun.com>
>>
>>
>>
>>> Hi,
>>>
>>> I see the following code in BundleArchive.java:
>>> // Check if the location string represents a reference URL.
>>> if ((location != null) &&
>>> location.startsWith(REFERENCE_PROTOCOL))
>>> {
>>> // Reference URLs only support the file protocol.
>>> location = location.substring(REFERENCE_PROTOCOL.length());
>>> if (!location.startsWith(FILE_PROTOCOL))
>>> {
>>> throw new IOException("Reference URLs can only be files:
>>> " + location);
>>> }
>>> ...
>>>
>>> Why does reference: scheme only work if the underlying URL represents a
>>> file? Can't the code assume that for all other protocols, the underlying
>>> stream is a JarInputStream and proceed from there?
>>>
>>>
>>>
>>
>> how would you use a non-file URL "in-place"? you'd need some way to
>> convert
>> it to a file location
>> otherwise you'd have to copy the stream, which would defeat the purpose of
>> using "reference:"
>>
>>
> I don't understand how a file URL makes any difference when the actual
> resource is a Jar file as opposed to a directory. The code still has to open
> a JarInputStream to read it. The same will be the case even when the
> resource is not a physical file.
>
for Jar files look at the "JarRevision" class: if it's a reference URL then
it uses the original Jar
to iterate through the entries, look up resources, etc. - streams are
created from the original
Jar file on the file-system as the access is presumably fast
if it's not a reference (or not a file URL) then the stream contents are
cached locally as a Jar
and the same code as above is used to iterate through, etc. as it's safer to
cache the content
locally than try to repeatedly access it remotely
now I guess it could open new streams to the remote URL each time it wanted
to scan the
contents or extract resources, but this could be slow and if the network was
intermittent it
could lead to some unexpected issues
Secondly, it is my understanding that the code copies embedded jar even for
> reference: scheme? Is this not true?
>
correct, but this is only for embedded jars and is another point where the
reference scheme
has to copy content rather than access it without copying - which is
basically why we don't use
it for non-file URLs
I guess I'm not seeing why you want to use the reference: scheme with
non-file URLs?
do you just want to avoid copying the remote content locally or is there
another reason?
Thanks,
> Sahoo
>
--
Cheers, Stuart
Re: Why reference: scheme is supported only for file URLs?
Posted by Karl Pauls <ka...@gmail.com>.
On Sun, Jun 7, 2009 at 6:05 PM, Sahoo<Sa...@sun.com> wrote:
> Stuart McCulloch wrote:
>>
>> 2009/6/7 Sahoo <Sa...@sun.com>
>>
>>
>>>
>>> Hi,
>>>
>>> I see the following code in BundleArchive.java:
>>> // Check if the location string represents a reference URL.
>>> if ((location != null) &&
>>> location.startsWith(REFERENCE_PROTOCOL))
>>> {
>>> // Reference URLs only support the file protocol.
>>> location = location.substring(REFERENCE_PROTOCOL.length());
>>> if (!location.startsWith(FILE_PROTOCOL))
>>> {
>>> throw new IOException("Reference URLs can only be files:
>>> " + location);
>>> }
>>> ...
>>>
>>> Why does reference: scheme only work if the underlying URL represents a
>>> file? Can't the code assume that for all other protocols, the underlying
>>> stream is a JarInputStream and proceed from there?
>>>
>>>
>>
>> how would you use a non-file URL "in-place"? you'd need some way to
>> convert
>> it to a file location
>> otherwise you'd have to copy the stream, which would defeat the purpose of
>> using "reference:"
>>
>
> I don't understand how a file URL makes any difference when the actual
> resource is a Jar file as opposed to a directory. The code still has to open
> a JarInputStream to read it. The same will be the case even when the
> resource is not a physical file.
>
> Secondly, it is my understanding that the code copies embedded jar even for
> reference: scheme? Is this not true?
Yes, unless it is an "exploded" jar (i.e., the url points to a
directory). I guess the problem is that opening the jar when it is not
on the file system would be pretty slow no?
regards,
Karl
> Thanks,
> Sahoo
>
--
Karl Pauls
karlpauls@gmail.com
Re: Why reference: scheme is supported only for file URLs?
Posted by Sahoo <Sa...@Sun.COM>.
Stuart McCulloch wrote:
> 2009/6/7 Sahoo <Sa...@sun.com>
>
>
>> Hi,
>>
>> I see the following code in BundleArchive.java:
>> // Check if the location string represents a reference URL.
>> if ((location != null) &&
>> location.startsWith(REFERENCE_PROTOCOL))
>> {
>> // Reference URLs only support the file protocol.
>> location = location.substring(REFERENCE_PROTOCOL.length());
>> if (!location.startsWith(FILE_PROTOCOL))
>> {
>> throw new IOException("Reference URLs can only be files:
>> " + location);
>> }
>> ...
>>
>> Why does reference: scheme only work if the underlying URL represents a
>> file? Can't the code assume that for all other protocols, the underlying
>> stream is a JarInputStream and proceed from there?
>>
>>
>
> how would you use a non-file URL "in-place"? you'd need some way to convert
> it to a file location
> otherwise you'd have to copy the stream, which would defeat the purpose of
> using "reference:"
>
I don't understand how a file URL makes any difference when the actual
resource is a Jar file as opposed to a directory. The code still has to
open a JarInputStream to read it. The same will be the case even when
the resource is not a physical file.
Secondly, it is my understanding that the code copies embedded jar even
for reference: scheme? Is this not true?
Thanks,
Sahoo
Re: Why reference: scheme is supported only for file URLs?
Posted by Stuart McCulloch <mc...@gmail.com>.
2009/6/7 Sahoo <Sa...@sun.com>
> Hi,
>
> I see the following code in BundleArchive.java:
> // Check if the location string represents a reference URL.
> if ((location != null) &&
> location.startsWith(REFERENCE_PROTOCOL))
> {
> // Reference URLs only support the file protocol.
> location = location.substring(REFERENCE_PROTOCOL.length());
> if (!location.startsWith(FILE_PROTOCOL))
> {
> throw new IOException("Reference URLs can only be files:
> " + location);
> }
> ...
>
> Why does reference: scheme only work if the underlying URL represents a
> file? Can't the code assume that for all other protocols, the underlying
> stream is a JarInputStream and proceed from there?
>
how would you use a non-file URL "in-place"? you'd need some way to convert
it to a file location
otherwise you'd have to copy the stream, which would defeat the purpose of
using "reference:"
Thanks,
> Sahoo
>
--
Cheers, Stuart