You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@shindig.apache.org by Dan Dumont <dd...@us.ibm.com> on 2012/02/22 15:32:39 UTC
Authenticated, binary downloads in gadgets
Similar to the proxied-form-post feature that I recently added to shindig,
there is a need to make an authenticated file download in a gadget.
Authentication for a file download is likely needed and since the shindig
server handles that for the gadget, requests directly to the remote
service would be rejected (without some monkeying around with one-time-use
tokens).
It would seem that the content proxy (getProxyUrl) is inappropriate here,
as it doesn't look like it can or should accept headers and authentication
params.
makeRequest and osapi.http.* apis also seem inappropriate for this use
case, as they contain a bunch of extra information in the response that
isn't wanted (we just need the binary data to let the browser save the
file).
So it looks like the only option is a new feature that would allow for
streamed file downloads through an OS proxy. Hopefully, like the
proxied-form-post feature, we can reuse most of the existing makeRequest
server logic to do this for us. I'm wondering if we create a
gadgets.files.* namespace to house these new functions, or if we should
keep them in gadgets.io?
Any thoughts on all of this?
streamed request in gadgets
Posted by Wei CSDL Shi <sh...@cn.ibm.com>.
In enterprise gadget development, there are requirements to allow files
upload/download up to 512MB. Shindig server is processing the fetched
response from target url every time, convert the stream fetched into byte
array and store the byte array in HttpResponse, later on the HttpResponse
can be copied to a ServletResponse and return to client. This solution has
severe problem when the stream is huge: it could easily cause the JVM to
exhaust its heap under moderate to heavy load.
Dan's patch about proxied-form-post would solve the upload. We'd like to
propose a download solution: streamed request.
The idea is to add a new field in gadgets.io.RequestParameters.STREAMED,
and update gadgets.io.getProxyUrl() to consider the case that this is a
streamed request, and return a proxied URL like this:
http://localhost:8080/gadgets/proxy?container=default&refresh=0&url=http%3A%2F%2Flocalhost%3A8080%2Ftest%2Ftest.rar&
streamed=true
Then Shindig server side ProxyHandler would be able to tell whether the
response stream need to be processed in Shindig server or not. If it is a
streamed request, Shindig server won't process it, just set the stream into
HttpResponse and return it to ServletReponse. All the other processes such
as response headers, security, caching etc will be kept the same as they
were.
The changes need to be made in Shindig server are Proxy URL handling,
HttpResponse, HttpResponseBuilder, ProxyHandler, ProxyServlet,
DefaultRequestPipeline and BasicHttpFetcher.
Could you guys provide comment on this solution and see if we miss
anything?
Thanks
Best Regards
Marshall Shi(Shi Wei)
Mark Weitzel
<weitzelm.work@gm
ail.com> To
dev@shindig.apache.org
2012-02-26 21:14 cc
Subject
Please respond to Re: Authenticated, binary downloads
dev@shindig.apach in gadgets
e.org
Dan,
We've been talking about aligning with cmis so the use of files may be
misleading even if we add it as osapi
On Feb 22, 2012 9:33 AM, "Dan Dumont" <dd...@us.ibm.com> wrote:
> Similar to the proxied-form-post feature that I recently added to
shindig,
> there is a need to make an authenticated file download in a gadget.
> Authentication for a file download is likely needed and since the shindig
> server handles that for the gadget, requests directly to the remote
> service would be rejected (without some monkeying around with
one-time-use
> tokens).
>
> It would seem that the content proxy (getProxyUrl) is inappropriate here,
> as it doesn't look like it can or should accept headers and
authentication
> params.
> makeRequest and osapi.http.* apis also seem inappropriate for this use
> case, as they contain a bunch of extra information in the response that
> isn't wanted (we just need the binary data to let the browser save the
> file).
>
> So it looks like the only option is a new feature that would allow for
> streamed file downloads through an OS proxy. Hopefully, like the
> proxied-form-post feature, we can reuse most of the existing makeRequest
> server logic to do this for us. I'm wondering if we create a
> gadgets.files.* namespace to house these new functions, or if we should
> keep them in gadgets.io?
>
> Any thoughts on all of this?
Re: Authenticated, binary downloads in gadgets
Posted by Mark Weitzel <we...@gmail.com>.
Dan,
We've been talking about aligning with cmis so the use of files may be
misleading even if we add it as osapi
On Feb 22, 2012 9:33 AM, "Dan Dumont" <dd...@us.ibm.com> wrote:
> Similar to the proxied-form-post feature that I recently added to shindig,
> there is a need to make an authenticated file download in a gadget.
> Authentication for a file download is likely needed and since the shindig
> server handles that for the gadget, requests directly to the remote
> service would be rejected (without some monkeying around with one-time-use
> tokens).
>
> It would seem that the content proxy (getProxyUrl) is inappropriate here,
> as it doesn't look like it can or should accept headers and authentication
> params.
> makeRequest and osapi.http.* apis also seem inappropriate for this use
> case, as they contain a bunch of extra information in the response that
> isn't wanted (we just need the binary data to let the browser save the
> file).
>
> So it looks like the only option is a new feature that would allow for
> streamed file downloads through an OS proxy. Hopefully, like the
> proxied-form-post feature, we can reuse most of the existing makeRequest
> server logic to do this for us. I'm wondering if we create a
> gadgets.files.* namespace to house these new functions, or if we should
> keep them in gadgets.io?
>
> Any thoughts on all of this?