You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@wicket.apache.org by Gabor Szokoli <sz...@gmail.com> on 2007/12/11 23:39:46 UTC
"Streaming" a huge ZIP file
Hi there,
The downside of an easy-to-use framework is the influx of users with
little understanding of the underlying servlet technology, like me.
Observe:
I'd like to provide a ZIP file (could be TAR, anything Windows PCs can
save and later extract from) as a download, one that is potentially
too big to be kept in memory in full.
So I guess I'd need to "stream" it in chunks, or as a java stream or channel.
Can I do that from wicket?
Or does the servlet interface require a fully assembled response in memory?
Currently I assemble the ZIP file in a byte array in full, but it
feels awkward. Would extending WebResource fit the bill?
ByteArrayOutputStream bout = new ByteArrayOutputStream();
ZipOutputStream zout = new ZipOutputStream(bout);
// Add entries
zout.finish();
zout.close();
Resource resource = new ByteArrayResource("application/zip",
bout.toByteArray());
ResourceStreamRequestTarget target = new
ResourceStreamRequestTarget(resource.getResourceStream());
target.setFileName(new
SimpleDateFormat("yyyyMMddHHmmss").format(Calendar.getInstance().getTime())
+ ".zip");
getRequestCycle().setRequestTarget(target);
All suggestions are welcome.
Gabor Szokoli
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
For additional commands, e-mail: users-help@wicket.apache.org
Re: "Streaming" a huge ZIP file
Posted by Johan Compagner <jc...@gmail.com>.
you can't update a page and download in 1 request.
You have to do this in 2. First update the page that does a call back to the
server when loaded that loads the resource
johan
On Dec 17, 2007 9:40 AM, Gabor Szokoli <sz...@gmail.com> wrote:
> Hi,
>
> This looks completely straight-forward, we'll switch to it as soon as
> move to rc3.
>
> On Dec 16, 2007 8:33 PM, Johan Compagner <jc...@gmail.com> wrote:
>
> > RequestCycle.get().setRequestTarget(new
> ResourceStreamRequestTarget(writer,"
> > test.zip"));
>
> Maybe this is dumb, but if I encounter an exception in the write
> method (during content generation), can I "unsnatch" the stream and
> hand it back to wicket for displaying regular pages? I realise the
> headers like content type and even some content might have been
> streamed out already, but who knows.
>
> Slightly related, what would be the best way for a single onclick()
> event to result in both a wicket page refresh and a download?
>
>
> Gabor Szokoli
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
> For additional commands, e-mail: users-help@wicket.apache.org
>
>
Re: "Streaming" a huge ZIP file
Posted by Gabor Szokoli <sz...@gmail.com>.
Hi,
This looks completely straight-forward, we'll switch to it as soon as
move to rc3.
On Dec 16, 2007 8:33 PM, Johan Compagner <jc...@gmail.com> wrote:
> RequestCycle.get().setRequestTarget(new ResourceStreamRequestTarget(writer,"
> test.zip"));
Maybe this is dumb, but if I encounter an exception in the write
method (during content generation), can I "unsnatch" the stream and
hand it back to wicket for displaying regular pages? I realise the
headers like content type and even some content might have been
streamed out already, but who knows.
Slightly related, what would be the best way for a single onclick()
event to result in both a wicket page refresh and a download?
Gabor Szokoli
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
For additional commands, e-mail: users-help@wicket.apache.org
Re: "Streaming" a huge ZIP file
Posted by Johan Compagner <jc...@gmail.com>.
Stupid me of not knowing anymore what i committed myself
AbstractResourceStreamWriter writer = new AbstractResourceStreamWriter()
{
write(OutputStream output)
{
// do your generation and output
}
};
RequestCycle.get().setRequestTarget(new ResourceStreamRequestTarget(writer,"
test.zip"));
johan
P.S. not everything of this example is yet in a release (will be 1.3rc3)
On Dec 13, 2007 1:16 AM, Gabor Szokoli <sz...@gmail.com> wrote:
> On Dec 12, 2007 11:58 PM, Igor Vaynberg <ig...@gmail.com> wrote:
> > ((WebResponse)r.getResponse).setAttachmentHeader("foo.xls");
>
> Indeed!
> Thank you.
>
>
> Gabor Szokoli
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
> For additional commands, e-mail: users-help@wicket.apache.org
>
>
Re: "Streaming" a huge ZIP file
Posted by Gabor Szokoli <sz...@gmail.com>.
On Dec 12, 2007 11:58 PM, Igor Vaynberg <ig...@gmail.com> wrote:
> ((WebResponse)r.getResponse).setAttachmentHeader("foo.xls");
Indeed!
Thank you.
Gabor Szokoli
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
For additional commands, e-mail: users-help@wicket.apache.org
Re: "Streaming" a huge ZIP file
Posted by Igor Vaynberg <ig...@gmail.com>.
((WebResponse)r.getResponse).setAttachmentHeader("foo.xls");
-igor
On Dec 12, 2007 2:45 PM, Gabor Szokoli <sz...@gmail.com> wrote:
> On Dec 12, 2007 7:23 PM, Igor Vaynberg <ig...@gmail.com> wrote:
> > correct. only that one user will have to wait for the download to
> > complete. if that is undesired then use a shared resource.
>
> This is what we experience, and it is fine.
> Good natural limit on the amount of load each user can cause by accident :-)
>
> Here's Igors previous pseudocode in Java for the sake of completeness.
> My question is: how do I set the downloaded filename now?
>
> @Override
> public void onSubmit() {
>
> getRequestCycle().setRequestTarget(new IRequestTarget() {
>
> public void detach(RequestCycle arg0) {
> // TODO Auto-generated method stub
> }
>
> public void respond(RequestCycle r) {
> Response response = r.getResponse();
> response.setContentType("application/zip");
> OutputStream out=response.getOutputStream();
> ZipOutputStream zout = new ZipOutputStream(out);
>
> for (Entry f : getFileList()) {
> addFileToZip(zout, f);
> }
> try {
> zout.finish();
> zout.close();
> } catch (IOException e) {
> LOGGER.warn("IO exception while streaming ZIP file: ", e);
> }
>
> }});
>
>
> }
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
> For additional commands, e-mail: users-help@wicket.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
For additional commands, e-mail: users-help@wicket.apache.org
Re: "Streaming" a huge ZIP file
Posted by Gabor Szokoli <sz...@gmail.com>.
On Dec 12, 2007 7:23 PM, Igor Vaynberg <ig...@gmail.com> wrote:
> correct. only that one user will have to wait for the download to
> complete. if that is undesired then use a shared resource.
This is what we experience, and it is fine.
Good natural limit on the amount of load each user can cause by accident :-)
Here's Igors previous pseudocode in Java for the sake of completeness.
My question is: how do I set the downloaded filename now?
@Override
public void onSubmit() {
getRequestCycle().setRequestTarget(new IRequestTarget() {
public void detach(RequestCycle arg0) {
// TODO Auto-generated method stub
}
public void respond(RequestCycle r) {
Response response = r.getResponse();
response.setContentType("application/zip");
OutputStream out=response.getOutputStream();
ZipOutputStream zout = new ZipOutputStream(out);
for (Entry f : getFileList()) {
addFileToZip(zout, f);
}
try {
zout.finish();
zout.close();
} catch (IOException e) {
LOGGER.warn("IO exception while streaming ZIP file: ", e);
}
}});
}
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
For additional commands, e-mail: users-help@wicket.apache.org
Re: "Streaming" a huge ZIP file
Posted by Igor Vaynberg <ig...@gmail.com>.
correct. only that one user will have to wait for the download to
complete. if that is undesired then use a shared resource.
-igor
On Dec 12, 2007 12:18 AM, Gabor Szokoli <sz...@gmail.com> wrote:
> Thanks all!
>
> On Dec 12, 2007 8:48 AM, Thomas Singer <wi...@regnis.de> wrote:
> > IIRC, DownloadLink and Igor's anonymous class will lock the session, so if
> > you have one downloading over a very slow connection, other users will get a
> > timeout.
>
> I beleive I have separate sessions for each user, so only the same
> users other tabs would experience unresponsiveness, right?
>
> We'll test and report back soon.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
> For additional commands, e-mail: users-help@wicket.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
For additional commands, e-mail: users-help@wicket.apache.org
Re: "Streaming" a huge ZIP file
Posted by Gabor Szokoli <sz...@gmail.com>.
Thanks all!
On Dec 12, 2007 8:48 AM, Thomas Singer <wi...@regnis.de> wrote:
> IIRC, DownloadLink and Igor's anonymous class will lock the session, so if
> you have one downloading over a very slow connection, other users will get a
> timeout.
I beleive I have separate sessions for each user, so only the same
users other tabs would experience unresponsiveness, right?
We'll test and report back soon.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
For additional commands, e-mail: users-help@wicket.apache.org
Re: "Streaming" a huge ZIP file
Posted by Thomas Singer <wi...@regnis.de>.
IIRC, DownloadLink and Igor's anonymous class will lock the session, so if
you have one downloading over a very slow connection, other users will get a
timeout.
Cheers,
Tom
Igor Vaynberg wrote:
> getrequestcycle().setrequesttarget(new irequesttarget() {
> respond(response r) { outputstream out=r.getoutputstream(); stream
> your date into out; }}}
>
> -igor
>
>
> On Dec 11, 2007 2:52 PM, Gabor Szokoli <sz...@gmail.com> wrote:
>> I guess I forgot to mentionthe single most important aspect of my question:
>> The archive not present on the file system, it is dynamically
>> assembled on the fly from individual files.
>> (the whole point is downloading multiple files in one go.)
>>
>>
>> Gabor Szokoli
>>
>>
>> On Dec 11, 2007 11:45 PM, Jeremy Levy <je...@gmail.com> wrote:
>>> Gabor,
>>>
>>> Try DownloadLink
>>> http://wicket.sourceforge.net/apidocs/wicket/markup/html/link/DownloadLink.html
>>>
>>> J
>>>
>>>
>>> On Dec 11, 2007 5:39 PM, Gabor Szokoli <sz...@gmail.com> wrote:
>>>
>>>> Hi there,
>>>>
>>>> The downside of an easy-to-use framework is the influx of users with
>>>> little understanding of the underlying servlet technology, like me.
>>>> Observe:
>>>>
>>>> I'd like to provide a ZIP file (could be TAR, anything Windows PCs can
>>>> save and later extract from) as a download, one that is potentially
>>>> too big to be kept in memory in full.
>>>> So I guess I'd need to "stream" it in chunks, or as a java stream or
>>>> channel.
>>>> Can I do that from wicket?
>>>> Or does the servlet interface require a fully assembled response in
>>>> memory?
>>>>
>>>> Currently I assemble the ZIP file in a byte array in full, but it
>>>> feels awkward. Would extending WebResource fit the bill?
>>>>
>>>> ByteArrayOutputStream bout = new
>>>> ByteArrayOutputStream();
>>>> ZipOutputStream zout = new ZipOutputStream(bout);
>>>> // Add entries
>>>> zout.finish();
>>>> zout.close();
>>>>
>>>> Resource resource = new
>>>> ByteArrayResource("application/zip",
>>>> bout.toByteArray());
>>>> ResourceStreamRequestTarget target = new
>>>> ResourceStreamRequestTarget(resource.getResourceStream());
>>>> target.setFileName(new
>>>> SimpleDateFormat("yyyyMMddHHmmss").format(Calendar.getInstance
>>>> ().getTime())
>>>> + ".zip");
>>>> getRequestCycle().setRequestTarget(target);
>>>>
>>>>
>>>> All suggestions are welcome.
>>>>
>>>>
>>>> Gabor Szokoli
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
>>>> For additional commands, e-mail: users-help@wicket.apache.org
>>>>
>>>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
>> For additional commands, e-mail: users-help@wicket.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
> For additional commands, e-mail: users-help@wicket.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
For additional commands, e-mail: users-help@wicket.apache.org
Re: "Streaming" a huge ZIP file
Posted by Igor Vaynberg <ig...@gmail.com>.
getrequestcycle().setrequesttarget(new irequesttarget() {
respond(response r) { outputstream out=r.getoutputstream(); stream
your date into out; }}}
-igor
On Dec 11, 2007 2:52 PM, Gabor Szokoli <sz...@gmail.com> wrote:
> I guess I forgot to mentionthe single most important aspect of my question:
> The archive not present on the file system, it is dynamically
> assembled on the fly from individual files.
> (the whole point is downloading multiple files in one go.)
>
>
> Gabor Szokoli
>
>
> On Dec 11, 2007 11:45 PM, Jeremy Levy <je...@gmail.com> wrote:
> > Gabor,
> >
> > Try DownloadLink
> > http://wicket.sourceforge.net/apidocs/wicket/markup/html/link/DownloadLink.html
> >
> > J
> >
> >
> > On Dec 11, 2007 5:39 PM, Gabor Szokoli <sz...@gmail.com> wrote:
> >
> > > Hi there,
> > >
> > > The downside of an easy-to-use framework is the influx of users with
> > > little understanding of the underlying servlet technology, like me.
> > > Observe:
> > >
> > > I'd like to provide a ZIP file (could be TAR, anything Windows PCs can
> > > save and later extract from) as a download, one that is potentially
> > > too big to be kept in memory in full.
> > > So I guess I'd need to "stream" it in chunks, or as a java stream or
> > > channel.
> > > Can I do that from wicket?
> > > Or does the servlet interface require a fully assembled response in
> > > memory?
> > >
> > > Currently I assemble the ZIP file in a byte array in full, but it
> > > feels awkward. Would extending WebResource fit the bill?
> > >
> > > ByteArrayOutputStream bout = new
> > > ByteArrayOutputStream();
> > > ZipOutputStream zout = new ZipOutputStream(bout);
> > > // Add entries
> > > zout.finish();
> > > zout.close();
> > >
> > > Resource resource = new
> > > ByteArrayResource("application/zip",
> > > bout.toByteArray());
> > > ResourceStreamRequestTarget target = new
> > > ResourceStreamRequestTarget(resource.getResourceStream());
> > > target.setFileName(new
> > > SimpleDateFormat("yyyyMMddHHmmss").format(Calendar.getInstance
> > > ().getTime())
> > > + ".zip");
> > > getRequestCycle().setRequestTarget(target);
> > >
> > >
> > > All suggestions are welcome.
> > >
> > >
> > > Gabor Szokoli
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
> > > For additional commands, e-mail: users-help@wicket.apache.org
> > >
> > >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
> For additional commands, e-mail: users-help@wicket.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
For additional commands, e-mail: users-help@wicket.apache.org
Re: "Streaming" a huge ZIP file
Posted by Johan Compagner <jc...@gmail.com>.
But still youj could generate to a tmp file onm disk and stream that.
But you xan also generate on the fly and stream that direcly to a
response stream
On 12/11/07, Gabor Szokoli <sz...@gmail.com> wrote:
> I guess I forgot to mentionthe single most important aspect of my question:
> The archive not present on the file system, it is dynamically
> assembled on the fly from individual files.
> (the whole point is downloading multiple files in one go.)
>
>
> Gabor Szokoli
>
> On Dec 11, 2007 11:45 PM, Jeremy Levy <je...@gmail.com> wrote:
> > Gabor,
> >
> > Try DownloadLink
> >
> http://wicket.sourceforge.net/apidocs/wicket/markup/html/link/DownloadLink.html
> >
> > J
> >
> >
> > On Dec 11, 2007 5:39 PM, Gabor Szokoli <sz...@gmail.com> wrote:
> >
> > > Hi there,
> > >
> > > The downside of an easy-to-use framework is the influx of users with
> > > little understanding of the underlying servlet technology, like me.
> > > Observe:
> > >
> > > I'd like to provide a ZIP file (could be TAR, anything Windows PCs can
> > > save and later extract from) as a download, one that is potentially
> > > too big to be kept in memory in full.
> > > So I guess I'd need to "stream" it in chunks, or as a java stream or
> > > channel.
> > > Can I do that from wicket?
> > > Or does the servlet interface require a fully assembled response in
> > > memory?
> > >
> > > Currently I assemble the ZIP file in a byte array in full, but it
> > > feels awkward. Would extending WebResource fit the bill?
> > >
> > > ByteArrayOutputStream bout = new
> > > ByteArrayOutputStream();
> > > ZipOutputStream zout = new ZipOutputStream(bout);
> > > // Add entries
> > > zout.finish();
> > > zout.close();
> > >
> > > Resource resource = new
> > > ByteArrayResource("application/zip",
> > > bout.toByteArray());
> > > ResourceStreamRequestTarget target = new
> > > ResourceStreamRequestTarget(resource.getResourceStream());
> > > target.setFileName(new
> > > SimpleDateFormat("yyyyMMddHHmmss").format(Calendar.getInstance
> > > ().getTime())
> > > + ".zip");
> > > getRequestCycle().setRequestTarget(target);
> > >
> > >
> > > All suggestions are welcome.
> > >
> > >
> > > Gabor Szokoli
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
> > > For additional commands, e-mail: users-help@wicket.apache.org
> > >
> > >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
> For additional commands, e-mail: users-help@wicket.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
For additional commands, e-mail: users-help@wicket.apache.org
Re: "Streaming" a huge ZIP file
Posted by Gabor Szokoli <sz...@gmail.com>.
I guess I forgot to mentionthe single most important aspect of my question:
The archive not present on the file system, it is dynamically
assembled on the fly from individual files.
(the whole point is downloading multiple files in one go.)
Gabor Szokoli
On Dec 11, 2007 11:45 PM, Jeremy Levy <je...@gmail.com> wrote:
> Gabor,
>
> Try DownloadLink
> http://wicket.sourceforge.net/apidocs/wicket/markup/html/link/DownloadLink.html
>
> J
>
>
> On Dec 11, 2007 5:39 PM, Gabor Szokoli <sz...@gmail.com> wrote:
>
> > Hi there,
> >
> > The downside of an easy-to-use framework is the influx of users with
> > little understanding of the underlying servlet technology, like me.
> > Observe:
> >
> > I'd like to provide a ZIP file (could be TAR, anything Windows PCs can
> > save and later extract from) as a download, one that is potentially
> > too big to be kept in memory in full.
> > So I guess I'd need to "stream" it in chunks, or as a java stream or
> > channel.
> > Can I do that from wicket?
> > Or does the servlet interface require a fully assembled response in
> > memory?
> >
> > Currently I assemble the ZIP file in a byte array in full, but it
> > feels awkward. Would extending WebResource fit the bill?
> >
> > ByteArrayOutputStream bout = new
> > ByteArrayOutputStream();
> > ZipOutputStream zout = new ZipOutputStream(bout);
> > // Add entries
> > zout.finish();
> > zout.close();
> >
> > Resource resource = new
> > ByteArrayResource("application/zip",
> > bout.toByteArray());
> > ResourceStreamRequestTarget target = new
> > ResourceStreamRequestTarget(resource.getResourceStream());
> > target.setFileName(new
> > SimpleDateFormat("yyyyMMddHHmmss").format(Calendar.getInstance
> > ().getTime())
> > + ".zip");
> > getRequestCycle().setRequestTarget(target);
> >
> >
> > All suggestions are welcome.
> >
> >
> > Gabor Szokoli
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
> > For additional commands, e-mail: users-help@wicket.apache.org
> >
> >
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
For additional commands, e-mail: users-help@wicket.apache.org
Re: "Streaming" a huge ZIP file
Posted by Jeremy Levy <je...@gmail.com>.
Gabor,
Try DownloadLink
http://wicket.sourceforge.net/apidocs/wicket/markup/html/link/DownloadLink.html
J
On Dec 11, 2007 5:39 PM, Gabor Szokoli <sz...@gmail.com> wrote:
> Hi there,
>
> The downside of an easy-to-use framework is the influx of users with
> little understanding of the underlying servlet technology, like me.
> Observe:
>
> I'd like to provide a ZIP file (could be TAR, anything Windows PCs can
> save and later extract from) as a download, one that is potentially
> too big to be kept in memory in full.
> So I guess I'd need to "stream" it in chunks, or as a java stream or
> channel.
> Can I do that from wicket?
> Or does the servlet interface require a fully assembled response in
> memory?
>
> Currently I assemble the ZIP file in a byte array in full, but it
> feels awkward. Would extending WebResource fit the bill?
>
> ByteArrayOutputStream bout = new
> ByteArrayOutputStream();
> ZipOutputStream zout = new ZipOutputStream(bout);
> // Add entries
> zout.finish();
> zout.close();
>
> Resource resource = new
> ByteArrayResource("application/zip",
> bout.toByteArray());
> ResourceStreamRequestTarget target = new
> ResourceStreamRequestTarget(resource.getResourceStream());
> target.setFileName(new
> SimpleDateFormat("yyyyMMddHHmmss").format(Calendar.getInstance
> ().getTime())
> + ".zip");
> getRequestCycle().setRequestTarget(target);
>
>
> All suggestions are welcome.
>
>
> Gabor Szokoli
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
> For additional commands, e-mail: users-help@wicket.apache.org
>
>