You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@poi.apache.org by bu...@apache.org on 2019/01/22 20:35:24 UTC
[Bug 63100] New: Streaming data for browsers
https://bz.apache.org/bugzilla/show_bug.cgi?id=63100
Bug ID: 63100
Summary: Streaming data for browsers
Product: POI
Version: 3.17-FINAL
Hardware: PC
OS: Mac OS X 10.1
Status: NEW
Severity: normal
Priority: P2
Component: SXSSF
Assignee: dev@poi.apache.org
Reporter: matija.obreza@gmail.com
Target Milestone: ---
SXSSF works as designed and manages a small memory footprint when generating
large files from a database. But it only writes data to an output stream once
everything has been written to SXSSF. This is problematic when used in web
applications:
In our use case (our website allows users to generate Excel from the database),
generating the SXSSF on the server takes about 5 minutes. Most clients give up
within a minute (or the browser does it automatically), or the proxy times out
due to no data being sent. Some users also retry the download request. A new
request for download is initiated (while the server is busy generating the
SXSSF for a client that already gave up). This can potentially lead to DOS.
To work around this issue, I've implemented a super-streaming version of SXSSF,
a `SuperSXSSF`, that relies on `rowWriter` callback to generate row data.
With this approach our service is able to stream the generated Excel directly
to the client and, best of all, is terminated in case the user cancels the
download request.
The `SuperSXSSF` prevents both download timeouts and potential DOS, while
allowing developers all other XSSF actions (i.e. define styles) that don't take
much processing time.
Now what?
Modifications at:
https://gitlab.croptrust.org/genesys-pgr/genesys-server/tree/master/src/main/java/org/apache/poi/xssf/streaming
Use case:
https://gitlab.croptrust.org/genesys-pgr/genesys-server/blob/master/src/main/java/org/genesys2/server/service/impl/DownloadServiceImpl.java
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org
For additional commands, e-mail: dev-help@poi.apache.org
[Bug 63100] Streaming data for browsers
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=63100
--- Comment #2 from Matija Obreza <ma...@gmail.com> ---
(In reply to PJ Fanning from comment #1)
> Your project seems useful. Could you open source your own jar (ie publish it
> maven central)?
>
> We can link to your page from our
> https://poi.apache.org/related-projects.html
The changes are implemented directly in the project
https://gitlab.croptrust.org/genesys-pgr/genesys-server (Apache v2 licensed)
because it is much simpler to trick the Java classloader to use our `.class`
files vs making a whole new jar just for the few updates I needed.
Changes to original SXSSF code are at
https://gitlab.croptrust.org/genesys-pgr/genesys-server/commits/master/src/main/java/org/apache/poi/xssf/streaming,
specifically at
https://gitlab.croptrust.org/genesys-pgr/genesys-server/commit/bac27c01a997ff8cfc4352018639e685712f3136
I've been on git for long, how do I make a merge request to your code?
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org
For additional commands, e-mail: dev-help@poi.apache.org
[Bug 63100] Streaming data for browsers
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=63100
--- Comment #4 from PJ Fanning <fa...@yahoo.com> ---
You could fork https://github.com/apache/poi and submit a pull request there.
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org
For additional commands, e-mail: dev-help@poi.apache.org
[Bug 63100] Streaming data for browsers
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=63100
--- Comment #5 from Matija Obreza <ma...@gmail.com> ---
https://github.com/apache/poi/pull/141
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org
For additional commands, e-mail: dev-help@poi.apache.org
[Bug 63100] Streaming data for browsers
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=63100
--- Comment #1 from PJ Fanning <fa...@yahoo.com> ---
Your project seems useful. Could you open source your own jar (ie publish it
maven central)?
We can link to your page from our https://poi.apache.org/related-projects.html
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org
For additional commands, e-mail: dev-help@poi.apache.org
[Bug 63100] Streaming data for browsers
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=63100
Dominik Stadler <do...@gmx.at> changed:
What |Removed |Added
----------------------------------------------------------------------------
Severity|normal |enhancement
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org
For additional commands, e-mail: dev-help@poi.apache.org
[Bug 63100] Streaming data for browsers
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=63100
--- Comment #3 from Matija Obreza <ma...@gmail.com> ---
Our maven artifacts are in Central
http://central.maven.org/maven2/org/genesys-pgr/
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org
For additional commands, e-mail: dev-help@poi.apache.org
[Bug 63100] Streaming data for browsers
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=63100
PJ Fanning <fa...@yahoo.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|NEW |RESOLVED
Resolution|--- |FIXED
--- Comment #6 from PJ Fanning <fa...@yahoo.com> ---
DeferredGeneration example in poi-examples is based on this
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org
For additional commands, e-mail: dev-help@poi.apache.org