You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@oodt.apache.org by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org> on 2012/01/19 03:26:41 UTC
[jira] [Commented] (OODT-341) XMLPS should be able to stream large
results
[ https://issues.apache.org/jira/browse/OODT-341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13188885#comment-13188885 ]
jiraposter@reviews.apache.org commented on OODT-341:
----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3543/
-----------------------------------------------------------
Review request for oodt.
Summary
-------
See OODT-341: https://issues.apache.org/jira/browse/OODT-341
* CDEResult extends org.apache.oodt.xmlquery.Result
* CDEResult mimetype is always "text/plain"
* CDEResult size is always 0
* CDEResult inputstream is CDEResultInputStream
* CDEResultInputStream has IO methods and wraps a CDEResult
* CDEResult wraps a ResultSet and returns rows as Strings, applying MappingFuncs if a Mapping is provided, and appending constant fields if a List of CDEValues is provided.
* ProductQueryServlet relies on servlet container to handle Content-Length (and possibly Transfer-Encoding: chunked)
This addresses bug OODT-341.
https://issues.apache.org/jira/browse/OODT-341
Diffs
-----
trunk/grid/src/main/java/org/apache/oodt/grid/ProductQueryServlet.java 1183564
trunk/xmlps/pom.xml 1233127
trunk/xmlps/src/main/java/org/apache/oodt/xmlps/product/DBMSExecutor.java 1233127
trunk/xmlps/src/main/java/org/apache/oodt/xmlps/product/XMLPSProductHandler.java 1233127
trunk/xmlps/src/main/java/org/apache/oodt/xmlps/structs/CDEResult.java 1233127
trunk/xmlps/src/main/java/org/apache/oodt/xmlps/structs/CDEResultInputStream.java PRE-CREATION
trunk/xmlps/src/test/java/org/apache/oodt/xmlps/structs/TestCDEResult.java PRE-CREATION
trunk/xmlps/src/test/java/org/apache/oodt/xmlps/structs/TestCDEResultInputStream.java PRE-CREATION
Diff: https://reviews.apache.org/r/3543/diff
Testing
-------
Runs in Tomcat 7. I've used it at CHLA.
NOT tested in other app servers.
Added 2 unit tests: TestCDEResult and TestCDEResultInputStream, which both pass.
Thanks,
rickdn
> XMLPS should be able to stream large results
> --------------------------------------------
>
> Key: OODT-341
> URL: https://issues.apache.org/jira/browse/OODT-341
> Project: OODT
> Issue Type: Improvement
> Components: xmlps
> Affects Versions: 0.4
> Reporter: Ricky Nguyen
> Assignee: Chris A. Mattmann
> Fix For: 0.4
>
> Attachments: rickdn-grid-oodt-341.patch, rickdn-xmlps-oodt-341.patch, rickdn-xmlps-oodt-341_1.patch
>
>
> Currently, XMLPS stores *ALL* the rows of a ResultSet in a CDEResult object. In addition, this CDEResult is converted to a String for the HTTPResponse, nearly doubling the memory usage. With large results, heap space can easily run out, despite increasing max heap space for the servlet container (e.g. -Xmx1024m). XMLPS should be able to stream/chunk its results, taking into consideration the following:
> # ResultSets _represent_ an iterable collection of rows without actually _storing_ all the rows in memory
> # HTTPResponse#getOutputStream() offers a streaming response, where chunked transfer-encoding and content-length headers are managed automatically by the servlet container (such as Tomcat)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira