You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@commons.apache.org by "Michael Osipov (JIRA)" <ji...@apache.org> on 2015/08/12 20:49:45 UTC

[jira] [Created] (DBUTILS-126) Add iterable interface approach

Michael Osipov created DBUTILS-126:
--------------------------------------

             Summary: Add iterable interface approach
                 Key: DBUTILS-126
                 URL: https://issues.apache.org/jira/browse/DBUTILS-126
             Project: Commons DbUtils
          Issue Type: New Feature
    Affects Versions: 1.5
            Reporter: Michael Osipov


Currently, DbUtils allows to load all data at once into memory. In many cases, when dealing with 100 000+ records, this will cause an OOME. I have to iterate over 100 000 lines with 10 columns and retrieve subdata which are in turn indexed with Lucene.

Because I cannot iterate over the entire dataset without memory problems, I have to retrieve them pagewise. (fiddling with rownums in Oracle).

An iterator interface which streams this down from the DB to the client would solve such a problem, very much like [Spring JdbcTemplate extension that returns iterators|https://github.com/alexkasko/springjdbc-iterable]. This works very well in a Spring app downloading a table with two million records directly to the HTTP client without any buffers.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)