You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user-java@ibatis.apache.org by Tomáš Procházka <t....@centrum.cz> on 2010/02/18 16:16:15 UTC
Re: read large result set in Ibatis3
Hi.
I investigated this more detailed and ResultHandler is not what I neeed :-( I need keep jdbc statement open and get new item when I need. ResultHandler read all item and for every item call handleResult() I can handle when next item will be read.
Ideal will be possibility to add method to mapper will return object on which I can call something like getNext(), hasNext() and close(). Statement will be open and Ibatis will be read and convert to object only one Object for every getNext call.
It something like this possible in actual version?
-------------------------- Original message --------------------------
From: Clinton Begin <cl...@gmail.com>
Subject: read large result set in Ibatis3
Date: Sunday, January 17, 2010, 10:54:34 PM
Attachments: Zpráva.html
msgid:16178eb11001171354i56ee636ehdef2384f6919403a@mail.gmail.com
C> That's correct. At this time, you have to call it through sqlSession.
C> I don't think it would be too hard to add mapper support. If it's
C> really important for you, then file a Jira ticket and I'll see what I can do.
C> Clinton
C> 2010/1/17 Tomáš Procházka <t....@centrum.cz>
C> Yes.
C> And I found ResultHandler only in this method:
C> void select
C> (String statement, Object parameter, RowBounds rowBounds,
C> ResultHandler handler)
C> which is directly in SqlSession. No possibilities to use ResultHandler in mapper class.
C> ______________________________________________________________
>> Od: "Clinton Begin" <cl...@gmail.com>
>> Komu: user-java@ibatis.apache.org
>> Datum: 17.01.2010 15:40
>> Předmět: Re: read large result set in Ibatis3
>>
>>Have you read the documentation?
>>
>>2010/1/17 Tomáš Procházka <t....@centrum.cz>
>>
>>>
>>> Thanks you for reply.
>>> I need read data only from one table (queue), so I have no accociation or
>>> collection mapping.
>>>
>>> But what I must do?
>>>
>>> Is this sufficient:
>>>
>>> @Select({"SELECT * FROM send)
>>> List<Send> getAllItems();
>>> ?
>>>
>>> Or I must do anything else, like use some special class instead of List?
>>>
>>> ______________________________________________________________
>>> > Od: "Clinton Begin" <cl...@gmail.com>
>>> > Komu: user-java@ibatis.apache.org
>>> > Datum: 17.01.2010 01:11
>>> > Předmět: Re: read large result set in Ibatis3
>>> >
>>> >Yes, you can use ResultHAndler in iBATIS 3. The key to know is that the
>>> >large result set will be fine if the query is as simple as you've
>>> >presented. However, if you use associations or collections mappings, then
>>> >you could run into memory issues.
>>> >
>>> >Cheers,
>>> >Clinton
>>> >
>>> >2010/1/16 Tomáš Procházka <t....@centrum.cz>
>>> >
>>> >>
>>> >> Hi.
>>> >>
>>> >> I need read large result set from DB (like 300 000 rows).
>>> >>
>>> >> It's possible use this:
>>> >>
>>> >> @Select({"SELECT * FROM send)
>>> >> List<Send> getAllItems();
>>> >>
>>> >> ?
>>> >>
>>> >> but instead of read all rows to List use server cursor and read only
>>> items
>>> >> which will be accessed by Iterator.
>>> >>
>>> >> I found some old article about this problem like:
>>> >>
>>> >>
>>> >>
>>> http://stackoverflow.com/questions/1344362/java-retrieving-large-amounts-of-data-from-a-db-using-ibatis
>>> >>
>>> >> But I don't know, that this is for iBatis 3.0.
>>> >>
>>> >> Exist same ResultHandler implementation that can do this?
>>> >>
>>> >>
>>> >> --
>>> >> Tomáš Procházka
>>> >>
>>> >> ---------------------------------------------------------------------
>>> >> To unsubscribe, e-mail: user-java-unsubscribe@ibatis.apache.org
>>> >> For additional commands, e-mail: user-java-help@ibatis.apache.org
>>> >>
>>> >>
>>> >
>>> >
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: user-java-unsubscribe@ibatis.apache.org
>>> For additional commands, e-mail: user-java-help@ibatis.apache.org
>>>
>>>
>>
>>
C> ---------------------------------------------------------------------
C> To unsubscribe, e-mail: user-java-unsubscribe@ibatis.apache.org
C> For additional commands, e-mail: user-java-help@ibatis.apache.org
----------------------- End of original message ----------------------
--
Tomas Prochazka
--------------------------- Původní zpráva ---------------------------
Odesilatel: Clinton Begin <cl...@gmail.com>
Předmět: read large result set in Ibatis3
Datum: 17. ledna 2010, 14:54:34 (GMT -0700)
Přílohy: Zpráva.html
C> That's correct. At this time, you have to call it through sqlSession.
C> I don't think it would be too hard to add mapper support. If it's
C> really important for you, then file a Jira ticket and I'll see what I can do.
C> Clinton
C> 2010/1/17 Tomáš Procházka <t....@centrum.cz>
C> Yes.
C> And I found ResultHandler only in this method:
C> void select
C> (String statement, Object parameter, RowBounds rowBounds,
C> ResultHandler handler)
C> which is directly in SqlSession. No possibilities to use ResultHandler in mapper class.
C> ______________________________________________________________
>> Od: "Clinton Begin" <cl...@gmail.com>
>> Komu: user-java@ibatis.apache.org
>> Datum: 17.01.2010 15:40
>> Předmět: Re: read large result set in Ibatis3
>>Have you read the documentation?
>>2010/1/17 Tomáš Procházka <t....@centrum.cz>
>>> Thanks you for reply.
>>> I need read data only from one table (queue), so I have no accociation or
>>> collection mapping.
>>> But what I must do?
>>> Is this sufficient:
>>> @Select({"SELECT * FROM send)
>>> List<Send> getAllItems();
>>> ?
>>> Or I must do anything else, like use some special class instead of List?
>>> ______________________________________________________________
>>> > Od: "Clinton Begin" <cl...@gmail.com>
>>> > Komu: user-java@ibatis.apache.org
>>> > Datum: 17.01.2010 01:11
>>> > Předmět: Re: read large result set in Ibatis3
>>> >
>>> >Yes, you can use ResultHAndler in iBATIS 3. The key to know is that the
>>> >large result set will be fine if the query is as simple as you've
>>> >presented. However, if you use associations or collections mappings, then
>>> >you could run into memory issues.
>>> >
>>> >Cheers,
>>> >Clinton
>>> >
>>> >2010/1/16 Tomáš Procházka <t....@centrum.cz>
>>> >
>>> >>
>>> >> Hi.
>>> >>
>>> >> I need read large result set from DB (like 300 000 rows).
>>> >>
>>> >> It's possible use this:
>>> >>
>>> >> @Select({"SELECT * FROM send)
>>> >> List<Send> getAllItems();
>>> >>
>>> >> ?
>>> >>
>>> >> but instead of read all rows to List use server cursor and read only
>>> items
>>> >> which will be accessed by Iterator.
>>> >>
>>> >> I found some old article about this problem like:
>>> >>
>>> >>
>>> >>
>>> http://stackoverflow.com/questions/1344362/java-retrieving-large-amounts-of-data-from-a-db-using-ibatis
>>> >>
>>> >> But I don't know, that this is for iBatis 3.0.
>>> >>
>>> >> Exist same ResultHandler implementation that can do this?
>>> >>
>>> >>
>>> >> --
>>> >> Tomáš Procházka
>>> >>
>>> >> ---------------------------------------------------------------------
>>> >> To unsubscribe, e-mail: user-java-unsubscribe@ibatis.apache.org
>>> >> For additional commands, e-mail: user-java-help@ibatis.apache.org
>>> >>
>>> >>
>>> >
>>> >
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: user-java-unsubscribe@ibatis.apache.org
>>> For additional commands, e-mail: user-java-help@ibatis.apache.org
C> ---------------------------------------------------------------------
C> To unsubscribe, e-mail: user-java-unsubscribe@ibatis.apache.org
C> For additional commands, e-mail: user-java-help@ibatis.apache.org
------------------------ Konec původní zprávy ------------------------
--
Tomáš Procházka
---------------------------------------------------------------------
To unsubscribe, e-mail: user-java-unsubscribe@ibatis.apache.org
For additional commands, e-mail: user-java-help@ibatis.apache.org
Re: read large result set in Ibatis3
Posted by Clinton Begin <cl...@gmail.com>.
Ibatis shouldn't block that way. Can you confirm that it's not a table lock?
On 2010-02-24, Tomáš Procházka <t....@centrum.cz> wrote:
>
> I don't need expose JDBC. I only continuously read large table as queue and
> process every item.
>
> I already used ResultHandler and ArrayBlockingQueue in handleResult(). If
> the ArrayBlockingQueue is full reading is stoped until consuper thread
> process all items.
>
> But when ArrayBlockingQueue block reading I can't call another iBatis
> request. Ibatis are blocked during whole query processing with
> ResultHandler? I don't use the same Db.getSession();
>
> ______________________________________________________________
>> Od: "Guy Rouillier" <gu...@burntmail.com>
>> Komu: user-java@ibatis.apache.org
>> Datum: 18.02.2010 18:45
>> Předmět: Re: read large result set in Ibatis3
>>
>>On 2/18/2010 10:16 AM, Tomáš Procházka wrote:
>>> Ideal will be possibility to add method to mapper will return object
>>> on which I can call something like getNext(), hasNext() and close().
>>> Statement will be open and Ibatis will be read and convert to object
>>> only one Object for every getNext call.
>>
>>That is very JDBC-ish. From my perspective, iBATIS attempts to hide all
>> that so that the programmer can just deal with lists of objects.
>> ResultHandler lets you change how the list gets built. What you are
>> asking is for JDBC to be exposed again, something like ResultSetHandler.
>> I guess it would be possible to make that pluggable, but then you are
>> undoing much of the framework.
>>
>>-- Guy Rouillier
>>
>>---------------------------------------------------------------------
>>To unsubscribe, e-mail: user-java-unsubscribe@ibatis.apache.org
>>For additional commands, e-mail: user-java-help@ibatis.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-java-unsubscribe@ibatis.apache.org
> For additional commands, e-mail: user-java-help@ibatis.apache.org
>
>
--
Sent from my mobile device
---------------------------------------------------------------------
To unsubscribe, e-mail: user-java-unsubscribe@ibatis.apache.org
For additional commands, e-mail: user-java-help@ibatis.apache.org
Re: read large result set in Ibatis3
Posted by Tomáš Procházka <t....@centrum.cz>.
I don't need expose JDBC. I only continuously read large table as queue and process every item.
I already used ResultHandler and ArrayBlockingQueue in handleResult(). If the ArrayBlockingQueue is full reading is stoped until consuper thread process all items.
But when ArrayBlockingQueue block reading I can't call another iBatis request. Ibatis are blocked during whole query processing with ResultHandler? I don't use the same Db.getSession();
______________________________________________________________
> Od: "Guy Rouillier" <gu...@burntmail.com>
> Komu: user-java@ibatis.apache.org
> Datum: 18.02.2010 18:45
> Předmět: Re: read large result set in Ibatis3
>
>On 2/18/2010 10:16 AM, Tomáš Procházka wrote:
>> Ideal will be possibility to add method to mapper will return object
>> on which I can call something like getNext(), hasNext() and close().
>> Statement will be open and Ibatis will be read and convert to object
>> only one Object for every getNext call.
>
>That is very JDBC-ish. From my perspective, iBATIS attempts to hide all that so that the programmer can just deal with lists of objects. ResultHandler lets you change how the list gets built. What you are asking is for JDBC to be exposed again, something like ResultSetHandler. I guess it would be possible to make that pluggable, but then you are undoing much of the framework.
>
>-- Guy Rouillier
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: user-java-unsubscribe@ibatis.apache.org
>For additional commands, e-mail: user-java-help@ibatis.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-java-unsubscribe@ibatis.apache.org
For additional commands, e-mail: user-java-help@ibatis.apache.org
Re: read large result set in Ibatis3
Posted by Martin Ellis <el...@gmail.com>.
Hi Davide,
Did you see my other posts on this subject?
http://www.mail-archive.com/user-java@ibatis.apache.org/msg15588.html
http://www.mail-archive.com/user-java@ibatis.apache.org/msg15589.html
Perhaps that's what you're after?
Martin
On 25 February 2010 10:02, Davide Rogora <dr...@unionefiduciaria.it> wrote:
> I agree with Martin.
> In my company we use iBatis and we appreciate both result mapping and dynamic SQL features, but sometimes we only need dynamic SQL with direct access to the underlying jdbc cursor.
> For instance... we use JasperReports for report generation (and dynamic queries are very useful with reports), but we cannot use iBatis because we have memory issues (and bad performances) for large reports; it would be nice if we could call a dynamic query and pass the underlying jdbc resultset to JasperReports.
>
> Davide.
>
>
>
> -----Messaggio originale-----
> Da: Martin Ellis [mailto:ellis.m.a@gmail.com]
> Inviato: giovedì 18 febbraio 2010 19.27
> A: user-java@ibatis.apache.org
> Oggetto: Re: read large result set in Ibatis3
>
>
>> On 2/18/2010 10:16 AM, Tomáš Procházka wrote:
>>> Ideal will be possibility to add method to mapper will return object
>>> on which I can call something like getNext(), hasNext() and close().
>
> On 18 February 2010 17:44, Guy Rouillier <gu...@burntmail.com> wrote:
>> That is very JDBC-ish. From my perspective, iBATIS attempts to hide
>> all that so that the programmer can just deal with lists of objects.
>> ResultHandler lets you change how the list gets built. What you are
>> asking is for JDBC to be exposed again, something like
>> ResultSetHandler. I guess it would be possible to make that
>> pluggable, but then you are undoing much of the framework.
>
> Only the result handling side of the framework. You'd still have the dynamic SQL, and parameter handling without it. That in itself is probably enough incentive to use iBATIS.
>
> Martin
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-java-unsubscribe@ibatis.apache.org
> For additional commands, e-mail: user-java-help@ibatis.apache.org
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-java-unsubscribe@ibatis.apache.org
> For additional commands, e-mail: user-java-help@ibatis.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-java-unsubscribe@ibatis.apache.org
For additional commands, e-mail: user-java-help@ibatis.apache.org
Article: Scala and Ibatis 3.0
Posted by "Eduardo M. Cavalcanti" <ec...@eversystems.com.br>.
Hello,
Vasil Remeniuk has published this article:
http://vasilrem.com/blog/software-development/using-ibatis-3-0-with-scala/
Regards.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-java-unsubscribe@ibatis.apache.org
For additional commands, e-mail: user-java-help@ibatis.apache.org
R: read large result set in Ibatis3
Posted by Davide Rogora <dr...@unionefiduciaria.it>.
I agree with Martin.
In my company we use iBatis and we appreciate both result mapping and dynamic SQL features, but sometimes we only need dynamic SQL with direct access to the underlying jdbc cursor.
For instance... we use JasperReports for report generation (and dynamic queries are very useful with reports), but we cannot use iBatis because we have memory issues (and bad performances) for large reports; it would be nice if we could call a dynamic query and pass the underlying jdbc resultset to JasperReports.
Davide.
-----Messaggio originale-----
Da: Martin Ellis [mailto:ellis.m.a@gmail.com]
Inviato: giovedì 18 febbraio 2010 19.27
A: user-java@ibatis.apache.org
Oggetto: Re: read large result set in Ibatis3
> On 2/18/2010 10:16 AM, Tomáš Procházka wrote:
>> Ideal will be possibility to add method to mapper will return object
>> on which I can call something like getNext(), hasNext() and close().
On 18 February 2010 17:44, Guy Rouillier <gu...@burntmail.com> wrote:
> That is very JDBC-ish. From my perspective, iBATIS attempts to hide
> all that so that the programmer can just deal with lists of objects.
> ResultHandler lets you change how the list gets built. What you are
> asking is for JDBC to be exposed again, something like
> ResultSetHandler. I guess it would be possible to make that
> pluggable, but then you are undoing much of the framework.
Only the result handling side of the framework. You'd still have the dynamic SQL, and parameter handling without it. That in itself is probably enough incentive to use iBATIS.
Martin
---------------------------------------------------------------------
To unsubscribe, e-mail: user-java-unsubscribe@ibatis.apache.org
For additional commands, e-mail: user-java-help@ibatis.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: user-java-unsubscribe@ibatis.apache.org
For additional commands, e-mail: user-java-help@ibatis.apache.org
Re: read large result set in Ibatis3
Posted by Martin Ellis <el...@gmail.com>.
> On 2/18/2010 10:16 AM, Tomáš Procházka wrote:
>> Ideal will be possibility to add method to mapper will return object
>> on which I can call something like getNext(), hasNext() and close().
On 18 February 2010 17:44, Guy Rouillier <gu...@burntmail.com> wrote:
> That is very JDBC-ish. From my perspective, iBATIS attempts to hide all
> that so that the programmer can just deal with lists of objects.
> ResultHandler lets you change how the list gets built. What you are asking
> is for JDBC to be exposed again, something like ResultSetHandler. I guess
> it would be possible to make that pluggable, but then you are undoing much
> of the framework.
Only the result handling side of the framework. You'd still have the
dynamic SQL, and parameter handling without it. That in itself is
probably enough incentive to use iBATIS.
Martin
---------------------------------------------------------------------
To unsubscribe, e-mail: user-java-unsubscribe@ibatis.apache.org
For additional commands, e-mail: user-java-help@ibatis.apache.org
Re: read large result set in Ibatis3
Posted by Guy Rouillier <gu...@burntmail.com>.
On 2/18/2010 10:16 AM, Tomáš Procházka wrote:
> Ideal will be possibility to add method to mapper will return object
> on which I can call something like getNext(), hasNext() and close().
> Statement will be open and Ibatis will be read and convert to object
> only one Object for every getNext call.
That is very JDBC-ish. From my perspective, iBATIS attempts to hide all
that so that the programmer can just deal with lists of objects.
ResultHandler lets you change how the list gets built. What you are
asking is for JDBC to be exposed again, something like ResultSetHandler.
I guess it would be possible to make that pluggable, but then you are
undoing much of the framework.
--
Guy Rouillier
---------------------------------------------------------------------
To unsubscribe, e-mail: user-java-unsubscribe@ibatis.apache.org
For additional commands, e-mail: user-java-help@ibatis.apache.org