You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ibatis.apache.org by Corneliu Rudeanu <cr...@yahoo.com> on 2005/09/13 18:54:34 UTC
Patch submission
Hi!
I made a patch for SQLMapClient which allows reusal of ResultSet's
while iterating through a paginated query list. How can I submit the
patch?
A more detailed description: the current implementation is re-running
the query when user asks for some page, skipping the results
(resultSet.next()) until the requested page is read. This is a problem
while iterating through large results, and in my opinion that is the
exact purpose for having paginated queries. My patch keeps the
ResultSet cached in the PaginatedList, and invokes next()'s when user
asks for next page. When a page "from the past" (i.e. going to a page
with lower index than current - 1) the query runs again, with a
behaviour similar to current implementation. Note: the optimization
with next+previous pages being cached was kept in my patch.
I have little experience with SVN. Please send some instructions on how
to submit the patch in order to be evaluated.
Regards,
Corneliu Rudeanu
______________________________________________________
Yahoo! for Good
Donate to the Hurricane Katrina relief effort.
http://store.yahoo.com/redcross-donate3/
Re: Patch submission
Posted by Larry Meadors <la...@gmail.com>.
You could put it in JIRA as a feature request, and attach it there.
http://issues.apache.org/jira/browse/IBATIS
However, I suspect that most people will not want this behavior,
because it will tie up database resources..
Larry
On 9/13/05, Corneliu Rudeanu <cr...@yahoo.com> wrote:
>
>
> Hi!
>
> I made a patch for SQLMapClient which allows reusal of ResultSet's
> while iterating through a paginated query list. How can I submit the
> patch?
>
> A more detailed description: the current implementation is re-running
> the query when user asks for some page, skipping the results
> (resultSet.next()) until the requested page is read. This is a problem
> while iterating through large results, and in my opinion that is the
> exact purpose for having paginated queries. My patch keeps the
> ResultSet cached in the PaginatedList, and invokes next()'s when user
> asks for next page. When a page "from the past" (i.e. going to a page
> with lower index than current - 1) the query runs again, with a
> behaviour similar to current implementation. Note: the optimization
> with next+previous pages being cached was kept in my patch.
>
> I have little experience with SVN. Please send some instructions on how
> to submit the patch in order to be evaluated.
>
> Regards,
> Corneliu Rudeanu
>
>
>
>
> ______________________________________________________
> Yahoo! for Good
> Donate to the Hurricane Katrina relief effort.
> http://store.yahoo.com/redcross-donate3/
>
>
Re: Patch submission
Posted by Corneliu Rudeanu <cr...@yahoo.com>.
RowHandler is a solution, but not exactly what I've been looking for.
Given another API which expects an Iterator over the dataset (like "I
need to call BigStore.loadfrom(Iterator<DataItem>)") it is quite
difficult to make the conversion from a RowHandler being called
repeatedly to an Iterator. Not impossible, indeed: a limited length
queue and some thread pumping events in it should do the trick.
Another project constraint is to use released third party modules only,
so yes, I will have to implement the trick using RowHandler (and not to
use the patch I was talking below).
Thanks for the ideea,
Corneliu Rudeanu
--- Larry Meadors <la...@gmail.com> wrote:
> Oh, why didn't you say so in the first place? ;-)
>
> Use a RowHandler...or better still a stored procedure.
>
> Larry
>
>
> On 9/14/05, Corneliu Rudeanu <cr...@yahoo.com> wrote:
> >
> > Maybe I should have described the problem that determined me to
> write
> > the patch in the first place. I use iBATIS in a project where I
> have to
> > fetch some 4 million rows and process them. Using queryForList is
> not a
> > good ideea, since it would load all data in memory when returning
> the
> > results. I need to iterate the results, not to have them all at
> once in
> > memory.
> >
> > One dirty trick I applied was to get a java.sql.Connection from
> > iBATIS's sqlMapClient
> > (sqlMapClient.openSession().getCurrentConnection()) and solve the
> > iteration by hand using JDBC code. I don't consider this a solution
> and
> > I would need iBATIS to offer such a "queryForIterable" method. If
> > queryForPaginated list should remain as now, would you agree to the
> new
> > (iterable) method to be exposed by SQLMapClient?
> >
> > Best regards,
> > Corneliu Rudeanu
> >
> >
> > --- Brandon Goodin <br...@gmail.com> wrote:
> >
> > > Tying up connections in the PaginatedArrayList would not be a
> > > desireable functionality to me. I would recommend you write a
> version
> > > of PaginatedArrayList for your own personal use. If you are
> handling
> > > large resultsets I would recommend against the use of
> > > PaginatedArrayList anyways. It is meant for small result
> scrolling.
> > >
> > > Brandon
> > >
> > > On 9/13/05, Corneliu Rudeanu <cr...@yahoo.com> wrote:
> > > >
> > > >
> > > > Hi!
> > > >
> > > > I made a patch for SQLMapClient which allows reusal of
> ResultSet's
> > > > while iterating through a paginated query list. How can I
> submit
> > > the
> > > > patch?
> > > >
> > > > A more detailed description: the current implementation is
> > > re-running
> > > > the query when user asks for some page, skipping the results
> > > > (resultSet.next()) until the requested page is read. This is a
> > > problem
> > > > while iterating through large results, and in my opinion that
> is
> > > the
> > > > exact purpose for having paginated queries. My patch keeps the
> > > > ResultSet cached in the PaginatedList, and invokes next()'s
> when
> > > user
> > > > asks for next page. When a page "from the past" (i.e. going to
> a
> > > page
> > > > with lower index than current - 1) the query runs again, with a
> > > > behaviour similar to current implementation. Note: the
> optimization
> > > > with next+previous pages being cached was kept in my patch.
> > > >
> > > > I have little experience with SVN. Please send some
> instructions on
> > > how
> > > > to submit the patch in order to be evaluated.
> > > >
> > > > Regards,
> > > > Corneliu Rudeanu
> > > >
> > > >
> > > >
> > > >
> > > > ______________________________________________________
> > > > Yahoo! for Good
> > > > Donate to the Hurricane Katrina relief effort.
> > > > http://store.yahoo.com/redcross-donate3/
> > > >
> > > >
> > >
> >
> >
> > __________________________________________________
> > Do You Yahoo!?
> > Tired of spam? Yahoo! Mail has the best spam protection around
> > http://mail.yahoo.com
> >
>
__________________________________
Yahoo! Mail - PC Magazine Editors' Choice 2005
http://mail.yahoo.com
Re: Patch submission
Posted by Larry Meadors <la...@gmail.com>.
Oh, why didn't you say so in the first place? ;-)
Use a RowHandler...or better still a stored procedure.
Larry
On 9/14/05, Corneliu Rudeanu <cr...@yahoo.com> wrote:
>
> Maybe I should have described the problem that determined me to write
> the patch in the first place. I use iBATIS in a project where I have to
> fetch some 4 million rows and process them. Using queryForList is not a
> good ideea, since it would load all data in memory when returning the
> results. I need to iterate the results, not to have them all at once in
> memory.
>
> One dirty trick I applied was to get a java.sql.Connection from
> iBATIS's sqlMapClient
> (sqlMapClient.openSession().getCurrentConnection()) and solve the
> iteration by hand using JDBC code. I don't consider this a solution and
> I would need iBATIS to offer such a "queryForIterable" method. If
> queryForPaginated list should remain as now, would you agree to the new
> (iterable) method to be exposed by SQLMapClient?
>
> Best regards,
> Corneliu Rudeanu
>
>
> --- Brandon Goodin <br...@gmail.com> wrote:
>
> > Tying up connections in the PaginatedArrayList would not be a
> > desireable functionality to me. I would recommend you write a version
> > of PaginatedArrayList for your own personal use. If you are handling
> > large resultsets I would recommend against the use of
> > PaginatedArrayList anyways. It is meant for small result scrolling.
> >
> > Brandon
> >
> > On 9/13/05, Corneliu Rudeanu <cr...@yahoo.com> wrote:
> > >
> > >
> > > Hi!
> > >
> > > I made a patch for SQLMapClient which allows reusal of ResultSet's
> > > while iterating through a paginated query list. How can I submit
> > the
> > > patch?
> > >
> > > A more detailed description: the current implementation is
> > re-running
> > > the query when user asks for some page, skipping the results
> > > (resultSet.next()) until the requested page is read. This is a
> > problem
> > > while iterating through large results, and in my opinion that is
> > the
> > > exact purpose for having paginated queries. My patch keeps the
> > > ResultSet cached in the PaginatedList, and invokes next()'s when
> > user
> > > asks for next page. When a page "from the past" (i.e. going to a
> > page
> > > with lower index than current - 1) the query runs again, with a
> > > behaviour similar to current implementation. Note: the optimization
> > > with next+previous pages being cached was kept in my patch.
> > >
> > > I have little experience with SVN. Please send some instructions on
> > how
> > > to submit the patch in order to be evaluated.
> > >
> > > Regards,
> > > Corneliu Rudeanu
> > >
> > >
> > >
> > >
> > > ______________________________________________________
> > > Yahoo! for Good
> > > Donate to the Hurricane Katrina relief effort.
> > > http://store.yahoo.com/redcross-donate3/
> > >
> > >
> >
>
>
> __________________________________________________
> Do You Yahoo!?
> Tired of spam? Yahoo! Mail has the best spam protection around
> http://mail.yahoo.com
>
Re: Patch submission
Posted by Corneliu Rudeanu <cr...@yahoo.com>.
Maybe I should have described the problem that determined me to write
the patch in the first place. I use iBATIS in a project where I have to
fetch some 4 million rows and process them. Using queryForList is not a
good ideea, since it would load all data in memory when returning the
results. I need to iterate the results, not to have them all at once in
memory.
One dirty trick I applied was to get a java.sql.Connection from
iBATIS's sqlMapClient
(sqlMapClient.openSession().getCurrentConnection()) and solve the
iteration by hand using JDBC code. I don't consider this a solution and
I would need iBATIS to offer such a "queryForIterable" method. If
queryForPaginated list should remain as now, would you agree to the new
(iterable) method to be exposed by SQLMapClient?
Best regards,
Corneliu Rudeanu
--- Brandon Goodin <br...@gmail.com> wrote:
> Tying up connections in the PaginatedArrayList would not be a
> desireable functionality to me. I would recommend you write a version
> of PaginatedArrayList for your own personal use. If you are handling
> large resultsets I would recommend against the use of
> PaginatedArrayList anyways. It is meant for small result scrolling.
>
> Brandon
>
> On 9/13/05, Corneliu Rudeanu <cr...@yahoo.com> wrote:
> >
> >
> > Hi!
> >
> > I made a patch for SQLMapClient which allows reusal of ResultSet's
> > while iterating through a paginated query list. How can I submit
> the
> > patch?
> >
> > A more detailed description: the current implementation is
> re-running
> > the query when user asks for some page, skipping the results
> > (resultSet.next()) until the requested page is read. This is a
> problem
> > while iterating through large results, and in my opinion that is
> the
> > exact purpose for having paginated queries. My patch keeps the
> > ResultSet cached in the PaginatedList, and invokes next()'s when
> user
> > asks for next page. When a page "from the past" (i.e. going to a
> page
> > with lower index than current - 1) the query runs again, with a
> > behaviour similar to current implementation. Note: the optimization
> > with next+previous pages being cached was kept in my patch.
> >
> > I have little experience with SVN. Please send some instructions on
> how
> > to submit the patch in order to be evaluated.
> >
> > Regards,
> > Corneliu Rudeanu
> >
> >
> >
> >
> > ______________________________________________________
> > Yahoo! for Good
> > Donate to the Hurricane Katrina relief effort.
> > http://store.yahoo.com/redcross-donate3/
> >
> >
>
__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com
Re: Patch submission
Posted by Brandon Goodin <br...@gmail.com>.
Tying up connections in the PaginatedArrayList would not be a
desireable functionality to me. I would recommend you write a version
of PaginatedArrayList for your own personal use. If you are handling
large resultsets I would recommend against the use of
PaginatedArrayList anyways. It is meant for small result scrolling.
Brandon
On 9/13/05, Corneliu Rudeanu <cr...@yahoo.com> wrote:
>
>
> Hi!
>
> I made a patch for SQLMapClient which allows reusal of ResultSet's
> while iterating through a paginated query list. How can I submit the
> patch?
>
> A more detailed description: the current implementation is re-running
> the query when user asks for some page, skipping the results
> (resultSet.next()) until the requested page is read. This is a problem
> while iterating through large results, and in my opinion that is the
> exact purpose for having paginated queries. My patch keeps the
> ResultSet cached in the PaginatedList, and invokes next()'s when user
> asks for next page. When a page "from the past" (i.e. going to a page
> with lower index than current - 1) the query runs again, with a
> behaviour similar to current implementation. Note: the optimization
> with next+previous pages being cached was kept in my patch.
>
> I have little experience with SVN. Please send some instructions on how
> to submit the patch in order to be evaluated.
>
> Regards,
> Corneliu Rudeanu
>
>
>
>
> ______________________________________________________
> Yahoo! for Good
> Donate to the Hurricane Katrina relief effort.
> http://store.yahoo.com/redcross-donate3/
>
>