You are viewing a plain text version of this content. The canonical link for it is here.
Posted to derby-user@db.apache.org by Brian Peterson <pu...@verizon.net> on 2009/01/20 05:19:50 UTC
iterating over millions of rows
I have a big table, about 1 million rows, that I'm doing a simple "select *"
over. The table is depressingly simple, basically a big VARCHAR for bit data
that stores some serialized bytes. When I profile using VisualVM it seems
that it is spending most of its time in
org.apache.derby.impl.store.raw.data.RAFContainer4.readFully
If I remember correctly, this gets invoked something like 3000 times. Is
there anything I can do to speed up iterating over this table? It is taking
about 30s to iterate over the 1 million records, but I could have up to 25
million.
It is an embedded db using 10.4, JDK 1.6.0_07, running on a Windows XP SP2
machine. I have page size set to the max, 32K, and the page case size set to
6000 pages.
Is there anything I can do, or have I just run up against how fast Windows
can read off of the disk?
Brian
RE: iterating over millions of rows
Posted by de...@segel.com.
Did you try using an index?
> -----Original Message-----
> From: Brian Peterson [mailto:publicayers@verizon.net]
> Sent: Monday, January 19, 2009 10:20 PM
> To: 'Derby Discussion'
> Subject: iterating over millions of rows
>
> I have a big table, about 1 million rows, that I'm doing a simple "select
> *"
> over. The table is depressingly simple, basically a big VARCHAR for bit
> data
> that stores some serialized bytes. When I profile using VisualVM it seems
> that it is spending most of its time in
>
> org.apache.derby.impl.store.raw.data.RAFContainer4.readFully
>
> If I remember correctly, this gets invoked something like 3000 times. Is
> there anything I can do to speed up iterating over this table? It is
> taking
> about 30s to iterate over the 1 million records, but I could have up to 25
> million.
>
> It is an embedded db using 10.4, JDK 1.6.0_07, running on a Windows XP SP2
> machine. I have page size set to the max, 32K, and the page case size set
> to
> 6000 pages.
>
> Is there anything I can do, or have I just run up against how fast Windows
> can read off of the disk?
>
> Brian
>
>