You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by David kerber <dc...@verizon.net> on 2007/12/20 19:01:01 UTC
Re: Help with java Lists - followup
Thanks for all the suggestions! Thanks to you guys and gals, a report
that had been taking 4 to 5 minutes to process is now taking less than
10 seconds to generate!
D
David kerber wrote:
> I'm running this app in a tomcat 5.5.x container,I am trying to figure
> out which form of a list is the best for what I need to do here:
>
> I need to have some kind of list or collection that I can search
> quickly for a specific entry, and then start stepping through the list
> item by item from that point. The items in the list are all of the
> same user-defined class (not primitives or pre-defined java classes).
> My current implementation just uses an iterator starting at the
> beginning, and just walking through until I find the item I want, and
> continuing to walk from there, but I'm running into performance issues
> with this.
>
> The individual lists aren't particularly big: single-digits to at
> most a couple hundred items, but I do this processing in a big outer
> loop which will need to scale up to a few thousand iterations of
> different instances of these lists. What these actually are is
> individual stores' inventory data, with each list being a list of
> their inventory data for a month or so, and I will eventually need to
> iterate over a couple thousand stores for reporting.
>
> If it seems like I'll never get reasonable speed this way, I could
> switch to calling all the stores' data from the database at once,
> making the lists huge, but only needing to load them once. However,
> this makes speed in searching the lists much more of an issue, and I
> don't know which way is going to give me the best overall performance
> for this report generation.
>
> Any suggestions are welcome, including suggestions for reworking the
> basic design...
>
> D
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Help with java Lists - followup
Posted by David kerber <dc...@verizon.net>.
Steve Ochani wrote:
> On 20 Dec 2007 at 13:01, David kerber wrote:
>
>
>> Thanks for all the suggestions! Thanks to you guys and gals, a
>> report
>> that had been taking 4 to 5 minutes to process is now taking less
>> than
>> 10 seconds to generate!
>>
>>
>
> Hi,
>
> What was the basic overall solution you ended up with?
>
>
> -Steve O.
>
The fundamental change was using nested HashMaps of a couple different
types, so I had several objects like:
HashMap<Integer, SortedMap<Integer, Volume>> volumesHashHash;
Where Volume is one of my classes that stores the various data fields I
need, the first Integer is the PKey of the tank we are reporting on so I
can keep it in sync with other HashMaps storing other info, and the 2nd
Integer is a sequential index for which day I'm on for each tank. There
is a separate plain hashmap of <Integer, Integer> that tracks the
current index for each tank. If I were using java 6, I would have
built-in navigation methods so I wouldn't need to handle the indexes
separately, but this method wasn't difficult to work with. I know from
this short description it seems like I could have just done a sequential
scan through my data to generate the report, but there's more to it than
I feel like describing here, and these constructs are being used in a
base class which is extended for generating a wide variety of reports
from the same data, so I needed to build in some more versatility than
each report alone would require.
I also did quite a bit of flattening of my code to eliminate several
levels of abstraction, which allowed me efficiently to load the hash
maps from the database with a single scan through the resultset.
I didn't benchmark the various changes separately so I don't know which
ones gave me the biggest gains, but the overall result worked out great.
D
>
>
>
>
>> D
>>
>>
>> David kerber wrote:
>>
>>> I'm running this app in a tomcat 5.5.x container,I am trying to
>>>
>> figure
>>
>>> out which form of a list is the best for what I need to do here:
>>>
>>> I need to have some kind of list or collection that I can search
>>> quickly for a specific entry, and then start stepping through the
>>>
>> list
>>
>>> item by item from that point. The items in the list are all of
>>>
>> the
>>
>>> same user-defined class (not primitives or pre-defined java
>>>
>> classes).
>>
>>> My current implementation just uses an iterator starting at the
>>> beginning, and just walking through until I find the item I want,
>>>
>> and
>>
>>> continuing to walk from there, but I'm running into performance
>>>
>> issues
>>
>>> with this.
>>>
>>> The individual lists aren't particularly big: single-digits to at
>>> most a couple hundred items, but I do this processing in a big
>>>
>> outer
>>
>>> loop which will need to scale up to a few thousand iterations of
>>> different instances of these lists. What these actually are is
>>> individual stores' inventory data, with each list being a list of
>>> their inventory data for a month or so, and I will eventually need
>>>
>> to
>>
>>> iterate over a couple thousand stores for reporting.
>>>
>>> If it seems like I'll never get reasonable speed this way, I could
>>> switch to calling all the stores' data from the database at once,
>>> making the lists huge, but only needing to load them once.
>>>
>> However,
>>
>>> this makes speed in searching the lists much more of an issue, and
>>>
>> I
>>
>>> don't know which way is going to give me the best overall
>>>
>> performance
>>
>>> for this report generation.
>>>
>>> Any suggestions are welcome, including suggestions for reworking
>>>
>> the
>>
>>> basic design...
>>>
>>> D
>>>
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Help with java Lists - followup
Posted by Steve Ochani <oc...@ncc.edu>.
On 20 Dec 2007 at 13:01, David kerber wrote:
> Thanks for all the suggestions! Thanks to you guys and gals, a
> report
> that had been taking 4 to 5 minutes to process is now taking less
> than
> 10 seconds to generate!
>
Hi,
What was the basic overall solution you ended up with?
-Steve O.
> D
>
>
> David kerber wrote:
> > I'm running this app in a tomcat 5.5.x container,I am trying to
> figure
> > out which form of a list is the best for what I need to do here:
> >
> > I need to have some kind of list or collection that I can search
> > quickly for a specific entry, and then start stepping through the
> list
> > item by item from that point. The items in the list are all of
> the
> > same user-defined class (not primitives or pre-defined java
> classes).
> > My current implementation just uses an iterator starting at the
> > beginning, and just walking through until I find the item I want,
> and
> > continuing to walk from there, but I'm running into performance
> issues
> > with this.
> >
> > The individual lists aren't particularly big: single-digits to at
> > most a couple hundred items, but I do this processing in a big
> outer
> > loop which will need to scale up to a few thousand iterations of
> > different instances of these lists. What these actually are is
> > individual stores' inventory data, with each list being a list of
> > their inventory data for a month or so, and I will eventually need
> to
> > iterate over a couple thousand stores for reporting.
> >
> > If it seems like I'll never get reasonable speed this way, I could
> > switch to calling all the stores' data from the database at once,
> > making the lists huge, but only needing to load them once.
> However,
> > this makes speed in searching the lists much more of an issue, and
> I
> > don't know which way is going to give me the best overall
> performance
> > for this report generation.
> >
> > Any suggestions are welcome, including suggestions for reworking
> the
> > basic design...
> >
> > D
>
>
>
> --------------------------------------------------------------------
> -
> To start a new topic, e-mail: users@tomcat.apache.org
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org