You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by Ted Yu <yu...@gmail.com> on 2011/03/16 17:30:57 UTC
reduce copying of HRegionInfo's
Hi,
Please refer to 'One of the regionserver aborted, then the master shut down
itself' on user list for background.
I looked at calls to this method:
public void assignUserRegions(List<HRegionInfo> regions, List<HServerInfo>
servers)
Here is one from AssignmentManager:
assignUserRegions(Arrays.asList(regions.keySet().toArray(new
HRegionInfo[0])), servers);
Here is another from Master:
this.assignmentManager.assignUserRegions(Arrays.asList(newRegions),
servers);
I propose changing the first parameter to HRegionInfo[] so that extra
copying can be avoided - considering that the number of regions in
production is high.
A bigger task is to reduce memory footprint of AssignmentManager.
I will present my further findings.
Thanks
Re: reduce copying of HRegionInfo's
Posted by Ted Yu <yu...@gmail.com>.
Patch has been uploaded to https://issues.apache.org/jira/browse/HBASE-3657
On Wed, Mar 16, 2011 at 11:12 AM, Ted Yu <yu...@gmail.com> wrote:
> It would eliminate this copy:
>
> 165 public static <T> List<T> asList(T... array) {
> 166 return new ArrayList<T>(array);
> 167 }
> 168
>
> On Wed, Mar 16, 2011 at 10:27 AM, Stack <st...@duboce.net> wrote:
>
>> Will that eliminate copying Ted? If so, go for it (Thanks for looking
>> into this).
>> St.Ack
>>
>> On Wed, Mar 16, 2011 at 9:30 AM, Ted Yu <yu...@gmail.com> wrote:
>> > Hi,
>> > Please refer to 'One of the regionserver aborted, then the master shut
>> down
>> > itself' on user list for background.
>> >
>> > I looked at calls to this method:
>> > public void assignUserRegions(List<HRegionInfo> regions,
>> List<HServerInfo>
>> > servers)
>> > Here is one from AssignmentManager:
>> > assignUserRegions(Arrays.asList(regions.keySet().toArray(new
>> > HRegionInfo[0])), servers);
>> > Here is another from Master:
>> > this.assignmentManager.assignUserRegions(Arrays.asList(newRegions),
>> > servers);
>> >
>> > I propose changing the first parameter to HRegionInfo[] so that extra
>> > copying can be avoided - considering that the number of regions in
>> > production is high.
>> >
>> > A bigger task is to reduce memory footprint of AssignmentManager.
>> >
>> > I will present my further findings.
>> >
>> > Thanks
>> >
>>
>
>
Re: reduce copying of HRegionInfo's
Posted by Ted Yu <yu...@gmail.com>.
It would eliminate this copy:
165 public static <T> List<T> asList(T... array) {
166 return new ArrayList<T>(array);
167 }
168
On Wed, Mar 16, 2011 at 10:27 AM, Stack <st...@duboce.net> wrote:
> Will that eliminate copying Ted? If so, go for it (Thanks for looking
> into this).
> St.Ack
>
> On Wed, Mar 16, 2011 at 9:30 AM, Ted Yu <yu...@gmail.com> wrote:
> > Hi,
> > Please refer to 'One of the regionserver aborted, then the master shut
> down
> > itself' on user list for background.
> >
> > I looked at calls to this method:
> > public void assignUserRegions(List<HRegionInfo> regions,
> List<HServerInfo>
> > servers)
> > Here is one from AssignmentManager:
> > assignUserRegions(Arrays.asList(regions.keySet().toArray(new
> > HRegionInfo[0])), servers);
> > Here is another from Master:
> > this.assignmentManager.assignUserRegions(Arrays.asList(newRegions),
> > servers);
> >
> > I propose changing the first parameter to HRegionInfo[] so that extra
> > copying can be avoided - considering that the number of regions in
> > production is high.
> >
> > A bigger task is to reduce memory footprint of AssignmentManager.
> >
> > I will present my further findings.
> >
> > Thanks
> >
>
Re: reduce copying of HRegionInfo's
Posted by Stack <st...@duboce.net>.
Will that eliminate copying Ted? If so, go for it (Thanks for looking
into this).
St.Ack
On Wed, Mar 16, 2011 at 9:30 AM, Ted Yu <yu...@gmail.com> wrote:
> Hi,
> Please refer to 'One of the regionserver aborted, then the master shut down
> itself' on user list for background.
>
> I looked at calls to this method:
> public void assignUserRegions(List<HRegionInfo> regions, List<HServerInfo>
> servers)
> Here is one from AssignmentManager:
> assignUserRegions(Arrays.asList(regions.keySet().toArray(new
> HRegionInfo[0])), servers);
> Here is another from Master:
> this.assignmentManager.assignUserRegions(Arrays.asList(newRegions),
> servers);
>
> I propose changing the first parameter to HRegionInfo[] so that extra
> copying can be avoided - considering that the number of regions in
> production is high.
>
> A bigger task is to reduce memory footprint of AssignmentManager.
>
> I will present my further findings.
>
> Thanks
>