You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Bing Li <lb...@gmail.com> on 2013/08/12 19:00:42 UTC
Performance Are Affected? - Table and Family
Dear all,
I have one additional question about table and family.
A table which has less families is faster than the one which has more
families if the amount of data they have is the same? Correct or not?
Is it a higher performance design to put fewer families into a table?
Thanks so much!
Best regards,
Bing
On Tue, Aug 13, 2013 at 12:31 AM, Stas Maksimov <ma...@gmail.com> wrote:
> Hi there,
>
> On your second point, I don't think column family can ever be an optional
> parameter, so I'm not sure this understanding is correct.
>
> Regards,
> Stas.
>
>
> On 12 August 2013 17:22, Bing Li <lb...@gmail.com> wrote:
>>
>> Hi, all,
>>
>> My understandings about HBase table and its family are as follows.
>>
>> 1) Each table can consist of multiple families;
>>
>> 2) When retrieving with SingleColumnValueFilter, if the family is
>> specified, other families contained in the same table are not
>> affected.
>>
>> Are these claims right? But I got a problem which conflicts with the
>> above understandings.
>>
>> In the following code, even though no any data in the family of
>> ContrivedNeighborStructure.NODE_NEIGHBOR_IN_GROUP_BASICS_FAMILY, the
>> for-loop runs many times if other families has the column of
>> ContrivedNeighborStructure.NODE_NEIGHBOR_IN_GROUP_DC_KEY_COLUMN.
>>
>> Is that normal in HBase? If so, I think it is not a good design. No
>> column overlaps must exist among the families of the same table?
>> Otherwise, retrieving the table must cause waste of scanning loops?
>>
>> Thanks so much!
>>
>> Best wishes,
>> Bing
>>
>> SingleColumnValueFilter dcKeyFilter = new
>>
>> SingleColumnValueFilter(ContrivedNeighborStructure.NODE_NEIGHBOR_IN_GROUP_BASICS_FAMILY,
>> ContrivedNeighborStructure.NODE_NEIGHBOR_IN_GROUP_DC_KEY_COLUMN,
>> CompareFilter.CompareOp.EQUAL, new SubstringComparator(dcKey));
>> Scan scan = new Scan();
>> scan.setFilter(dcKeyFilter);
>> scan.setCaching(Parameters.CACHING_SIZE);
>> scan.setBatch(Parameters.BATCHING_SIZE);
>>
>> String qualifier;
>> String hostNodeKey = SocialRole.NO_NODE_KEY;
>> String groupKey = SocialGroup.NO_GROUP_KEY;
>> int timingScale = TimingScale.NO_TIMING_SCALE;
>> String key;
>> try
>> {
>> ResultScanner scanner = this.neighborTable.getScanner(scan);
>> for (Result result : scanner)
>> {
>> for (KeyValue kv : result.raw())
>> {
>> qualifier = Bytes.toString(kv.getQualifier());
>> if
>>
>> (qualifier.equals(ContrivedNeighborStructure.NODE_NEIGHBOR_IN_GROUP_HOST_NODE_KEY_STRING_COLUMN))
>> {
>> hostNodeKey = Bytes.toString(kv.getValue());
>> }
>> else if
>>
>> (qualifier.equals(ContrivedNeighborStructure.NODE_NEIGHBOR_IN_GROUP_HOST_GROUP_KEY_STRING_COLUMN))
>> {
>> groupKey = Bytes.toString(kv.getValue());
>> }
>> else if
>>
>> (qualifier.equals(ContrivedNeighborStructure.NODE_NEIGHBOR_IN_GROUP_TIMING_SCALE_STRING_COLUMN))
>> {
>> timingScale = Bytes.toInt(kv.getValue());
>> }
>> }
>> if (!hostNodeKey.equals(SocialRole.NO_NODE_KEY) &&
>> !groupKey.equals(SocialGroup.NO_GROUP_KEY) && timingScale !=
>> TimingScale.NO_TIMING_SCALE)
>> {
>> key = Tools.GetKeyOfNode(hostNodeKey, groupKey,
>> timingScale);
>> if (!neighborMap.containsKey(key))
>> {
>> neighborMap.put(key, new
>> NodeNeighborInGroup(hostNodeKey, groupKey, timingScale));
>> }
>> }
>> hostNodeKey = SocialRole.NO_NODE_KEY;
>> groupKey = SocialGroup.NO_GROUP_KEY;
>> timingScale = TimingScale.NO_TIMING_SCALE;
>> }
>> }
>> catch (IOException e)
>> {
>> e.printStackTrace();
>> }
>
>
Re: Performance Are Affected? - Table and Family
Posted by Stas Maksimov <ma...@gmail.com>.
Hi Bing,
Generally it is not advised to have more than 2-3 column families, unless
you are using them absolutely separately from each other. Please see here:
http://hbase.apache.org/book/number.of.cfs.html
Thanks,
Stas
On 12 August 2013 18:00, Bing Li <lb...@gmail.com> wrote:
> Dear all,
>
> I have one additional question about table and family.
>
> A table which has less families is faster than the one which has more
> families if the amount of data they have is the same? Correct or not?
>
> Is it a higher performance design to put fewer families into a table?
>
> Thanks so much!
>
> Best regards,
> Bing
>
>
> On Tue, Aug 13, 2013 at 12:31 AM, Stas Maksimov <ma...@gmail.com>
> wrote:
> > Hi there,
> >
> > On your second point, I don't think column family can ever be an optional
> > parameter, so I'm not sure this understanding is correct.
> >
> > Regards,
> > Stas.
> >
> >
> > On 12 August 2013 17:22, Bing Li <lb...@gmail.com> wrote:
> >>
> >> Hi, all,
> >>
> >> My understandings about HBase table and its family are as follows.
> >>
> >> 1) Each table can consist of multiple families;
> >>
> >> 2) When retrieving with SingleColumnValueFilter, if the family is
> >> specified, other families contained in the same table are not
> >> affected.
> >>
> >> Are these claims right? But I got a problem which conflicts with the
> >> above understandings.
> >>
> >> In the following code, even though no any data in the family of
> >> ContrivedNeighborStructure.NODE_NEIGHBOR_IN_GROUP_BASICS_FAMILY, the
> >> for-loop runs many times if other families has the column of
> >> ContrivedNeighborStructure.NODE_NEIGHBOR_IN_GROUP_DC_KEY_COLUMN.
> >>
> >> Is that normal in HBase? If so, I think it is not a good design. No
> >> column overlaps must exist among the families of the same table?
> >> Otherwise, retrieving the table must cause waste of scanning loops?
> >>
> >> Thanks so much!
> >>
> >> Best wishes,
> >> Bing
> >>
> >> SingleColumnValueFilter dcKeyFilter = new
> >>
> >>
> SingleColumnValueFilter(ContrivedNeighborStructure.NODE_NEIGHBOR_IN_GROUP_BASICS_FAMILY,
> >> ContrivedNeighborStructure.NODE_NEIGHBOR_IN_GROUP_DC_KEY_COLUMN,
> >> CompareFilter.CompareOp.EQUAL, new SubstringComparator(dcKey));
> >> Scan scan = new Scan();
> >> scan.setFilter(dcKeyFilter);
> >> scan.setCaching(Parameters.CACHING_SIZE);
> >> scan.setBatch(Parameters.BATCHING_SIZE);
> >>
> >> String qualifier;
> >> String hostNodeKey = SocialRole.NO_NODE_KEY;
> >> String groupKey = SocialGroup.NO_GROUP_KEY;
> >> int timingScale = TimingScale.NO_TIMING_SCALE;
> >> String key;
> >> try
> >> {
> >> ResultScanner scanner = this.neighborTable.getScanner(scan);
> >> for (Result result : scanner)
> >> {
> >> for (KeyValue kv : result.raw())
> >> {
> >> qualifier = Bytes.toString(kv.getQualifier());
> >> if
> >>
> >>
> (qualifier.equals(ContrivedNeighborStructure.NODE_NEIGHBOR_IN_GROUP_HOST_NODE_KEY_STRING_COLUMN))
> >> {
> >> hostNodeKey = Bytes.toString(kv.getValue());
> >> }
> >> else if
> >>
> >>
> (qualifier.equals(ContrivedNeighborStructure.NODE_NEIGHBOR_IN_GROUP_HOST_GROUP_KEY_STRING_COLUMN))
> >> {
> >> groupKey = Bytes.toString(kv.getValue());
> >> }
> >> else if
> >>
> >>
> (qualifier.equals(ContrivedNeighborStructure.NODE_NEIGHBOR_IN_GROUP_TIMING_SCALE_STRING_COLUMN))
> >> {
> >> timingScale = Bytes.toInt(kv.getValue());
> >> }
> >> }
> >> if (!hostNodeKey.equals(SocialRole.NO_NODE_KEY) &&
> >> !groupKey.equals(SocialGroup.NO_GROUP_KEY) && timingScale !=
> >> TimingScale.NO_TIMING_SCALE)
> >> {
> >> key = Tools.GetKeyOfNode(hostNodeKey, groupKey,
> >> timingScale);
> >> if (!neighborMap.containsKey(key))
> >> {
> >> neighborMap.put(key, new
> >> NodeNeighborInGroup(hostNodeKey, groupKey, timingScale));
> >> }
> >> }
> >> hostNodeKey = SocialRole.NO_NODE_KEY;
> >> groupKey = SocialGroup.NO_GROUP_KEY;
> >> timingScale = TimingScale.NO_TIMING_SCALE;
> >> }
> >> }
> >> catch (IOException e)
> >> {
> >> e.printStackTrace();
> >> }
> >
> >
>