You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Bing Li <lb...@gmail.com> on 2013/08/12 19:00:42 UTC

Performance Are Affected? - Table and Family

Dear all,

I have one additional question about table and family.

A table which has less families is faster than the one which has more
families if the amount of data they have is the same? Correct or not?

Is it a higher performance design to put fewer families into a table?

Thanks so much!

Best regards,
Bing


On Tue, Aug 13, 2013 at 12:31 AM, Stas Maksimov <ma...@gmail.com> wrote:
> Hi there,
>
> On your second point, I don't think column family can ever be an optional
> parameter, so I'm not sure this understanding is correct.
>
> Regards,
> Stas.
>
>
> On 12 August 2013 17:22, Bing Li <lb...@gmail.com> wrote:
>>
>> Hi, all,
>>
>> My understandings about HBase table and its family are as follows.
>>
>> 1) Each table can consist of multiple families;
>>
>> 2) When retrieving with SingleColumnValueFilter, if the family is
>> specified, other families contained in the same table are not
>> affected.
>>
>> Are these claims right? But I got a problem which conflicts with the
>> above understandings.
>>
>> In the following code, even though no any data in the family of
>> ContrivedNeighborStructure.NODE_NEIGHBOR_IN_GROUP_BASICS_FAMILY, the
>> for-loop runs many times if other families has the column of
>> ContrivedNeighborStructure.NODE_NEIGHBOR_IN_GROUP_DC_KEY_COLUMN.
>>
>> Is that normal in HBase? If so, I think it is not a good design. No
>> column overlaps must exist among the families of the same table?
>> Otherwise, retrieving the table must cause waste of scanning loops?
>>
>> Thanks so much!
>>
>> Best wishes,
>> Bing
>>
>>         SingleColumnValueFilter dcKeyFilter = new
>>
>> SingleColumnValueFilter(ContrivedNeighborStructure.NODE_NEIGHBOR_IN_GROUP_BASICS_FAMILY,
>> ContrivedNeighborStructure.NODE_NEIGHBOR_IN_GROUP_DC_KEY_COLUMN,
>> CompareFilter.CompareOp.EQUAL, new SubstringComparator(dcKey));
>>         Scan scan = new Scan();
>>         scan.setFilter(dcKeyFilter);
>>         scan.setCaching(Parameters.CACHING_SIZE);
>>         scan.setBatch(Parameters.BATCHING_SIZE);
>>
>>         String qualifier;
>>         String hostNodeKey = SocialRole.NO_NODE_KEY;
>>         String groupKey = SocialGroup.NO_GROUP_KEY;
>>         int timingScale = TimingScale.NO_TIMING_SCALE;
>>         String key;
>>         try
>>         {
>>             ResultScanner scanner = this.neighborTable.getScanner(scan);
>>             for (Result result : scanner)
>>             {
>>                 for (KeyValue kv : result.raw())
>>                 {
>>                     qualifier = Bytes.toString(kv.getQualifier());
>>                     if
>>
>> (qualifier.equals(ContrivedNeighborStructure.NODE_NEIGHBOR_IN_GROUP_HOST_NODE_KEY_STRING_COLUMN))
>>                     {
>>                         hostNodeKey = Bytes.toString(kv.getValue());
>>                     }
>>                     else if
>>
>> (qualifier.equals(ContrivedNeighborStructure.NODE_NEIGHBOR_IN_GROUP_HOST_GROUP_KEY_STRING_COLUMN))
>>                     {
>>                         groupKey = Bytes.toString(kv.getValue());
>>                     }
>>                     else if
>>
>> (qualifier.equals(ContrivedNeighborStructure.NODE_NEIGHBOR_IN_GROUP_TIMING_SCALE_STRING_COLUMN))
>>                     {
>>                         timingScale = Bytes.toInt(kv.getValue());
>>                     }
>>                 }
>>                 if (!hostNodeKey.equals(SocialRole.NO_NODE_KEY) &&
>> !groupKey.equals(SocialGroup.NO_GROUP_KEY) && timingScale !=
>> TimingScale.NO_TIMING_SCALE)
>>                 {
>>                     key = Tools.GetKeyOfNode(hostNodeKey, groupKey,
>> timingScale);
>>                     if (!neighborMap.containsKey(key))
>>                     {
>>                         neighborMap.put(key, new
>> NodeNeighborInGroup(hostNodeKey, groupKey, timingScale));
>>                     }
>>                 }
>>                 hostNodeKey = SocialRole.NO_NODE_KEY;
>>                 groupKey = SocialGroup.NO_GROUP_KEY;
>>                 timingScale = TimingScale.NO_TIMING_SCALE;
>>             }
>>         }
>>         catch (IOException e)
>>         {
>>             e.printStackTrace();
>>         }
>
>

Re: Performance Are Affected? - Table and Family

Posted by Stas Maksimov <ma...@gmail.com>.
Hi Bing,

Generally it is not advised to have more than 2-3 column families, unless
you are using them absolutely separately from each other. Please see here:
http://hbase.apache.org/book/number.of.cfs.html

Thanks,
Stas

On 12 August 2013 18:00, Bing Li <lb...@gmail.com> wrote:

> Dear all,
>
> I have one additional question about table and family.
>
> A table which has less families is faster than the one which has more
> families if the amount of data they have is the same? Correct or not?
>
> Is it a higher performance design to put fewer families into a table?
>
> Thanks so much!
>
> Best regards,
> Bing
>
>
> On Tue, Aug 13, 2013 at 12:31 AM, Stas Maksimov <ma...@gmail.com>
> wrote:
> > Hi there,
> >
> > On your second point, I don't think column family can ever be an optional
> > parameter, so I'm not sure this understanding is correct.
> >
> > Regards,
> > Stas.
> >
> >
> > On 12 August 2013 17:22, Bing Li <lb...@gmail.com> wrote:
> >>
> >> Hi, all,
> >>
> >> My understandings about HBase table and its family are as follows.
> >>
> >> 1) Each table can consist of multiple families;
> >>
> >> 2) When retrieving with SingleColumnValueFilter, if the family is
> >> specified, other families contained in the same table are not
> >> affected.
> >>
> >> Are these claims right? But I got a problem which conflicts with the
> >> above understandings.
> >>
> >> In the following code, even though no any data in the family of
> >> ContrivedNeighborStructure.NODE_NEIGHBOR_IN_GROUP_BASICS_FAMILY, the
> >> for-loop runs many times if other families has the column of
> >> ContrivedNeighborStructure.NODE_NEIGHBOR_IN_GROUP_DC_KEY_COLUMN.
> >>
> >> Is that normal in HBase? If so, I think it is not a good design. No
> >> column overlaps must exist among the families of the same table?
> >> Otherwise, retrieving the table must cause waste of scanning loops?
> >>
> >> Thanks so much!
> >>
> >> Best wishes,
> >> Bing
> >>
> >>         SingleColumnValueFilter dcKeyFilter = new
> >>
> >>
> SingleColumnValueFilter(ContrivedNeighborStructure.NODE_NEIGHBOR_IN_GROUP_BASICS_FAMILY,
> >> ContrivedNeighborStructure.NODE_NEIGHBOR_IN_GROUP_DC_KEY_COLUMN,
> >> CompareFilter.CompareOp.EQUAL, new SubstringComparator(dcKey));
> >>         Scan scan = new Scan();
> >>         scan.setFilter(dcKeyFilter);
> >>         scan.setCaching(Parameters.CACHING_SIZE);
> >>         scan.setBatch(Parameters.BATCHING_SIZE);
> >>
> >>         String qualifier;
> >>         String hostNodeKey = SocialRole.NO_NODE_KEY;
> >>         String groupKey = SocialGroup.NO_GROUP_KEY;
> >>         int timingScale = TimingScale.NO_TIMING_SCALE;
> >>         String key;
> >>         try
> >>         {
> >>             ResultScanner scanner = this.neighborTable.getScanner(scan);
> >>             for (Result result : scanner)
> >>             {
> >>                 for (KeyValue kv : result.raw())
> >>                 {
> >>                     qualifier = Bytes.toString(kv.getQualifier());
> >>                     if
> >>
> >>
> (qualifier.equals(ContrivedNeighborStructure.NODE_NEIGHBOR_IN_GROUP_HOST_NODE_KEY_STRING_COLUMN))
> >>                     {
> >>                         hostNodeKey = Bytes.toString(kv.getValue());
> >>                     }
> >>                     else if
> >>
> >>
> (qualifier.equals(ContrivedNeighborStructure.NODE_NEIGHBOR_IN_GROUP_HOST_GROUP_KEY_STRING_COLUMN))
> >>                     {
> >>                         groupKey = Bytes.toString(kv.getValue());
> >>                     }
> >>                     else if
> >>
> >>
> (qualifier.equals(ContrivedNeighborStructure.NODE_NEIGHBOR_IN_GROUP_TIMING_SCALE_STRING_COLUMN))
> >>                     {
> >>                         timingScale = Bytes.toInt(kv.getValue());
> >>                     }
> >>                 }
> >>                 if (!hostNodeKey.equals(SocialRole.NO_NODE_KEY) &&
> >> !groupKey.equals(SocialGroup.NO_GROUP_KEY) && timingScale !=
> >> TimingScale.NO_TIMING_SCALE)
> >>                 {
> >>                     key = Tools.GetKeyOfNode(hostNodeKey, groupKey,
> >> timingScale);
> >>                     if (!neighborMap.containsKey(key))
> >>                     {
> >>                         neighborMap.put(key, new
> >> NodeNeighborInGroup(hostNodeKey, groupKey, timingScale));
> >>                     }
> >>                 }
> >>                 hostNodeKey = SocialRole.NO_NODE_KEY;
> >>                 groupKey = SocialGroup.NO_GROUP_KEY;
> >>                 timingScale = TimingScale.NO_TIMING_SCALE;
> >>             }
> >>         }
> >>         catch (IOException e)
> >>         {
> >>             e.printStackTrace();
> >>         }
> >
> >
>