HBase GET operation max row size - partial results

[Veerraju Tadimeti <tv...@gmail.com>]

Hi,

If i use GET operation, is there any chance of getting partial result? If
Yes, under what circumstances.  Is there any way to reproduce it?

I am using GET operation in my coProcessor ( to the same region), adding
the resut to the List<Result> .  I am afraid that any chance  of partial
result when using GET operation, since GET uses SCAN operation internally.

Thanks,
Raju,
(972)273-0155.

Re: HBase GET operation max row size - partial results

[Yu Li <ca...@gmail.com>]

AFAIK there's no max result size or partial result for get request. If we
add such feature in future, we will add release note in JIRA.
(Actually we have implemented such limit in our customized version and it
requires CP to correctly handle it, we may upstream the feature later)

Some more detailed information at code level:

When saying "get uses scan internally", we mean it reuse the scan logic in
HRegion class. But at the rpc service level in RSRpcServices, there're two
different methods (get and scan) for these two kinds of requests, and
currently you'll only find below max result size limit in scan code path:

{code}

    long maxResultSize;

    if (scanner.getMaxResultSize() > 0) {

      maxResultSize = Math.min(scanner.getMaxResultSize(),
maxQuotaResultSize);

    } else {

      maxResultSize = maxQuotaResultSize;

    }

    ...

        ScannerContext.Builder contextBuilder = ScannerContext.newBuilder(
true);

        // maxResultSize - either we can reach this much size for all
cells(being read) data or sum

        // of heap size occupied by cells(being read). Cell data means its
key and value parts.

        contextBuilder.setSizeLimit(sizeScope, maxResultSize, maxResultSize
);

        ...

        ScannerContext scannerContext = contextBuilder.build();

        while (numOfResults < maxResults) {

          ...

          moreRows = scanner.nextRaw(values, scannerContext);

         ...
{code}

Hope it helps.

Best Regards,
Yu

On 27 July 2017 at 09:54, Anoop John <an...@gmail.com> wrote:

> You mean within your RegionObserver you are doing the get?  Within
> which hook?   What is the way you are doing the get?   Can u paste
> that sample code.
>
> -Anoop-
>
> On Wed, Jul 26, 2017 at 8:02 PM, Veerraju Tadimeti <tv...@gmail.com>
> wrote:
> > Hi,
> >
> > If i use GET operation, is there any chance of getting partial result? If
> > Yes, under what circumstances.  Is there any way to reproduce it?
> >
> > I am using GET operation in my coProcessor ( to the same region), adding
> > the resut to the List<Result> .  I am afraid that any chance  of partial
> > result when using GET operation, since GET uses SCAN operation
> internally.
> >
> > Thanks,
> > Raju,
> > (972)273-0155.
>

Re: HBase GET operation max row size - partial results

[Anoop John <an...@gmail.com>]

You mean within your RegionObserver you are doing the get?  Within
which hook?   What is the way you are doing the get?   Can u paste
that sample code.

-Anoop-

On Wed, Jul 26, 2017 at 8:02 PM, Veerraju Tadimeti <tv...@gmail.com> wrote:
> Hi,
>
> If i use GET operation, is there any chance of getting partial result? If
> Yes, under what circumstances.  Is there any way to reproduce it?
>
> I am using GET operation in my coProcessor ( to the same region), adding
> the resut to the List<Result> .  I am afraid that any chance  of partial
> result when using GET operation, since GET uses SCAN operation internally.
>
> Thanks,
> Raju,
> (972)273-0155.