Question about reading data very recently written to hbase

[Mike Thomsen <mi...@gmail.com>]

My team has a set of web services that read data from HBase and prepare it
to be exported as a report. The first call is an AJAX call that reads all
of the requested rows, generates the report pieces and returns a JSON map
to the calling web app saying what level of success it had in building up
the data for the report. If the data can be exported, it then display a
link to another web service that builds the report and sends it as a file
download to the user.

In the first call, it's pretty quick between reading the data and
immediately writing a copy of the report data to a table in HBase. So the
turn around time between the two web service calls is usually about five
seconds or less.

We sometimes get errors back from HBase on the first attempt because it
appears to have not committed the data from the Java client API to the
server. Something like that. It can't immediately find the data we just
generated. If we wait a few more seconds and try it again, everything works
just right. It just appears to be the case that the data isn't flushed into
a state where it's accessible yet.

To be frank, we're HBase novices and we're probably missing something
pretty simple and obvious. Is this to be expected? If so, does anyone have
a recommendation for how we can mitigate this I/O behavior?

Thanks,

Mike

Re: Question about reading data very recently written to hbase

[Mike Thomsen <mi...@gmail.com>]

I'm almost certain we're not doing that. Thanks for that!

On Mon, Oct 5, 2015 at 4:48 PM, Vladimir Rodionov <vl...@gmail.com>
wrote:

> HBase writes are consistent.  Writes are available immediately only after
> table's flush on a client side.
>  (HTable.flushCommits())
>
> -Vlad
>
>
>
> On Mon, Oct 5, 2015 at 1:30 PM, Mike Thomsen <mi...@gmail.com>
> wrote:
>
> > My team has a set of web services that read data from HBase and prepare
> it
> > to be exported as a report. The first call is an AJAX call that reads all
> > of the requested rows, generates the report pieces and returns a JSON map
> > to the calling web app saying what level of success it had in building up
> > the data for the report. If the data can be exported, it then display a
> > link to another web service that builds the report and sends it as a file
> > download to the user.
> >
> > In the first call, it's pretty quick between reading the data and
> > immediately writing a copy of the report data to a table in HBase. So the
> > turn around time between the two web service calls is usually about five
> > seconds or less.
> >
> > We sometimes get errors back from HBase on the first attempt because it
> > appears to have not committed the data from the Java client API to the
> > server. Something like that. It can't immediately find the data we just
> > generated. If we wait a few more seconds and try it again, everything
> works
> > just right. It just appears to be the case that the data isn't flushed
> into
> > a state where it's accessible yet.
> >
> > To be frank, we're HBase novices and we're probably missing something
> > pretty simple and obvious. Is this to be expected? If so, does anyone
> have
> > a recommendation for how we can mitigate this I/O behavior?
> >
> > Thanks,
> >
> > Mike
> >
>

Re: Question about reading data very recently written to hbase

[Vladimir Rodionov <vl...@gmail.com>]

HBase writes are consistent.  Writes are available immediately only after
table's flush on a client side.
 (HTable.flushCommits())

-Vlad



On Mon, Oct 5, 2015 at 1:30 PM, Mike Thomsen <mi...@gmail.com> wrote:

> My team has a set of web services that read data from HBase and prepare it
> to be exported as a report. The first call is an AJAX call that reads all
> of the requested rows, generates the report pieces and returns a JSON map
> to the calling web app saying what level of success it had in building up
> the data for the report. If the data can be exported, it then display a
> link to another web service that builds the report and sends it as a file
> download to the user.
>
> In the first call, it's pretty quick between reading the data and
> immediately writing a copy of the report data to a table in HBase. So the
> turn around time between the two web service calls is usually about five
> seconds or less.
>
> We sometimes get errors back from HBase on the first attempt because it
> appears to have not committed the data from the Java client API to the
> server. Something like that. It can't immediately find the data we just
> generated. If we wait a few more seconds and try it again, everything works
> just right. It just appears to be the case that the data isn't flushed into
> a state where it's accessible yet.
>
> To be frank, we're HBase novices and we're probably missing something
> pretty simple and obvious. Is this to be expected? If so, does anyone have
> a recommendation for how we can mitigate this I/O behavior?
>
> Thanks,
>
> Mike
>