HTable and streaming

[Simon Majou <si...@majou.org>]

Hi,

I don't see any methods returning streams in the Java client HTable :
http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/HTable.html

That means we need to use the thrift client to get streams ?

Simon

Re: HTable and streaming

[Asaf Mesika <as...@gmail.com>]

What do you mean by indirect blocks?


On Tue, Jun 4, 2013 at 7:22 AM, Mohit Anchlia <mo...@gmail.com>wrote:

> Better approach would be to break the data in chunks and create a behaviour
> similar to indirect blocks.
>
> On Mon, Jun 3, 2013 at 9:12 PM, Asaf Mesika <as...@gmail.com> wrote:
>
> > I guess one can hack opening a socket from a Coprocessor Endpoint and
> push
> > its scanned data, thus achieving a stream.
> >
> >
> > On Sun, Jun 2, 2013 at 12:42 AM, Stack <st...@duboce.net> wrote:
> >
> > > Yeah, no streaming API in our current client (nor does our thrift
> client
> > > give you a streaming API).
> > > St.Ack
> > >
> > >
> > > On Sat, Jun 1, 2013 at 8:21 AM, Simon Majou <si...@majou.org> wrote:
> > >
> > > > No I don't want to scan a table, I want a stream of one result. In
> case
> > > for
> > > > example of big records.
> > > >
> > > > With thrift it is up to the client to deal with the response, so in
> > > theory
> > > > you can build a client which returns streams.
> > > >
> > > > But when I look at the current implementation of thrift for node for
> > > > instance (https://github.com/apache/thrift/tree/trunk/lib/nodejs),
> it
> > > > sends
> > > > back only full formed results (as the Java client).
> > > >
> > > >
> > > > Simon
> > > >
> > > >
> > > > On Sat, Jun 1, 2013 at 5:14 PM, Ted Yu <yu...@gmail.com> wrote:
> > > >
> > > > > I assume you want to scan the table:
> > > > >
> > > > >
> > > >
> > >
> >
> http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/HTable.html#getScanner(byte[])
> > > > >
> > > > > Can you point out which thrift client methods provide streaming
> > > > capability
> > > > > ?
> > > > >
> > > > > Thanks
> > > > >
> > > > > On Sat, Jun 1, 2013 at 8:09 AM, Simon Majou <si...@majou.org>
> wrote:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > I don't see any methods returning streams in the Java client
> > HTable :
> > > > > >
> > > > >
> > > >
> > >
> >
> http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/HTable.html
> > > > > >
> > > > > > That means we need to use the thrift client to get streams ?
> > > > > >
> > > > > > Simon
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: HTable and streaming

[Mohit Anchlia <mo...@gmail.com>]

Better approach would be to break the data in chunks and create a behaviour
similar to indirect blocks.

On Mon, Jun 3, 2013 at 9:12 PM, Asaf Mesika <as...@gmail.com> wrote:

> I guess one can hack opening a socket from a Coprocessor Endpoint and push
> its scanned data, thus achieving a stream.
>
>
> On Sun, Jun 2, 2013 at 12:42 AM, Stack <st...@duboce.net> wrote:
>
> > Yeah, no streaming API in our current client (nor does our thrift client
> > give you a streaming API).
> > St.Ack
> >
> >
> > On Sat, Jun 1, 2013 at 8:21 AM, Simon Majou <si...@majou.org> wrote:
> >
> > > No I don't want to scan a table, I want a stream of one result. In case
> > for
> > > example of big records.
> > >
> > > With thrift it is up to the client to deal with the response, so in
> > theory
> > > you can build a client which returns streams.
> > >
> > > But when I look at the current implementation of thrift for node for
> > > instance (https://github.com/apache/thrift/tree/trunk/lib/nodejs), it
> > > sends
> > > back only full formed results (as the Java client).
> > >
> > >
> > > Simon
> > >
> > >
> > > On Sat, Jun 1, 2013 at 5:14 PM, Ted Yu <yu...@gmail.com> wrote:
> > >
> > > > I assume you want to scan the table:
> > > >
> > > >
> > >
> >
> http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/HTable.html#getScanner(byte[])
> > > >
> > > > Can you point out which thrift client methods provide streaming
> > > capability
> > > > ?
> > > >
> > > > Thanks
> > > >
> > > > On Sat, Jun 1, 2013 at 8:09 AM, Simon Majou <si...@majou.org> wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > I don't see any methods returning streams in the Java client
> HTable :
> > > > >
> > > >
> > >
> >
> http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/HTable.html
> > > > >
> > > > > That means we need to use the thrift client to get streams ?
> > > > >
> > > > > Simon
> > > > >
> > > >
> > >
> >
>

Re: HTable and streaming

[Azuryy Yu <az...@gmail.com>]

This should be a big patch, HBase need to provide streaming access, such as
HTableInputStream, HTableOutputStream etc.


On Tue, Jun 4, 2013 at 12:12 PM, Asaf Mesika <as...@gmail.com> wrote:

> I guess one can hack opening a socket from a Coprocessor Endpoint and push
> its scanned data, thus achieving a stream.
>
>
> On Sun, Jun 2, 2013 at 12:42 AM, Stack <st...@duboce.net> wrote:
>
> > Yeah, no streaming API in our current client (nor does our thrift client
> > give you a streaming API).
> > St.Ack
> >
> >
> > On Sat, Jun 1, 2013 at 8:21 AM, Simon Majou <si...@majou.org> wrote:
> >
> > > No I don't want to scan a table, I want a stream of one result. In case
> > for
> > > example of big records.
> > >
> > > With thrift it is up to the client to deal with the response, so in
> > theory
> > > you can build a client which returns streams.
> > >
> > > But when I look at the current implementation of thrift for node for
> > > instance (https://github.com/apache/thrift/tree/trunk/lib/nodejs), it
> > > sends
> > > back only full formed results (as the Java client).
> > >
> > >
> > > Simon
> > >
> > >
> > > On Sat, Jun 1, 2013 at 5:14 PM, Ted Yu <yu...@gmail.com> wrote:
> > >
> > > > I assume you want to scan the table:
> > > >
> > > >
> > >
> >
> http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/HTable.html#getScanner(byte[])
> > > >
> > > > Can you point out which thrift client methods provide streaming
> > > capability
> > > > ?
> > > >
> > > > Thanks
> > > >
> > > > On Sat, Jun 1, 2013 at 8:09 AM, Simon Majou <si...@majou.org> wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > I don't see any methods returning streams in the Java client
> HTable :
> > > > >
> > > >
> > >
> >
> http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/HTable.html
> > > > >
> > > > > That means we need to use the thrift client to get streams ?
> > > > >
> > > > > Simon
> > > > >
> > > >
> > >
> >
>

Re: HTable and streaming

[Asaf Mesika <as...@gmail.com>]

I guess one can hack opening a socket from a Coprocessor Endpoint and push
its scanned data, thus achieving a stream.


On Sun, Jun 2, 2013 at 12:42 AM, Stack <st...@duboce.net> wrote:

> Yeah, no streaming API in our current client (nor does our thrift client
> give you a streaming API).
> St.Ack
>
>
> On Sat, Jun 1, 2013 at 8:21 AM, Simon Majou <si...@majou.org> wrote:
>
> > No I don't want to scan a table, I want a stream of one result. In case
> for
> > example of big records.
> >
> > With thrift it is up to the client to deal with the response, so in
> theory
> > you can build a client which returns streams.
> >
> > But when I look at the current implementation of thrift for node for
> > instance (https://github.com/apache/thrift/tree/trunk/lib/nodejs), it
> > sends
> > back only full formed results (as the Java client).
> >
> >
> > Simon
> >
> >
> > On Sat, Jun 1, 2013 at 5:14 PM, Ted Yu <yu...@gmail.com> wrote:
> >
> > > I assume you want to scan the table:
> > >
> > >
> >
> http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/HTable.html#getScanner(byte[])
> > >
> > > Can you point out which thrift client methods provide streaming
> > capability
> > > ?
> > >
> > > Thanks
> > >
> > > On Sat, Jun 1, 2013 at 8:09 AM, Simon Majou <si...@majou.org> wrote:
> > >
> > > > Hi,
> > > >
> > > > I don't see any methods returning streams in the Java client HTable :
> > > >
> > >
> >
> http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/HTable.html
> > > >
> > > > That means we need to use the thrift client to get streams ?
> > > >
> > > > Simon
> > > >
> > >
> >
>

Re: HTable and streaming

[Stack <st...@duboce.net>]

Yeah, no streaming API in our current client (nor does our thrift client
give you a streaming API).
St.Ack


On Sat, Jun 1, 2013 at 8:21 AM, Simon Majou <si...@majou.org> wrote:

> No I don't want to scan a table, I want a stream of one result. In case for
> example of big records.
>
> With thrift it is up to the client to deal with the response, so in theory
> you can build a client which returns streams.
>
> But when I look at the current implementation of thrift for node for
> instance (https://github.com/apache/thrift/tree/trunk/lib/nodejs), it
> sends
> back only full formed results (as the Java client).
>
>
> Simon
>
>
> On Sat, Jun 1, 2013 at 5:14 PM, Ted Yu <yu...@gmail.com> wrote:
>
> > I assume you want to scan the table:
> >
> >
> http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/HTable.html#getScanner(byte[])
> >
> > Can you point out which thrift client methods provide streaming
> capability
> > ?
> >
> > Thanks
> >
> > On Sat, Jun 1, 2013 at 8:09 AM, Simon Majou <si...@majou.org> wrote:
> >
> > > Hi,
> > >
> > > I don't see any methods returning streams in the Java client HTable :
> > >
> >
> http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/HTable.html
> > >
> > > That means we need to use the thrift client to get streams ?
> > >
> > > Simon
> > >
> >
>

Re: HTable and streaming

[Simon Majou <si...@majou.org>]

No I don't want to scan a table, I want a stream of one result. In case for
example of big records.

With thrift it is up to the client to deal with the response, so in theory
you can build a client which returns streams.

But when I look at the current implementation of thrift for node for
instance (https://github.com/apache/thrift/tree/trunk/lib/nodejs), it sends
back only full formed results (as the Java client).


Simon


On Sat, Jun 1, 2013 at 5:14 PM, Ted Yu <yu...@gmail.com> wrote:

> I assume you want to scan the table:
>
> http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/HTable.html#getScanner(byte[])
>
> Can you point out which thrift client methods provide streaming capability
> ?
>
> Thanks
>
> On Sat, Jun 1, 2013 at 8:09 AM, Simon Majou <si...@majou.org> wrote:
>
> > Hi,
> >
> > I don't see any methods returning streams in the Java client HTable :
> >
> http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/HTable.html
> >
> > That means we need to use the thrift client to get streams ?
> >
> > Simon
> >
>

Re: HTable and streaming

[Ted Yu <yu...@gmail.com>]

I assume you want to scan the table:
http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/HTable.html#getScanner(byte[])

Can you point out which thrift client methods provide streaming capability ?

Thanks

On Sat, Jun 1, 2013 at 8:09 AM, Simon Majou <si...@majou.org> wrote:

> Hi,
>
> I don't see any methods returning streams in the Java client HTable :
> http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/HTable.html
>
> That means we need to use the thrift client to get streams ?
>
> Simon
>