You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Hari Sreekumar <hs...@clickable.com> on 2011/02/01 19:57:24 UTC
Queries regarding REST API
Hi,
I am planning to implement the REST API to query my HBase tables in
production. The documentation mentions that the stargate api is not
production ready. What are the major issues in the REST api that would make
it a risk to use it on production?
Secondly, I am getting base64 encoded values when I use REST. Is there any
other encoding possible? Decoding base64 on client side will be a big effort
for me. Is there a way I can get it in a non-encoded way?
Thanks,
Hari
Re: Queries regarding REST API
Posted by Andrew Purtell <ap...@apache.org>.
I mean support for the text/plain content-type with a similar behavior as application/octet-stream, except rather than providing 8 bit data all non-ASCII characters will be suitably escaped.
To do so for the XML and JSON cases, the way Jersey works we'd have to convert all byte[] to strings with escapes internally and then undo this if serving binary, which would be a significant tax on performance.
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White)
--- On Thu, 2/3/11, Hari Sreekumar <hs...@clickable.com> wrote:
> From: Hari Sreekumar <hs...@clickable.com>
> Subject: Re: Queries regarding REST API
> To: user@hbase.apache.org, apurtell@apache.org
> Date: Thursday, February 3, 2011, 10:29 PM
> Hi Andrew,
>
> Plain text output would be great for me. Please add it if
> it isn't too much effort. You mean plain text as in text/plain
> or plain text within the xml/json?
>
> Thanks a lot,
> Hari
>
> On Fri, Feb 4, 2011 at 12:37 AM, Andrew Purtell <ap...@apache.org>
> wrote:
>
> > > Thanks guys for the replies. Is there any
> difference if I use json
> > > representation (application/json)?
> >
> > Key, column, and value will also be base64 encoded if
> using JSON
> > representation
> >
> > > Can I get data is plain text with the special
> characters
> > > escaped in some way?
> >
> > I could add a text/plain representation option that
> would send back
> > ruby-style escaped text data if this would be useful
> to you. (This is what
> > our Bytes.toBinaryString() helper produces.)
> >
> > Best regards,
> >
> > - Andy
> >
> > Problems worthy of attack prove their worth by hitting
> back. - Piet Hein
> > (via Tom White)
> >
> >
> > --- On Thu, 2/3/11, Hari Sreekumar <hs...@clickable.com>
> wrote:
> >
> > > From: Hari Sreekumar <hs...@clickable.com>
> > > Subject: Re: Queries regarding REST API
> > > To: user@hbase.apache.org,
> apurtell@apache.org
> > > Date: Thursday, February 3, 2011, 1:10 AM
> > > Thanks guys for the replies. Is there
> > > any difference if I use json
> > > representation (application/json)? Can I get
> another
> > > encoding if I go that
> > > way? Can I get data is plain text with the
> special
> > > characters escaped in
> > > some way?
> > >
> > > hari
> > >
> > > On Wed, Feb 2, 2011 at 1:50 AM, Andrew Purtell
> <ap...@apache.org>
> > > wrote:
> > >
> > > > > The problem is how do you represent
> what could be
> > > binary
> > > > > data [...]
> > > >
> > > > Exactly. Base64 it required if you are using
> XML
> > > representation (text/xml)
> > > > because the basic data type in HBase is
> byte[].
> > > >
> > > > You also have the option of binary
> representations,
> > > either protobuf
> > > > (application/x-protobuf) or raw value
> > > (application/octet-stream). In those
> > > > cases the data is sent in binary form.
> > > >
> > > > Using protobufs requires your client do the
> necessary
> > > unmarshalling of
> > > > course.
> > > >
> > > > Using raw (application/octet-stream)
> currently has
> > > limitations. See
> > > > https://issues.apache.org/jira/browse/HBASE-1788. It's
> > > not possible to
> > > > query more than one version of a value at a
> time or
> > > return more than one
> > > > value out of a scanner at a time or insert
> more than
> > > one value at a time.
> > > > Adding multipart support would solve that
> problem.
> > > >
> > > > Best regards,
> > > >
> > > > - Andy
> > > >
> > > > Problems worthy of attack prove their worth
> by hitting
> > > back.
> > > > - Piet Hein (via Tom White)
> > > >
> > > >
> > > > --- On Tue, 2/1/11, Stack <st...@duboce.net>
> > > wrote:
> > > >
> > > > > From: Stack <st...@duboce.net>
> > > > > Subject: Re: Queries regarding REST
> API
> > > > > To: user@hbase.apache.org
> > > > > Date: Tuesday, February 1, 2011, 11:21
> AM
> > > > > On Tue, Feb 1, 2011 at 10:57 AM, Hari
> > > > > Sreekumar
> > > > > <hs...@clickable.com>
> > > > > wrote:
> > > > > > Hi,
> > > > > >
> > > > > > I am planning to implement the
> REST API to
> > > query my
> > > > > > HBase tables in production. The
> > > documentation mentions that the
> > > > > > stargate api is not production
> ready.
> > > > >
> > > > > Sorry. Where did you see
> that. We
> > > need to fix
> > > > > that. I believe at least the
> yfrog folks
> > > are using REST in
> > > > > production (Correct me if I'm wrong
> Jack).
> > > > >
> > > > >
> > > > > > Secondly, I am getting base64
> encoded values
> > > when I use REST.
> > > > > > Is there any other encoding
> possible?
> > > Decoding base64 on client
> > > > > > side will be a big effort> for
> me. Is
> > > there a way I can get it
> > > > > > in a non-encoded way?
> > > > > >
> > > > >
> > > > > I'lll let those that know the REST
> gateway better
> > > than I
> > > > > answer this question. The problem
> is how do
> > > you represent what
> > > > > could be binary data in an URL.
> > > > >
> > > > > St.Ack
> > > > >
> > > >
> > > >
> > > >
> > > >
> > >
> >
> >
> >
> >
>
Re: Queries regarding REST API
Posted by Hari Sreekumar <hs...@clickable.com>.
Hi Andrew,
Plain text output would be great for me. Please add it if it isn't too much
effort. You mean plain text as in text/plain of plain text within the
xml/json?
Thanks a lot,
Hari
On Fri, Feb 4, 2011 at 12:37 AM, Andrew Purtell <ap...@apache.org> wrote:
> > Thanks guys for the replies. Is there any difference if I use json
> > representation (application/json)?
>
> Key, column, and value will also be base64 encoded if using JSON
> representation
>
> > Can I get data is plain text with the special characters
> > escaped in some way?
>
> I could add a text/plain representation option that would send back
> ruby-style escaped text data if this would be useful to you. (This is what
> our Bytes.toBinaryString() helper produces.)
>
> Best regards,
>
> - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>
>
> --- On Thu, 2/3/11, Hari Sreekumar <hs...@clickable.com> wrote:
>
> > From: Hari Sreekumar <hs...@clickable.com>
> > Subject: Re: Queries regarding REST API
> > To: user@hbase.apache.org, apurtell@apache.org
> > Date: Thursday, February 3, 2011, 1:10 AM
> > Thanks guys for the replies. Is there
> > any difference if I use json
> > representation (application/json)? Can I get another
> > encoding if I go that
> > way? Can I get data is plain text with the special
> > characters escaped in
> > some way?
> >
> > hari
> >
> > On Wed, Feb 2, 2011 at 1:50 AM, Andrew Purtell <ap...@apache.org>
> > wrote:
> >
> > > > The problem is how do you represent what could be
> > binary
> > > > data [...]
> > >
> > > Exactly. Base64 it required if you are using XML
> > representation (text/xml)
> > > because the basic data type in HBase is byte[].
> > >
> > > You also have the option of binary representations,
> > either protobuf
> > > (application/x-protobuf) or raw value
> > (application/octet-stream). In those
> > > cases the data is sent in binary form.
> > >
> > > Using protobufs requires your client do the necessary
> > unmarshalling of
> > > course.
> > >
> > > Using raw (application/octet-stream) currently has
> > limitations. See
> > > https://issues.apache.org/jira/browse/HBASE-1788. It's
> > not possible to
> > > query more than one version of a value at a time or
> > return more than one
> > > value out of a scanner at a time or insert more than
> > one value at a time.
> > > Adding multipart support would solve that problem.
> > >
> > > Best regards,
> > >
> > > - Andy
> > >
> > > Problems worthy of attack prove their worth by hitting
> > back.
> > > - Piet Hein (via Tom White)
> > >
> > >
> > > --- On Tue, 2/1/11, Stack <st...@duboce.net>
> > wrote:
> > >
> > > > From: Stack <st...@duboce.net>
> > > > Subject: Re: Queries regarding REST API
> > > > To: user@hbase.apache.org
> > > > Date: Tuesday, February 1, 2011, 11:21 AM
> > > > On Tue, Feb 1, 2011 at 10:57 AM, Hari
> > > > Sreekumar
> > > > <hs...@clickable.com>
> > > > wrote:
> > > > > Hi,
> > > > >
> > > > > I am planning to implement the REST API to
> > query my
> > > > > HBase tables in production. The
> > documentation mentions that the
> > > > > stargate api is not production ready.
> > > >
> > > > Sorry. Where did you see that. We
> > need to fix
> > > > that. I believe at least the yfrog folks
> > are using REST in
> > > > production (Correct me if I'm wrong Jack).
> > > >
> > > >
> > > > > Secondly, I am getting base64 encoded values
> > when I use REST.
> > > > > Is there any other encoding possible?
> > Decoding base64 on client
> > > > > side will be a big effort> for me. Is
> > there a way I can get it
> > > > > in a non-encoded way?
> > > > >
> > > >
> > > > I'lll let those that know the REST gateway better
> > than I
> > > > answer this question. The problem is how do
> > you represent what
> > > > could be binary data in an URL.
> > > >
> > > > St.Ack
> > > >
> > >
> > >
> > >
> > >
> >
>
>
>
>
Re: Queries regarding REST API
Posted by Andrew Purtell <ap...@apache.org>.
> Thanks guys for the replies. Is there any difference if I use json
> representation (application/json)?
Key, column, and value will also be base64 encoded if using JSON representation
> Can I get data is plain text with the special characters
> escaped in some way?
I could add a text/plain representation option that would send back ruby-style escaped text data if this would be useful to you. (This is what our Bytes.toBinaryString() helper produces.)
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White)
--- On Thu, 2/3/11, Hari Sreekumar <hs...@clickable.com> wrote:
> From: Hari Sreekumar <hs...@clickable.com>
> Subject: Re: Queries regarding REST API
> To: user@hbase.apache.org, apurtell@apache.org
> Date: Thursday, February 3, 2011, 1:10 AM
> Thanks guys for the replies. Is there
> any difference if I use json
> representation (application/json)? Can I get another
> encoding if I go that
> way? Can I get data is plain text with the special
> characters escaped in
> some way?
>
> hari
>
> On Wed, Feb 2, 2011 at 1:50 AM, Andrew Purtell <ap...@apache.org>
> wrote:
>
> > > The problem is how do you represent what could be
> binary
> > > data [...]
> >
> > Exactly. Base64 it required if you are using XML
> representation (text/xml)
> > because the basic data type in HBase is byte[].
> >
> > You also have the option of binary representations,
> either protobuf
> > (application/x-protobuf) or raw value
> (application/octet-stream). In those
> > cases the data is sent in binary form.
> >
> > Using protobufs requires your client do the necessary
> unmarshalling of
> > course.
> >
> > Using raw (application/octet-stream) currently has
> limitations. See
> > https://issues.apache.org/jira/browse/HBASE-1788. It's
> not possible to
> > query more than one version of a value at a time or
> return more than one
> > value out of a scanner at a time or insert more than
> one value at a time.
> > Adding multipart support would solve that problem.
> >
> > Best regards,
> >
> > - Andy
> >
> > Problems worthy of attack prove their worth by hitting
> back.
> > - Piet Hein (via Tom White)
> >
> >
> > --- On Tue, 2/1/11, Stack <st...@duboce.net>
> wrote:
> >
> > > From: Stack <st...@duboce.net>
> > > Subject: Re: Queries regarding REST API
> > > To: user@hbase.apache.org
> > > Date: Tuesday, February 1, 2011, 11:21 AM
> > > On Tue, Feb 1, 2011 at 10:57 AM, Hari
> > > Sreekumar
> > > <hs...@clickable.com>
> > > wrote:
> > > > Hi,
> > > >
> > > > I am planning to implement the REST API to
> query my
> > > > HBase tables in production. The
> documentation mentions that the
> > > > stargate api is not production ready.
> > >
> > > Sorry. Where did you see that. We
> need to fix
> > > that. I believe at least the yfrog folks
> are using REST in
> > > production (Correct me if I'm wrong Jack).
> > >
> > >
> > > > Secondly, I am getting base64 encoded values
> when I use REST.
> > > > Is there any other encoding possible?
> Decoding base64 on client
> > > > side will be a big effort> for me. Is
> there a way I can get it
> > > > in a non-encoded way?
> > > >
> > >
> > > I'lll let those that know the REST gateway better
> than I
> > > answer this question. The problem is how do
> you represent what
> > > could be binary data in an URL.
> > >
> > > St.Ack
> > >
> >
> >
> >
> >
>
Re: Queries regarding REST API
Posted by Hari Sreekumar <hs...@clickable.com>.
Thanks guys for the replies. Is there any difference if I use json
representation (application/json)? Can I get another encoding if I go that
way? Can I get data is plain text with the special characters escaped in
some way?
hari
On Wed, Feb 2, 2011 at 1:50 AM, Andrew Purtell <ap...@apache.org> wrote:
> > The problem is how do you represent what could be binary
> > data [...]
>
> Exactly. Base64 it required if you are using XML representation (text/xml)
> because the basic data type in HBase is byte[].
>
> You also have the option of binary representations, either protobuf
> (application/x-protobuf) or raw value (application/octet-stream). In those
> cases the data is sent in binary form.
>
> Using protobufs requires your client do the necessary unmarshalling of
> course.
>
> Using raw (application/octet-stream) currently has limitations. See
> https://issues.apache.org/jira/browse/HBASE-1788. It's not possible to
> query more than one version of a value at a time or return more than one
> value out of a scanner at a time or insert more than one value at a time.
> Adding multipart support would solve that problem.
>
> Best regards,
>
> - Andy
>
> Problems worthy of attack prove their worth by hitting back.
> - Piet Hein (via Tom White)
>
>
> --- On Tue, 2/1/11, Stack <st...@duboce.net> wrote:
>
> > From: Stack <st...@duboce.net>
> > Subject: Re: Queries regarding REST API
> > To: user@hbase.apache.org
> > Date: Tuesday, February 1, 2011, 11:21 AM
> > On Tue, Feb 1, 2011 at 10:57 AM, Hari
> > Sreekumar
> > <hs...@clickable.com>
> > wrote:
> > > Hi,
> > >
> > > I am planning to implement the REST API to query my
> > > HBase tables in production. The documentation mentions that the
> > > stargate api is not production ready.
> >
> > Sorry. Where did you see that. We need to fix
> > that. I believe at least the yfrog folks are using REST in
> > production (Correct me if I'm wrong Jack).
> >
> >
> > > Secondly, I am getting base64 encoded values when I use REST.
> > > Is there any other encoding possible? Decoding base64 on client
> > > side will be a big effort> for me. Is there a way I can get it
> > > in a non-encoded way?
> > >
> >
> > I'lll let those that know the REST gateway better than I
> > answer this question. The problem is how do you represent what
> > could be binary data in an URL.
> >
> > St.Ack
> >
>
>
>
>
Re: Queries regarding REST API
Posted by Andrew Purtell <ap...@apache.org>.
> The problem is how do you represent what could be binary
> data [...]
Exactly. Base64 it required if you are using XML representation (text/xml) because the basic data type in HBase is byte[].
You also have the option of binary representations, either protobuf (application/x-protobuf) or raw value (application/octet-stream). In those cases the data is sent in binary form.
Using protobufs requires your client do the necessary unmarshalling of course.
Using raw (application/octet-stream) currently has limitations. See https://issues.apache.org/jira/browse/HBASE-1788. It's not possible to query more than one version of a value at a time or return more than one value out of a scanner at a time or insert more than one value at a time. Adding multipart support would solve that problem.
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back.
- Piet Hein (via Tom White)
--- On Tue, 2/1/11, Stack <st...@duboce.net> wrote:
> From: Stack <st...@duboce.net>
> Subject: Re: Queries regarding REST API
> To: user@hbase.apache.org
> Date: Tuesday, February 1, 2011, 11:21 AM
> On Tue, Feb 1, 2011 at 10:57 AM, Hari
> Sreekumar
> <hs...@clickable.com>
> wrote:
> > Hi,
> >
> > I am planning to implement the REST API to query my
> > HBase tables in production. The documentation mentions that the
> > stargate api is not production ready.
>
> Sorry. Where did you see that. We need to fix
> that. I believe at least the yfrog folks are using REST in
> production (Correct me if I'm wrong Jack).
>
>
> > Secondly, I am getting base64 encoded values when I use REST.
> > Is there any other encoding possible? Decoding base64 on client
> > side will be a big effort> for me. Is there a way I can get it
> > in a non-encoded way?
> >
>
> I'lll let those that know the REST gateway better than I
> answer this question. The problem is how do you represent what
> could be binary data in an URL.
>
> St.Ack
>
Re: Queries regarding REST API
Posted by Stack <st...@duboce.net>.
On Tue, Feb 1, 2011 at 10:57 AM, Hari Sreekumar
<hs...@clickable.com> wrote:
> Hi,
>
> I am planning to implement the REST API to query my HBase tables in
> production. The documentation mentions that the stargate api is not
> production ready.
Sorry. Where did you see that. We need to fix that. I believe at
least the yfrog folks are using REST in production (Correct me if I'm
wrong Jack).
> Secondly, I am getting base64 encoded values when I use REST. Is there any
> other encoding possible? Decoding base64 on client side will be a big effort
> for me. Is there a way I can get it in a non-encoded way?
>
I'lll let those that know the REST gateway better than I answer this
question. The problem is how do you represent what could be binary
data in an URL.
St.Ack