You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Greg Cottman <gr...@quest.com> on 2011/07/19 08:57:00 UTC
Stargate: Only getting HTTP 200 responses in 0.90.x
Hi guys,
We're using the Stargate REST server to access HBase data and as of 0.90.1 we getting HTTP 200 packets where we used to get 201s and 204s. There seem to be two significant changes...
We used to get a 201 after creating a scanner with the scanner ID in the "Location" property. We still get this packet with a valid scanner ID but it's now an HTTP 200 packet. This doesn't matter too much because we are mostly interested in the Location property anyway.
The real problem is that we used to get an HTTP 204 when we exhausted the scanner, but now we get an 200 packet there too. The documentation on scanners at http://wiki.apache.org/hadoop/Hbase/Stargate#A5 says we should still expect this to be the case.
These 200 packets appear to have 20 bytes of garbage in the content. The content encoding says they're gzip streams but they're not, and they seem to appear on any 200 packet that doesn't carry a specific payload. Because it's a dodgy 200 packet at the end of the legitimate 200 packets carrying the scanner results, we are trying to decode this as just another data packet at which point we crash and burn. :-(
Anyone seeing anything like this? Any ideas on why the packets no longer match the documented protocol?
Thanks,
Greg.
________________________________
Greg Cottman
Technical Architect, Cloud Databases
Quest Software, Melbourne
Tel: +61 3 9811 8057
Greg.Cottman@quest.com<ma...@quest.com>
RE: Filters for non-Java clients?
Posted by Steinmaurer Thomas <Th...@scch.at>.
Ah, cool!
Thanks,
Thomas
-----Original Message-----
From: Ted Yu [mailto:yuzhihong@gmail.com]
Sent: Mittwoch, 10. August 2011 13:28
To: user@hbase.apache.org
Cc: <us...@hbase.apache.org>
Subject: Re: Filters for non-Java clients?
See HBase-4176.
Cheers
On Aug 10, 2011, at 2:36 AM, "Steinmaurer Thomas"
<Th...@scch.at> wrote:
> I have to jump in here as well. Are there any initiatives to bring the
> Thrift API in respect to filters somewhere near to Java API?
> Especially the QualifierFilter in combination with the
> RegexStringComparator would be a nice addition.
>
> Are there any starting points to possibly enhance the Thrift API
> ourselves?
>
> Thanks,
> Thomas
>
>
> -----Original Message-----
> From: Greg Cottman [mailto:greg.cottman@quest.com]
> Sent: Montag, 25. Juli 2011 07:20
> To: user@hbase.apache.org
> Subject: Filters for non-Java clients?
>
>
> We are using the REST interface because we have a C++ client, but get
> performance complaints arising from the fact that we have to fetch the
> entire table for any query.
>
> Is anyone considering exposing the Java Filter implementation for
> scanners in either the REST or Thrift APIs? If so, is there a
timeline?
>
> It wouldn't have to be as rich as the 42 implementations of the Java
> Filter interface. Just being able to do a simple range scan on one or
> more columns would be a really exciting prospect. :-)
>
> Cheers,
> Greg.
>
>
> Greg Cottman
> Technical Architect, Cloud Databases
> Quest Software, Melbourne
> Tel: +61 3 9811 8057
> Greg.Cottman@quest.com
>
>
>
Re: Filters for non-Java clients?
Posted by Ted Yu <yu...@gmail.com>.
See HBase-4176.
Cheers
On Aug 10, 2011, at 2:36 AM, "Steinmaurer Thomas" <Th...@scch.at> wrote:
> I have to jump in here as well. Are there any initiatives to bring the
> Thrift API in respect to filters somewhere near to Java API? Especially
> the QualifierFilter in combination with the RegexStringComparator would
> be a nice addition.
>
> Are there any starting points to possibly enhance the Thrift API
> ourselves?
>
> Thanks,
> Thomas
>
>
> -----Original Message-----
> From: Greg Cottman [mailto:greg.cottman@quest.com]
> Sent: Montag, 25. Juli 2011 07:20
> To: user@hbase.apache.org
> Subject: Filters for non-Java clients?
>
>
> We are using the REST interface because we have a C++ client, but get
> performance complaints arising from the fact that we have to fetch the
> entire table for any query.
>
> Is anyone considering exposing the Java Filter implementation for
> scanners in either the REST or Thrift APIs? If so, is there a timeline?
>
> It wouldn't have to be as rich as the 42 implementations of the Java
> Filter interface. Just being able to do a simple range scan on one or
> more columns would be a really exciting prospect. :-)
>
> Cheers,
> Greg.
>
>
> Greg Cottman
> Technical Architect, Cloud Databases
> Quest Software, Melbourne
> Tel: +61 3 9811 8057
> Greg.Cottman@quest.com
>
>
>
RE: Filters for non-Java clients?
Posted by Steinmaurer Thomas <Th...@scch.at>.
I have to jump in here as well. Are there any initiatives to bring the
Thrift API in respect to filters somewhere near to Java API? Especially
the QualifierFilter in combination with the RegexStringComparator would
be a nice addition.
Are there any starting points to possibly enhance the Thrift API
ourselves?
Thanks,
Thomas
-----Original Message-----
From: Greg Cottman [mailto:greg.cottman@quest.com]
Sent: Montag, 25. Juli 2011 07:20
To: user@hbase.apache.org
Subject: Filters for non-Java clients?
We are using the REST interface because we have a C++ client, but get
performance complaints arising from the fact that we have to fetch the
entire table for any query.
Is anyone considering exposing the Java Filter implementation for
scanners in either the REST or Thrift APIs? If so, is there a timeline?
It wouldn't have to be as rich as the 42 implementations of the Java
Filter interface. Just being able to do a simple range scan on one or
more columns would be a really exciting prospect. :-)
Cheers,
Greg.
Greg Cottman
Technical Architect, Cloud Databases
Quest Software, Melbourne
Tel: +61 3 9811 8057
Greg.Cottman@quest.com
RE: Filters for non-Java clients?
Posted by Greg Cottman <gr...@quest.com>.
Oops. My bad. Looking at http://wiki.apache.org/hadoop/Hbase/Stargate and the Scanner type definition does include a "filter" element. <blush> It may be worth mentioning the filter support in the section on Scanner creation, even if you attach a caveat about needing to be brave.
I'll have a look at ScannerModel in the Java API. Thanks Andy.
Cheers,
Greg.
-----Original Message-----
From: Andrew Purtell [mailto:apurtell@apache.org]
Sent: Tuesday, 26 July 2011 2:42 AM
To: user@hbase.apache.org
Subject: Re: Filters for non-Java clients?
The REST API has filter support.
Strictly speaking the representation is multilanguage, but only the Java API -- the ScannerModel class, ScannerModel.stringifyFilter -- has support for converting a Java filter tree into a JSON encoded representation of same. However you could do this in Java once to obtain the filter string and then use that string as-is with the C++ client.
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White)
----- Original Message -----
> From: Greg Cottman <gr...@quest.com>
> To: "user@hbase.apache.org" <us...@hbase.apache.org>
> Cc:
> Sent: Sunday, July 24, 2011 10:20 PM
> Subject: Filters for non-Java clients?
>
>
> We are using the REST interface because we have a C++ client, but get
> performance complaints arising from the fact that we have to fetch the entire
> table for any query.
>
> Is anyone considering exposing the Java Filter implementation for scanners in
> either the REST or Thrift APIs? If so, is there a timeline?
>
> It wouldn't have to be as rich as the 42 implementations of the Java Filter
> interface. Just being able to do a simple range scan on one or more columns
> would be a really exciting prospect. :-)
>
> Cheers,
> Greg.
>
>
> Greg Cottman
> Technical Architect, Cloud Databases
> Quest Software, Melbourne
> Tel: +61 3 9811 8057
> Greg.Cottman@quest.com
>
Re: Filters for non-Java clients?
Posted by Andrew Purtell <ap...@apache.org>.
The REST API has filter support.
Strictly speaking the representation is multilanguage, but only the Java API -- the ScannerModel class, ScannerModel.stringifyFilter -- has support for converting a Java filter tree into a JSON encoded representation of same. However you could do this in Java once to obtain the filter string and then use that string as-is with the C++ client.
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White)
----- Original Message -----
> From: Greg Cottman <gr...@quest.com>
> To: "user@hbase.apache.org" <us...@hbase.apache.org>
> Cc:
> Sent: Sunday, July 24, 2011 10:20 PM
> Subject: Filters for non-Java clients?
>
>
> We are using the REST interface because we have a C++ client, but get
> performance complaints arising from the fact that we have to fetch the entire
> table for any query.
>
> Is anyone considering exposing the Java Filter implementation for scanners in
> either the REST or Thrift APIs? If so, is there a timeline?
>
> It wouldn't have to be as rich as the 42 implementations of the Java Filter
> interface. Just being able to do a simple range scan on one or more columns
> would be a really exciting prospect. :-)
>
> Cheers,
> Greg.
>
>
> Greg Cottman
> Technical Architect, Cloud Databases
> Quest Software, Melbourne
> Tel: +61 3 9811 8057
> Greg.Cottman@quest.com
>
Re: Filters for non-Java clients?
Posted by Joey Echeverria <jo...@cloudera.com>.
Sounds like a good idea, file a JIRA.
-Joey
On Sun, Jul 24, 2011 at 10:20 PM, Greg Cottman <gr...@quest.com>wrote:
>
> We are using the REST interface because we have a C++ client, but get
> performance complaints arising from the fact that we have to fetch the
> entire table for any query.
>
> Is anyone considering exposing the Java Filter implementation for scanners
> in either the REST or Thrift APIs? If so, is there a timeline?
>
> It wouldn't have to be as rich as the 42 implementations of the Java Filter
> interface. Just being able to do a simple range scan on one or more columns
> would be a really exciting prospect. :-)
>
> Cheers,
> Greg.
>
>
> Greg Cottman
> Technical Architect, Cloud Databases
> Quest Software, Melbourne
> Tel: +61 3 9811 8057
> Greg.Cottman@quest.com
>
>
>
>
--
Joseph Echeverria
Cloudera, Inc.
443.305.9434
Filters for non-Java clients?
Posted by Greg Cottman <gr...@quest.com>.
We are using the REST interface because we have a C++ client, but get performance complaints arising from the fact that we have to fetch the entire table for any query.
Is anyone considering exposing the Java Filter implementation for scanners in either the REST or Thrift APIs? If so, is there a timeline?
It wouldn't have to be as rich as the 42 implementations of the Java Filter interface. Just being able to do a simple range scan on one or more columns would be a really exciting prospect. :-)
Cheers,
Greg.
Greg Cottman
Technical Architect, Cloud Databases
Quest Software, Melbourne
Tel: +61 3 9811 8057
Greg.Cottman@quest.com
Re: Stargate: Only getting HTTP 200 responses in 0.90.x
Posted by Andrew Purtell <ap...@apache.org>.
Hi Greg,
Yes the bug affects any use of the gzip filter.
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White)
----- Original Message -----
> From: Greg Cottman <gr...@quest.com>
> To: "user@hbase.apache.org" <us...@hbase.apache.org>
> Cc:
> Sent: Sunday, July 24, 2011 10:08 PM
> Subject: RE: Stargate: Only getting HTTP 200 responses in 0.90.x
>
> Hi Andrew,
>
> Thanks for the response.
>
> Our Content-Type is actually "text/xml", not JSON, but I can imagine
> the bug just generally relates to gzip encoding regardless of content type.
>
> I'll try out 0.90.4 when it's released.
>
> Thanks,
> Greg.
>
>
> -----Original Message-----
> From: Andrew Purtell [mailto:apurtell@apache.org]
> Sent: Saturday, 23 July 2011 4:20 PM
> To: user@hbase.apache.org
> Subject: Re: Stargate: Only getting HTTP 200 responses in 0.90.x
>
>> We used to get a 201 after creating a scanner with the scanner ID in
>> the "Location" property. We still get this packet with a valid
> scanner
>> ID but it's now an HTTP 200 packet.
>
>> The real problem is that we used to get an HTTP 204 when we exhausted the
>> scanner, but now we get an 200 packet there too.
>
>
> Both problems are due to using the gzip filter. Incorporated
> in https://issues.apache.org/jira/browse/HBASE-4105
>
> Best regards,
>
> - Andy
>
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein (via
> Tom White)
>
>
> ----- Original Message -----
>> From: Greg Cottman <gr...@quest.com>
>> To: "HBase List (user@hbase.apache.org)"
> <us...@hbase.apache.org>
>> Cc:
>> Sent: Monday, July 18, 2011 11:57 PM
>> Subject: Stargate: Only getting HTTP 200 responses in 0.90.x
>>
>> Hi guys,
>>
>> We're using the Stargate REST server to access HBase data and as of
> 0.90.1
>> we getting HTTP 200 packets where we used to get 201s and 204s. There seem
> to
>> be two significant changes...
>>
>> We used to get a 201 after creating a scanner with the scanner ID in the
>> "Location" property. We still get this packet with a valid
> scanner ID
>> but it's now an HTTP 200 packet. This doesn't matter too much
> because
>> we are mostly interested in the Location property anyway.
>>
>> The real problem is that we used to get an HTTP 204 when we exhausted the
>> scanner, but now we get an 200 packet there too. The documentation on
> scanners
>> at http://wiki.apache.org/hadoop/Hbase/Stargate#A5says we should still
> expect
>> this to be the case.
>>
>> These 200 packets appear to have 20 bytes of garbage in the content. The
>> content encoding says they're gzip streams but they're not, and
> they
>> seem to appear on any 200 packet that doesn't carry a specific
> payload.
>> Because it's a dodgy 200 packet at the end of the legitimate 200
> packets
>> carrying the scanner results, we are trying to decode this as just another
> data
>> packet at which point we crash and burn. :-(
>>
>> Anyone seeing anything like this? Any ideas on why the packets no longer
> match
>> the documented protocol?
>>
>> Thanks,
>> Greg.
>>
>>
>> ________________________________
>> Greg Cottman
>> Technical Architect, Cloud Databases
>> Quest Software, Melbourne
>> Tel: +61 3 9811 8057
>> Greg.Cottman@quest.com<ma...@quest.com>
>>
>
RE: Stargate: Only getting HTTP 200 responses in 0.90.x
Posted by Greg Cottman <gr...@quest.com>.
Hi Andrew,
Thanks for the response.
Our Content-Type is actually "text/xml", not JSON, but I can imagine the bug just generally relates to gzip encoding regardless of content type.
I'll try out 0.90.4 when it's released.
Thanks,
Greg.
-----Original Message-----
From: Andrew Purtell [mailto:apurtell@apache.org]
Sent: Saturday, 23 July 2011 4:20 PM
To: user@hbase.apache.org
Subject: Re: Stargate: Only getting HTTP 200 responses in 0.90.x
> We used to get a 201 after creating a scanner with the scanner ID in
> the "Location" property. We still get this packet with a valid scanner
> ID but it's now an HTTP 200 packet.
> The real problem is that we used to get an HTTP 204 when we exhausted the
> scanner, but now we get an 200 packet there too.
Both problems are due to using the gzip filter. Incorporated in https://issues.apache.org/jira/browse/HBASE-4105
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White)
----- Original Message -----
> From: Greg Cottman <gr...@quest.com>
> To: "HBase List (user@hbase.apache.org)" <us...@hbase.apache.org>
> Cc:
> Sent: Monday, July 18, 2011 11:57 PM
> Subject: Stargate: Only getting HTTP 200 responses in 0.90.x
>
> Hi guys,
>
> We're using the Stargate REST server to access HBase data and as of 0.90.1
> we getting HTTP 200 packets where we used to get 201s and 204s. There seem to
> be two significant changes...
>
> We used to get a 201 after creating a scanner with the scanner ID in the
> "Location" property. We still get this packet with a valid scanner ID
> but it's now an HTTP 200 packet. This doesn't matter too much because
> we are mostly interested in the Location property anyway.
>
> The real problem is that we used to get an HTTP 204 when we exhausted the
> scanner, but now we get an 200 packet there too. The documentation on scanners
> at http://wiki.apache.org/hadoop/Hbase/Stargate#A5says we should still expect
> this to be the case.
>
> These 200 packets appear to have 20 bytes of garbage in the content. The
> content encoding says they're gzip streams but they're not, and they
> seem to appear on any 200 packet that doesn't carry a specific payload.
> Because it's a dodgy 200 packet at the end of the legitimate 200 packets
> carrying the scanner results, we are trying to decode this as just another data
> packet at which point we crash and burn. :-(
>
> Anyone seeing anything like this? Any ideas on why the packets no longer match
> the documented protocol?
>
> Thanks,
> Greg.
>
>
> ________________________________
> Greg Cottman
> Technical Architect, Cloud Databases
> Quest Software, Melbourne
> Tel: +61 3 9811 8057
> Greg.Cottman@quest.com<ma...@quest.com>
>
Re: Stargate: Only getting HTTP 200 responses in 0.90.x
Posted by Andrew Purtell <ap...@apache.org>.
> We used to get a 201 after creating a scanner with the scanner ID in
> the "Location" property. We still get this packet with a valid scanner
> ID but it's now an HTTP 200 packet.
> The real problem is that we used to get an HTTP 204 when we exhausted the
> scanner, but now we get an 200 packet there too.
Both problems are due to using the gzip filter. Incorporated in https://issues.apache.org/jira/browse/HBASE-4105
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White)
----- Original Message -----
> From: Greg Cottman <gr...@quest.com>
> To: "HBase List (user@hbase.apache.org)" <us...@hbase.apache.org>
> Cc:
> Sent: Monday, July 18, 2011 11:57 PM
> Subject: Stargate: Only getting HTTP 200 responses in 0.90.x
>
> Hi guys,
>
> We're using the Stargate REST server to access HBase data and as of 0.90.1
> we getting HTTP 200 packets where we used to get 201s and 204s. There seem to
> be two significant changes...
>
> We used to get a 201 after creating a scanner with the scanner ID in the
> "Location" property. We still get this packet with a valid scanner ID
> but it's now an HTTP 200 packet. This doesn't matter too much because
> we are mostly interested in the Location property anyway.
>
> The real problem is that we used to get an HTTP 204 when we exhausted the
> scanner, but now we get an 200 packet there too. The documentation on scanners
> at http://wiki.apache.org/hadoop/Hbase/Stargate#A5says we should still expect
> this to be the case.
>
> These 200 packets appear to have 20 bytes of garbage in the content. The
> content encoding says they're gzip streams but they're not, and they
> seem to appear on any 200 packet that doesn't carry a specific payload.
> Because it's a dodgy 200 packet at the end of the legitimate 200 packets
> carrying the scanner results, we are trying to decode this as just another data
> packet at which point we crash and burn. :-(
>
> Anyone seeing anything like this? Any ideas on why the packets no longer match
> the documented protocol?
>
> Thanks,
> Greg.
>
>
> ________________________________
> Greg Cottman
> Technical Architect, Cloud Databases
> Quest Software, Melbourne
> Tel: +61 3 9811 8057
> Greg.Cottman@quest.com<ma...@quest.com>
>