You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Andrew Purtell <ap...@apache.org> on 2010/11/24 19:15:31 UTC
REST compression support (was Re: question about meta data query intensity)
Regards compressing the HTTP transactions between the REST server and REST client we punted on this back when Stargate had a WAR target so we could push that off to the servlet container configuration. Thanks for the question, which reminded me... I have just committed HBASE-3275, which is a trivial patch to support Accept-Encoding: gzip,deflate
Index: src/main/java/org/apache/hadoop/hbase/rest/Main.java
===================================================================
--- src/main/java/org/apache/hadoop/hbase/rest/Main.java (revision 1038732)
+++ src/main/java/org/apache/hadoop/hbase/rest/Main.java (working copy)
@@ -37,6 +37,7 @@
import org.mortbay.jetty.Server;
import org.mortbay.jetty.servlet.Context;
import org.mortbay.jetty.servlet.ServletHolder;
+import org.mortbay.servlet.GzipFilter;
import com.sun.jersey.spi.container.servlet.ServletContainer;
@@ -132,6 +133,7 @@
// set up context
Context context = new Context(server, "/", Context.SESSIONS);
context.addServlet(sh, "/*");
+ context.addFilter(GzipFilter.class, "/*", 0);
server.start();
server.join();
Regards interactions between HBase client and server, there is no option available for compressing Hadoop RPC.
- Andy
--- On Wed, 11/24/10, Jack Levin <ma...@gmail.com> wrote:
> From: Jack Levin <ma...@gmail.com>
> Subject: Re: question about meta data query intensity
> To: user@hbase.apache.org, apurtell@apache.org
> Date: Wednesday, November 24, 2010, 9:25 AM
>
> Yes, but that does not alleviate CPU contention should there be too
> many queries to a single region server. On a separate topic, is
> 'compression' in the works for REST gateway? Similar to
> mysql_client_compression? We plan to drop in 500K or
> more queries at a time into the REST, and it would be interesting
> to see the performance gain against uncompressed data.
>
> Thanks.
>
> -Jack
Re: REST compression support (was Re: question about meta data query intensity)
Posted by Jack Levin <ma...@gmail.com>.
Btw, does it mean, I can send in a compressed query? Or only receive
compressed data from REST or both?
-Jack
On Wed, Nov 24, 2010 at 10:15 AM, Andrew Purtell <ap...@apache.org> wrote:
> Regards compressing the HTTP transactions between the REST server and REST client we punted on this back when Stargate had a WAR target so we could push that off to the servlet container configuration. Thanks for the question, which reminded me... I have just committed HBASE-3275, which is a trivial patch to support Accept-Encoding: gzip,deflate
>
> Index: src/main/java/org/apache/hadoop/hbase/rest/Main.java
> ===================================================================
> --- src/main/java/org/apache/hadoop/hbase/rest/Main.java (revision 1038732)
> +++ src/main/java/org/apache/hadoop/hbase/rest/Main.java (working copy)
> @@ -37,6 +37,7 @@
> import org.mortbay.jetty.Server;
> import org.mortbay.jetty.servlet.Context;
> import org.mortbay.jetty.servlet.ServletHolder;
> +import org.mortbay.servlet.GzipFilter;
>
> import com.sun.jersey.spi.container.servlet.ServletContainer;
>
> @@ -132,6 +133,7 @@
> // set up context
> Context context = new Context(server, "/", Context.SESSIONS);
> context.addServlet(sh, "/*");
> + context.addFilter(GzipFilter.class, "/*", 0);
>
> server.start();
> server.join();
>
> Regards interactions between HBase client and server, there is no option available for compressing Hadoop RPC.
>
> - Andy
>
>
> --- On Wed, 11/24/10, Jack Levin <ma...@gmail.com> wrote:
>
>> From: Jack Levin <ma...@gmail.com>
>> Subject: Re: question about meta data query intensity
>> To: user@hbase.apache.org, apurtell@apache.org
>> Date: Wednesday, November 24, 2010, 9:25 AM
>>
>> Yes, but that does not alleviate CPU contention should there be too
>> many queries to a single region server. On a separate topic, is
>> 'compression' in the works for REST gateway? Similar to
>> mysql_client_compression? We plan to drop in 500K or
>> more queries at a time into the REST, and it would be interesting
>> to see the performance gain against uncompressed data.
>>
>> Thanks.
>>
>> -Jack
>
>
>
>
>
Re: REST compression support (was Re: question about meta data query intensity)
Posted by Jack Levin <ma...@gmail.com>.
Thats great! Thanks. this will help us reduce network context
switching as we remove the need to pass a lot of uncompressed packets.
-Jack
On Wed, Nov 24, 2010 at 10:15 AM, Andrew Purtell <ap...@apache.org> wrote:
> Regards compressing the HTTP transactions between the REST server and REST client we punted on this back when Stargate had a WAR target so we could push that off to the servlet container configuration. Thanks for the question, which reminded me... I have just committed HBASE-3275, which is a trivial patch to support Accept-Encoding: gzip,deflate
>
> Index: src/main/java/org/apache/hadoop/hbase/rest/Main.java
> ===================================================================
> --- src/main/java/org/apache/hadoop/hbase/rest/Main.java (revision 1038732)
> +++ src/main/java/org/apache/hadoop/hbase/rest/Main.java (working copy)
> @@ -37,6 +37,7 @@
> import org.mortbay.jetty.Server;
> import org.mortbay.jetty.servlet.Context;
> import org.mortbay.jetty.servlet.ServletHolder;
> +import org.mortbay.servlet.GzipFilter;
>
> import com.sun.jersey.spi.container.servlet.ServletContainer;
>
> @@ -132,6 +133,7 @@
> // set up context
> Context context = new Context(server, "/", Context.SESSIONS);
> context.addServlet(sh, "/*");
> + context.addFilter(GzipFilter.class, "/*", 0);
>
> server.start();
> server.join();
>
> Regards interactions between HBase client and server, there is no option available for compressing Hadoop RPC.
>
> - Andy
>
>
> --- On Wed, 11/24/10, Jack Levin <ma...@gmail.com> wrote:
>
>> From: Jack Levin <ma...@gmail.com>
>> Subject: Re: question about meta data query intensity
>> To: user@hbase.apache.org, apurtell@apache.org
>> Date: Wednesday, November 24, 2010, 9:25 AM
>>
>> Yes, but that does not alleviate CPU contention should there be too
>> many queries to a single region server. On a separate topic, is
>> 'compression' in the works for REST gateway? Similar to
>> mysql_client_compression? We plan to drop in 500K or
>> more queries at a time into the REST, and it would be interesting
>> to see the performance gain against uncompressed data.
>>
>> Thanks.
>>
>> -Jack
>
>
>
>
>