You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@thrift.apache.org by "henry.jykim@google" <he...@gmail.com> on 2014/01/14 02:49:39 UTC
any recommendable open-sources for c++ developers
hi, all
I am newbie to use the thrift library.
Our team’s legacy software is using CORBA very heavily.
yes, we got now the time to change it for more lighter, more faster.
There are 2 big libraries for bmt.
the first is ICE.
the second is THRIFT.
AYK, there are good and worse relatively.
I believe that the thrift is used very nicely in JAVA world. however does it be also used in C++ world?
does anybody be able to recommend good use-cases?
very thanks for your concerns.
RE: Methods to improve the performance of Thrift (C#) query for Hbase
Posted by Sanilpnair <sa...@arssoftware.com>.
Hi Jake,
Ok, thank you for your attention.
Regards
Sanil
-----Original Message-----
From: Jake Farrell [mailto:jfarrell@apache.org]
Sent: Thursday, January 16, 2014 12:24 PM
To: user@thrift.apache.org
Subject: Re: Methods to improve the performance of Thrift (C#) query for
Hbase
Hey Sanil
The filters you are talking about are specific to Apache HBase and are only
implemented using Apache Thrift as the transport. Please ask these
questions on the Apache HBase dev/user mailing list
-Jake
On Thu, Jan 16, 2014 at 1:45 AM, Sanilpnair
<sa...@arssoftware.com>wrote:
> Hi Jens,
>
> As we know Thrift is providing feature for filtering, this allows the user
> to perform server-side filtering when accessing HBase over Thrift.
>
> Eg: KeyOnlyFilter,FirstKeyOnlyFilter,PrefixFilter,ColumnPrefixFilter,
> MultipleColumnPrefixFilter etc
>
> Is KeyOnlyFilter/ FirstKeyOnlyFilter more faster(as compare to other
thrift
> filters)?
>
>
>
> Regards
> Sanil
>
>
>
> -----Original Message-----
> From: Sanilpnair [mailto:sanilpnair@arssoftware.com]
> Sent: Thursday, January 16, 2014 11:44 AM
> To: user@thrift.apache.org
> Subject: RE: Methods to improve the performance of Thrift (C#) query for
> Hbase
>
> Hi Jens,
>
> Thank you for your quick and valuable response. I will check on Hbase side
> anyway :)
>
>
> Regards
> Sanil
>
>
> -----Original Message-----
> From: Jens Geyer [mailto:jensgeyer@hotmail.com]
> Sent: Thursday, January 16, 2014 1:42 AM
> To: user@thrift.apache.org
> Subject: Re: Methods to improve the performance of Thrift (C#) query for
> Hbase
>
> Hi again,
>
> overlooked your qeustiosn at the bottom. Sorry about that.
>
> > 1) how many ways we can improve the performance(optimization) of Thrift
> > query to hbase.
>
> Again, I suspect this part being more an HBase issue. But I may be wrong,
> have no opportunity to test it.
>
> > 2) how to cache the Thrift query response on Hbase. Can we force to
cache
> > response on Hbase using thrift query(I couldn't find any additional
> > properties or attributes on Thrift scan/filter queries)
>
> Thrift does not provide any chaching mechanisms, if that is what you are
> looking for. However, you may want to serialize the data (using Thrift)
and
> put the serialized data into Memcached or Redis or the like. Start with
> Redis, if you don't have any of these.
>
> > 3) Is Thrift load balancing? how to manage load balancing of Thrift
> server
> > on Master node.
>
> Same here. To some extent, load balancing this is out of scope. There are
> some mechanisms, likeTThreadedServer or TThreadPool server, providing a
> very
>
> simple, round robin load balancing across threads. If you want a load
> balancer across processes or cluster nodes, although Thrift can help you
to
> achieve that, it does not provide anything like that out of the box.
>
> JensG
>
>
>
> -----Ursprüngliche Nachricht-----
> From: Jens Geyer
> Sent: Wednesday, January 15, 2014 9:00 PM
> To: user@thrift.apache.org
> Subject: Re: Methods to improve the performance of Thrift (C#) query for
> Hbase
>
> And you are sure that it is not an HBase problem?
> Just asking ...
>
>
> -----Ursprüngliche Nachricht-----
> From: Sanilpnair
> Sent: Wednesday, January 15, 2014 3:31 PM
> To: user@thrift.apache.org
> Subject: Methods to improve the performance of Thrift (C#) query for Hbase
>
> Hi All,
>
> I started Thrift coding (with c# and Hbase) recently. I need to fetch
> realtime data from the Hbase using Thrift, my problem is that "performance
> issue" with scanner - filter queries.
>
> Please let me know the performance enhancement methods while using thrift
> with c#. Sample code is given below:
>
> ---------------------------------------------------
> string TableName="testtable";
> Dictionary<byte[], byte[]> attributes = new Dictionary<byte[], byte[]>();
> TScan scanFilter = new TScan();
> scanFilter.FilterString =
> ToByteArray("((RowFilter(<=,'binary:1053_1371222000')) AND
> (RowFilter(>=,'binary:1053_1371217740'))) OR
> (RowFilter(<=,'binary:1055_1371222000')) AND
> (RowFilter(>=,'binary:1055_1371217740')))");
>
> var scanner = _client.scannerOpenWithScan(ToByteArray(TableName),
> scanFilter, attributes);
> for (var entry = _client.scannerGet(scanner); entry.Count > 0; entry =
> _client.scannerGet(scanner))
> {
>
> foreach (var rowResult in entry)
> {
> string rowkey =
> Encoding.UTF8.GetString(rowResult.Row);
> Console.Write("{0} => ", rowkey);
> }
>
> }
> _client.scannerClose(scanner);
>
> ----------------------------------------------------
> I tried scannerGetList also, single condition on filter is faster!
>
>
> My question is that:
>
> 1) how many ways we can improve the performance(optimization) of Thrift
> query to hbase.
> 2) how to cache the Thrift query response on Hbase. Can we force to cache
> response on Hbase using thrift query(I couldn't find any additional
> properties or attributes on Thrift scan/filter queries)
> 3) Is Thrift load balancing? how to manage load balancing of Thrift server
> on Master node.
>
>
> I am looking forward to hearing from you, hope you can help me.
>
>
> Regards
> Sanil
>
>
>
Re: Methods to improve the performance of Thrift (C#) query for Hbase
Posted by Jake Farrell <jf...@apache.org>.
Hey Sanil
The filters you are talking about are specific to Apache HBase and are only
implemented using Apache Thrift as the transport. Please ask these
questions on the Apache HBase dev/user mailing list
-Jake
On Thu, Jan 16, 2014 at 1:45 AM, Sanilpnair <sa...@arssoftware.com>wrote:
> Hi Jens,
>
> As we know Thrift is providing feature for filtering, this allows the user
> to perform server-side filtering when accessing HBase over Thrift.
>
> Eg: KeyOnlyFilter,FirstKeyOnlyFilter,PrefixFilter,ColumnPrefixFilter,
> MultipleColumnPrefixFilter etc
>
> Is KeyOnlyFilter/ FirstKeyOnlyFilter more faster(as compare to other thrift
> filters)?
>
>
>
> Regards
> Sanil
>
>
>
> -----Original Message-----
> From: Sanilpnair [mailto:sanilpnair@arssoftware.com]
> Sent: Thursday, January 16, 2014 11:44 AM
> To: user@thrift.apache.org
> Subject: RE: Methods to improve the performance of Thrift (C#) query for
> Hbase
>
> Hi Jens,
>
> Thank you for your quick and valuable response. I will check on Hbase side
> anyway :)
>
>
> Regards
> Sanil
>
>
> -----Original Message-----
> From: Jens Geyer [mailto:jensgeyer@hotmail.com]
> Sent: Thursday, January 16, 2014 1:42 AM
> To: user@thrift.apache.org
> Subject: Re: Methods to improve the performance of Thrift (C#) query for
> Hbase
>
> Hi again,
>
> overlooked your qeustiosn at the bottom. Sorry about that.
>
> > 1) how many ways we can improve the performance(optimization) of Thrift
> > query to hbase.
>
> Again, I suspect this part being more an HBase issue. But I may be wrong,
> have no opportunity to test it.
>
> > 2) how to cache the Thrift query response on Hbase. Can we force to cache
> > response on Hbase using thrift query(I couldn't find any additional
> > properties or attributes on Thrift scan/filter queries)
>
> Thrift does not provide any chaching mechanisms, if that is what you are
> looking for. However, you may want to serialize the data (using Thrift) and
> put the serialized data into Memcached or Redis or the like. Start with
> Redis, if you don't have any of these.
>
> > 3) Is Thrift load balancing? how to manage load balancing of Thrift
> server
> > on Master node.
>
> Same here. To some extent, load balancing this is out of scope. There are
> some mechanisms, likeTThreadedServer or TThreadPool server, providing a
> very
>
> simple, round robin load balancing across threads. If you want a load
> balancer across processes or cluster nodes, although Thrift can help you to
> achieve that, it does not provide anything like that out of the box.
>
> JensG
>
>
>
> -----Ursprüngliche Nachricht-----
> From: Jens Geyer
> Sent: Wednesday, January 15, 2014 9:00 PM
> To: user@thrift.apache.org
> Subject: Re: Methods to improve the performance of Thrift (C#) query for
> Hbase
>
> And you are sure that it is not an HBase problem?
> Just asking ...
>
>
> -----Ursprüngliche Nachricht-----
> From: Sanilpnair
> Sent: Wednesday, January 15, 2014 3:31 PM
> To: user@thrift.apache.org
> Subject: Methods to improve the performance of Thrift (C#) query for Hbase
>
> Hi All,
>
> I started Thrift coding (with c# and Hbase) recently. I need to fetch
> realtime data from the Hbase using Thrift, my problem is that "performance
> issue" with scanner - filter queries.
>
> Please let me know the performance enhancement methods while using thrift
> with c#. Sample code is given below:
>
> ---------------------------------------------------
> string TableName="testtable";
> Dictionary<byte[], byte[]> attributes = new Dictionary<byte[], byte[]>();
> TScan scanFilter = new TScan();
> scanFilter.FilterString =
> ToByteArray("((RowFilter(<=,'binary:1053_1371222000')) AND
> (RowFilter(>=,'binary:1053_1371217740'))) OR
> (RowFilter(<=,'binary:1055_1371222000')) AND
> (RowFilter(>=,'binary:1055_1371217740')))");
>
> var scanner = _client.scannerOpenWithScan(ToByteArray(TableName),
> scanFilter, attributes);
> for (var entry = _client.scannerGet(scanner); entry.Count > 0; entry =
> _client.scannerGet(scanner))
> {
>
> foreach (var rowResult in entry)
> {
> string rowkey =
> Encoding.UTF8.GetString(rowResult.Row);
> Console.Write("{0} => ", rowkey);
> }
>
> }
> _client.scannerClose(scanner);
>
> ----------------------------------------------------
> I tried scannerGetList also, single condition on filter is faster!
>
>
> My question is that:
>
> 1) how many ways we can improve the performance(optimization) of Thrift
> query to hbase.
> 2) how to cache the Thrift query response on Hbase. Can we force to cache
> response on Hbase using thrift query(I couldn't find any additional
> properties or attributes on Thrift scan/filter queries)
> 3) Is Thrift load balancing? how to manage load balancing of Thrift server
> on Master node.
>
>
> I am looking forward to hearing from you, hope you can help me.
>
>
> Regards
> Sanil
>
>
>
RE: Methods to improve the performance of Thrift (C#) query for Hbase
Posted by Sanilpnair <sa...@arssoftware.com>.
Hi Jens,
As we know Thrift is providing feature for filtering, this allows the user
to perform server-side filtering when accessing HBase over Thrift.
Eg: KeyOnlyFilter,FirstKeyOnlyFilter,PrefixFilter,ColumnPrefixFilter,
MultipleColumnPrefixFilter etc
Is KeyOnlyFilter/ FirstKeyOnlyFilter more faster(as compare to other thrift
filters)?
Regards
Sanil
-----Original Message-----
From: Sanilpnair [mailto:sanilpnair@arssoftware.com]
Sent: Thursday, January 16, 2014 11:44 AM
To: user@thrift.apache.org
Subject: RE: Methods to improve the performance of Thrift (C#) query for
Hbase
Hi Jens,
Thank you for your quick and valuable response. I will check on Hbase side
anyway :)
Regards
Sanil
-----Original Message-----
From: Jens Geyer [mailto:jensgeyer@hotmail.com]
Sent: Thursday, January 16, 2014 1:42 AM
To: user@thrift.apache.org
Subject: Re: Methods to improve the performance of Thrift (C#) query for
Hbase
Hi again,
overlooked your qeustiosn at the bottom. Sorry about that.
> 1) how many ways we can improve the performance(optimization) of Thrift
> query to hbase.
Again, I suspect this part being more an HBase issue. But I may be wrong,
have no opportunity to test it.
> 2) how to cache the Thrift query response on Hbase. Can we force to cache
> response on Hbase using thrift query(I couldn't find any additional
> properties or attributes on Thrift scan/filter queries)
Thrift does not provide any chaching mechanisms, if that is what you are
looking for. However, you may want to serialize the data (using Thrift) and
put the serialized data into Memcached or Redis or the like. Start with
Redis, if you don't have any of these.
> 3) Is Thrift load balancing? how to manage load balancing of Thrift server
> on Master node.
Same here. To some extent, load balancing this is out of scope. There are
some mechanisms, likeTThreadedServer or TThreadPool server, providing a very
simple, round robin load balancing across threads. If you want a load
balancer across processes or cluster nodes, although Thrift can help you to
achieve that, it does not provide anything like that out of the box.
JensG
-----Ursprüngliche Nachricht-----
From: Jens Geyer
Sent: Wednesday, January 15, 2014 9:00 PM
To: user@thrift.apache.org
Subject: Re: Methods to improve the performance of Thrift (C#) query for
Hbase
And you are sure that it is not an HBase problem?
Just asking ...
-----Ursprüngliche Nachricht-----
From: Sanilpnair
Sent: Wednesday, January 15, 2014 3:31 PM
To: user@thrift.apache.org
Subject: Methods to improve the performance of Thrift (C#) query for Hbase
Hi All,
I started Thrift coding (with c# and Hbase) recently. I need to fetch
realtime data from the Hbase using Thrift, my problem is that "performance
issue" with scanner - filter queries.
Please let me know the performance enhancement methods while using thrift
with c#. Sample code is given below:
---------------------------------------------------
string TableName="testtable";
Dictionary<byte[], byte[]> attributes = new Dictionary<byte[], byte[]>();
TScan scanFilter = new TScan();
scanFilter.FilterString =
ToByteArray("((RowFilter(<=,'binary:1053_1371222000')) AND
(RowFilter(>=,'binary:1053_1371217740'))) OR
(RowFilter(<=,'binary:1055_1371222000')) AND
(RowFilter(>=,'binary:1055_1371217740')))");
var scanner = _client.scannerOpenWithScan(ToByteArray(TableName),
scanFilter, attributes);
for (var entry = _client.scannerGet(scanner); entry.Count > 0; entry =
_client.scannerGet(scanner))
{
foreach (var rowResult in entry)
{
string rowkey =
Encoding.UTF8.GetString(rowResult.Row);
Console.Write("{0} => ", rowkey);
}
}
_client.scannerClose(scanner);
----------------------------------------------------
I tried scannerGetList also, single condition on filter is faster!
My question is that:
1) how many ways we can improve the performance(optimization) of Thrift
query to hbase.
2) how to cache the Thrift query response on Hbase. Can we force to cache
response on Hbase using thrift query(I couldn't find any additional
properties or attributes on Thrift scan/filter queries)
3) Is Thrift load balancing? how to manage load balancing of Thrift server
on Master node.
I am looking forward to hearing from you, hope you can help me.
Regards
Sanil
RE: Methods to improve the performance of Thrift (C#) query for Hbase
Posted by Sanilpnair <sa...@arssoftware.com>.
Hi Jens,
Thank you for your quick and valuable response. I will check on Hbase side
anyway :)
Regards
Sanil
ARS | Traffic & Transport Technology | India Sanil P Nair, Sr. Software
Engineer
131-135, Nila, Technopark, Trivandrum-695581; Tel: +91 0471 2527117, Ext
117; Mob: 9387 061600 - Web: www.arssoftware.com
-----Original Message-----
From: Jens Geyer [mailto:jensgeyer@hotmail.com]
Sent: Thursday, January 16, 2014 1:42 AM
To: user@thrift.apache.org
Subject: Re: Methods to improve the performance of Thrift (C#) query for
Hbase
Hi again,
overlooked your qeustiosn at the bottom. Sorry about that.
> 1) how many ways we can improve the performance(optimization) of Thrift
> query to hbase.
Again, I suspect this part being more an HBase issue. But I may be wrong,
have no opportunity to test it.
> 2) how to cache the Thrift query response on Hbase. Can we force to cache
> response on Hbase using thrift query(I couldn't find any additional
> properties or attributes on Thrift scan/filter queries)
Thrift does not provide any chaching mechanisms, if that is what you are
looking for. However, you may want to serialize the data (using Thrift) and
put the serialized data into Memcached or Redis or the like. Start with
Redis, if you don't have any of these.
> 3) Is Thrift load balancing? how to manage load balancing of Thrift server
> on Master node.
Same here. To some extent, load balancing this is out of scope. There are
some mechanisms, likeTThreadedServer or TThreadPool server, providing a very
simple, round robin load balancing across threads. If you want a load
balancer across processes or cluster nodes, although Thrift can help you to
achieve that, it does not provide anything like that out of the box.
JensG
-----Ursprüngliche Nachricht-----
From: Jens Geyer
Sent: Wednesday, January 15, 2014 9:00 PM
To: user@thrift.apache.org
Subject: Re: Methods to improve the performance of Thrift (C#) query for
Hbase
And you are sure that it is not an HBase problem?
Just asking ...
-----Ursprüngliche Nachricht-----
From: Sanilpnair
Sent: Wednesday, January 15, 2014 3:31 PM
To: user@thrift.apache.org
Subject: Methods to improve the performance of Thrift (C#) query for Hbase
Hi All,
I started Thrift coding (with c# and Hbase) recently. I need to fetch
realtime data from the Hbase using Thrift, my problem is that "performance
issue" with scanner - filter queries.
Please let me know the performance enhancement methods while using thrift
with c#. Sample code is given below:
---------------------------------------------------
string TableName="testtable";
Dictionary<byte[], byte[]> attributes = new Dictionary<byte[], byte[]>();
TScan scanFilter = new TScan();
scanFilter.FilterString =
ToByteArray("((RowFilter(<=,'binary:1053_1371222000')) AND
(RowFilter(>=,'binary:1053_1371217740'))) OR
(RowFilter(<=,'binary:1055_1371222000')) AND
(RowFilter(>=,'binary:1055_1371217740')))");
var scanner = _client.scannerOpenWithScan(ToByteArray(TableName),
scanFilter, attributes);
for (var entry = _client.scannerGet(scanner); entry.Count > 0; entry =
_client.scannerGet(scanner))
{
foreach (var rowResult in entry)
{
string rowkey =
Encoding.UTF8.GetString(rowResult.Row);
Console.Write("{0} => ", rowkey);
}
}
_client.scannerClose(scanner);
----------------------------------------------------
I tried scannerGetList also, single condition on filter is faster!
My question is that:
1) how many ways we can improve the performance(optimization) of Thrift
query to hbase.
2) how to cache the Thrift query response on Hbase. Can we force to cache
response on Hbase using thrift query(I couldn't find any additional
properties or attributes on Thrift scan/filter queries)
3) Is Thrift load balancing? how to manage load balancing of Thrift server
on Master node.
I am looking forward to hearing from you, hope you can help me.
Regards
Sanil
Re: Methods to improve the performance of Thrift (C#) query for Hbase
Posted by Jens Geyer <je...@hotmail.com>.
Hi again,
overlooked your qeustiosn at the bottom. Sorry about that.
> 1) how many ways we can improve the performance(optimization) of Thrift
> query to hbase.
Again, I suspect this part being more an HBase issue. But I may be wrong,
have no opportunity to test it.
> 2) how to cache the Thrift query response on Hbase. Can we force to cache
> response on Hbase using thrift query(I couldn't find any additional
> properties or attributes on Thrift scan/filter queries)
Thrift does not provide any chaching mechanisms, if that is what you are
looking for. However, you may want to serialize the data (using Thrift) and
put the serialized data into Memcached or Redis or the like. Start with
Redis, if you don't have any of these.
> 3) Is Thrift load balancing? how to manage load balancing of Thrift server
> on Master node.
Same here. To some extent, load balancing this is out of scope. There are
some mechanisms, likeTThreadedServer or TThreadPool server, providing a very
simple, round robin load balancing across threads. If you want a load
balancer across processes or cluster nodes, although Thrift can help you to
achieve that, it does not provide anything like that out of the box.
JensG
-----Ursprüngliche Nachricht-----
From: Jens Geyer
Sent: Wednesday, January 15, 2014 9:00 PM
To: user@thrift.apache.org
Subject: Re: Methods to improve the performance of Thrift (C#) query for
Hbase
And you are sure that it is not an HBase problem?
Just asking ...
-----Ursprüngliche Nachricht-----
From: Sanilpnair
Sent: Wednesday, January 15, 2014 3:31 PM
To: user@thrift.apache.org
Subject: Methods to improve the performance of Thrift (C#) query for Hbase
Hi All,
I started Thrift coding (with c# and Hbase) recently. I need to fetch
realtime data from the Hbase using Thrift, my problem is that "performance
issue" with scanner - filter queries.
Please let me know the performance enhancement methods while using thrift
with c#. Sample code is given below:
---------------------------------------------------
string TableName="testtable";
Dictionary<byte[], byte[]> attributes = new Dictionary<byte[], byte[]>();
TScan scanFilter = new TScan();
scanFilter.FilterString =
ToByteArray("((RowFilter(<=,'binary:1053_1371222000')) AND
(RowFilter(>=,'binary:1053_1371217740'))) OR
(RowFilter(<=,'binary:1055_1371222000')) AND
(RowFilter(>=,'binary:1055_1371217740')))");
var scanner = _client.scannerOpenWithScan(ToByteArray(TableName),
scanFilter, attributes);
for (var entry = _client.scannerGet(scanner); entry.Count > 0; entry =
_client.scannerGet(scanner))
{
foreach (var rowResult in entry)
{
string rowkey =
Encoding.UTF8.GetString(rowResult.Row);
Console.Write("{0} => ", rowkey);
}
}
_client.scannerClose(scanner);
----------------------------------------------------
I tried scannerGetList also, single condition on filter is faster!
My question is that:
1) how many ways we can improve the performance(optimization) of Thrift
query to hbase.
2) how to cache the Thrift query response on Hbase. Can we force to cache
response on Hbase using thrift query(I couldn't find any additional
properties or attributes on Thrift scan/filter queries)
3) Is Thrift load balancing? how to manage load balancing of Thrift server
on Master node.
I am looking forward to hearing from you, hope you can help me.
Regards
Sanil
Re: Methods to improve the performance of Thrift (C#) query for Hbase
Posted by Jens Geyer <je...@hotmail.com>.
And you are sure that it is not an HBase problem?
Just asking ...
-----Ursprüngliche Nachricht-----
From: Sanilpnair
Sent: Wednesday, January 15, 2014 3:31 PM
To: user@thrift.apache.org
Subject: Methods to improve the performance of Thrift (C#) query for Hbase
Hi All,
I started Thrift coding (with c# and Hbase) recently. I need to fetch
realtime data from the Hbase using Thrift, my problem is that "performance
issue" with scanner - filter queries.
Please let me know the performance enhancement methods while using thrift
with c#. Sample code is given below:
---------------------------------------------------
string TableName="testtable";
Dictionary<byte[], byte[]> attributes = new Dictionary<byte[], byte[]>();
TScan scanFilter = new TScan();
scanFilter.FilterString =
ToByteArray("((RowFilter(<=,'binary:1053_1371222000')) AND
(RowFilter(>=,'binary:1053_1371217740'))) OR
(RowFilter(<=,'binary:1055_1371222000')) AND
(RowFilter(>=,'binary:1055_1371217740')))");
var scanner = _client.scannerOpenWithScan(ToByteArray(TableName),
scanFilter, attributes);
for (var entry = _client.scannerGet(scanner); entry.Count > 0; entry =
_client.scannerGet(scanner))
{
foreach (var rowResult in entry)
{
string rowkey =
Encoding.UTF8.GetString(rowResult.Row);
Console.Write("{0} => ", rowkey);
}
}
_client.scannerClose(scanner);
----------------------------------------------------
I tried scannerGetList also, single condition on filter is faster!
My question is that:
1) how many ways we can improve the performance(optimization) of Thrift
query to hbase.
2) how to cache the Thrift query response on Hbase. Can we force to cache
response on Hbase using thrift query(I couldn't find any additional
properties or attributes on Thrift scan/filter queries)
3) Is Thrift load balancing? how to manage load balancing of Thrift server
on Master node.
I am looking forward to hearing from you, hope you can help me.
Regards
Sanil
Methods to improve the performance of Thrift (C#) query for Hbase
Posted by Sanilpnair <sa...@arssoftware.com>.
Hi All,
I started Thrift coding (with c# and Hbase) recently. I need to fetch
realtime data from the Hbase using Thrift, my problem is that "performance
issue" with scanner - filter queries.
Please let me know the performance enhancement methods while using thrift
with c#. Sample code is given below:
---------------------------------------------------
string TableName="testtable";
Dictionary<byte[], byte[]> attributes = new Dictionary<byte[], byte[]>();
TScan scanFilter = new TScan();
scanFilter.FilterString =
ToByteArray("((RowFilter(<=,'binary:1053_1371222000')) AND
(RowFilter(>=,'binary:1053_1371217740'))) OR
(RowFilter(<=,'binary:1055_1371222000')) AND
(RowFilter(>=,'binary:1055_1371217740')))");
var scanner = _client.scannerOpenWithScan(ToByteArray(TableName),
scanFilter, attributes);
for (var entry = _client.scannerGet(scanner); entry.Count > 0; entry =
_client.scannerGet(scanner))
{
foreach (var rowResult in entry)
{
string rowkey =
Encoding.UTF8.GetString(rowResult.Row);
Console.Write("{0} => ", rowkey);
}
}
_client.scannerClose(scanner);
----------------------------------------------------
I tried scannerGetList also, single condition on filter is faster!
My question is that:
1) how many ways we can improve the performance(optimization) of Thrift
query to hbase.
2) how to cache the Thrift query response on Hbase. Can we force to cache
response on Hbase using thrift query(I couldn't find any additional
properties or attributes on Thrift scan/filter queries)
3) Is Thrift load balancing? how to manage load balancing of Thrift server
on Master node.
I am looking forward to hearing from you, hope you can help me.
Regards
Sanil
Re: any recommendable open-sources for c++ developers
Posted by Rush Manbert <ru...@manbert.com>.
We use Thrift in both C++ and Java on Mac and Windows. Servers are Mac/Java, clients are Mac/Windows/C++. It all works very well. We also use C++ Thrift clients and servers within the client side machines and internally between server components. We even SWIG Thrift-based client side libraries for use by the server side code. And we also use it to "flatten" structures into buffers and files.
It is easy to write your own transports and protocols as extensions of the originals. It is easy to modify the code generator.
As I have said to a number of people, Thrift is the Swiss Army Knife of software.
- Rush
On Jan 13, 2014, at 5:49 PM, henry.jykim@google wrote:
> hi, all
> I am newbie to use the thrift library.
>
> Our team’s legacy software is using CORBA very heavily.
> yes, we got now the time to change it for more lighter, more faster.
>
> There are 2 big libraries for bmt.
> the first is ICE.
> the second is THRIFT.
>
> AYK, there are good and worse relatively.
>
> I believe that the thrift is used very nicely in JAVA world. however does it be also used in C++ world?
> does anybody be able to recommend good use-cases?
>
> very thanks for your concerns.