You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Todd Lipcon <to...@cloudera.com> on 2010/09/03 02:18:22 UTC

Re: thrift for hbase in CDH3 broken ?

Hi Jinsong,

Are you sure that the port you're connecting to is indeed the thrift server?

Unfortunately both the HBase thrift server and the Hue namenode plugin
listen on port 9090, so you might be having an issue where your HBase client
is trying to connect to the Namenode server instead of HBase.

You can verify the ports using a command like "/sbin/fuser -n tcp 9090" to
see which pid has it open, then cross reference against sudo jps.

Thanks
-Todd

On Thu, Sep 2, 2010 at 4:40 PM, Jinsong Hu <ji...@hotmail.com> wrote:

> Hi, There,
>  I am trying to test and see if thrift for hbase works. I followed the
> example from
>
> http://www.workhabit.com/labs/centos-55-and-thriftscribe
> http://incubator.apache.org/thrift/
> http://wiki.apache.org/hadoop/Hbase/ThriftApi
>
> and wrote test code: I found that client.getTableNames();
> returns all table names successfully, but the scanner never returned
> any record. and it even throws exception:
>
> org.apache.thrift.TApplicationException: get failed: unknown result
>        at
> org.apache.hadoop.hbase.thrift.generated.Hbase$Client.recv_get(Hbase.java:785)
>        at
> org.apache.hadoop.hbase.thrift.generated.Hbase$Client.get(Hbase.java:750)
>        at
> org.apache.hadoop.hbase.thrift.HbaseThriftTest.testThriftAPI(HbaseThriftTest.java:73)
>        at
> org.apache.hadoop.hbase.thrift.HbaseThriftTest.main(HbaseThriftTest.java:128)
>
> I checked the syntax and the table and confirmed that the table does have
> record and it should
> return result, but it doesn't. Can anybody tell me what is wrong ?
>
> At this time, I am suspecting the thrift server shipped with CDH3 may have
> problem. if there anyway
> I can get this test code working ?
>
> I have to use thrift-0.2.0-incubating.tar.gz , as I found that the code
> generated from hbase-thrift project
> doesn't even compile with 0.3.0 and 0.4.0 thrift.
>
>
> Jimmy.
>
>
>
>
>
>
>   public void testThriftAPI()
>   {
>       // Make socket
>       TSocket transport = new TSocket("10.20.12.13", 9090);
>
>
>
>       // Wrap in a protocol
>       TBinaryProtocol protocol = new  TBinaryProtocol(transport);
>
>       Hbase.Client client = new Hbase.Client(protocol);
>
>       try {
>           transport.open();
>           List<byte[]> tableNamesList=client.getTableNames();
>           for (byte [] name : tableNamesList)
>           {
>               System.out.println(new String(name));
>           }
>
>           String tableName="HEARTBEAT_CLUSTER";
>           String startRow="";
>           List<byte[]> columns= new ArrayList<byte[]>();
>           columns.add("fields:time_format".getBytes());
>           columns.add("fields:customer_id".getBytes());
>
>           int aScannerId=0;
>           try {
>
>               TCell cell=client.get(tableName.getBytes(),
> "2010-08\tproduction-2".getBytes(), "fields:customer_id".getBytes());
>               if (cell != null)
>               {
>                  String value= new String( cell.value);
>                  System.out.println(value);
>               }
>               aScannerId=client.scannerOpen(tableName.getBytes(),
> startRow.getBytes(), columns);
>               TRowResult result=null;
>               for (int i=0; i< 10; i++)
>               {
>                   try {
>                   result=client.scannerGet(aScannerId);
>                   } catch (Exception ex1)
>                   {
>
>                   }
>                   if (result ==null) break;
>
>                   String timeFormat=result.getFieldValue(0).toString();
>                   String customerId=result.getFieldValue(1).toString();
>                   System.out.println(timeFormat + "\t" + customerId);
>               }
>           } catch (Exception ex)
>           {
>               ex.printStackTrace();
>           }
>           finally {
>               if (aScannerId >0 ) client.scannerClose(aScannerId);
>           }
>       } catch (Exception exp)
>       {
>           exp.printStackTrace();
>       }
>       finally{
>           transport.close();
>       }
>
>   }
>



-- 
Todd Lipcon
Software Engineer, Cloudera

Re: CDH3 has 2 versions of hbase executable, one work, another doesn't

Posted by Todd Lipcon <to...@cloudera.com>.
Hi Jimmy,

This is a known bug in our packaging:
https://issues.cloudera.org/browse/DISTRO-23

We'll resolve it in our next beta.

Thanks
-Todd

On Fri, Sep 3, 2010 at 4:31 PM, Jinsong Hu <ji...@hotmail.com> wrote:

> I noticed that CDH3 has 2 executable
>
> /usr/bin/hbase
>
> /usr/lib/hbase/bin/hbase
>
> I compared them and they are different. it turns out that I run
>
> /usr/bin/hbase shell
>
> and then list table, it works, but if I run
>
> /usr/lib/hbase/bin/hbase shell
>
> and list tables, it freezes. In the next distribution, please make
> /usr/lib/hbase/bin/hbase
> a softlink to /usr/bin/hbase , or the otherway. and make sure the
> executable works.
>
> Jimmy.
>



-- 
Todd Lipcon
Software Engineer, Cloudera

Re: CDH3 has 2 versions of hbase executable, one work, another doesn't

Posted by Edward Capriolo <ed...@gmail.com>.
Jimmy.

Many people across the forums seem to be confusing cdh with the
upstream hadoop/hive/hbase projects. In this case the problem is in
the cdh packaging and not in upstream hbase.

The proper place to take up packaging issues is with the packager.
When you are new to software this might be confusing.

To be clear hadoop, hive, and hbase are open source products that come
in a tar.gz. if you notice an issue in the core functionality that
would be appropriate for xxxx-user@apache , but if you are running
into a problem with a derivative package this may not be the place to
get that resolved.

On Friday, September 3, 2010, Jinsong Hu <ji...@hotmail.com> wrote:
> I noticed that CDH3 has 2 executable
>
> /usr/bin/hbase
>
> /usr/lib/hbase/bin/hbase
>
> I compared them and they are different. it turns out that I run
>
> /usr/bin/hbase shell
>
> and then list table, it works, but if I run
>
> /usr/lib/hbase/bin/hbase shell
>
> and list tables, it freezes. In the next distribution, please make /usr/lib/hbase/bin/hbase
> a softlink to /usr/bin/hbase , or the otherway. and make sure the executable works.
>
> Jimmy.
>

CDH3 has 2 versions of hbase executable, one work, another doesn't

Posted by Jinsong Hu <ji...@hotmail.com>.
I noticed that CDH3 has 2 executable

/usr/bin/hbase

/usr/lib/hbase/bin/hbase

I compared them and they are different. it turns out that I run

/usr/bin/hbase shell

and then list table, it works, but if I run

/usr/lib/hbase/bin/hbase shell

and list tables, it freezes. In the next distribution, please make 
/usr/lib/hbase/bin/hbase
a softlink to /usr/bin/hbase , or the otherway. and make sure the executable 
works.

Jimmy.

Re: thrift for hbase in CDH3 broken ?

Posted by Jinsong Hu <ji...@hotmail.com>.
which hbase-thrift version do you use ? I used hbase-0.89 version and my 
perl version is 5.8.8.
and the hbase-thrift version I used is the  hbase-thrift-0.1.tar.gz     from
http://code.google.com/p/hbase-thrift/

I tried again and it just doesn't work, complaining

getRow failed: unknown result at 
/usr/lib/perl5/site_perl/5.8.8/Hbase/Hbase.pm l
ine 5898.


I suspect it is the problem of Hbase.pm version that is not compatible with 
hbase-0.89 .
the http://code.google.com/p/hbase-thrift/ mention the required hbase 
version is 0.18, which
is too old.

Jimmy

--------------------------------------------------
From: "Igor Ranitovic" <ir...@gmail.com>
Sent: Wednesday, September 08, 2010 3:58 PM
To: <us...@hbase.apache.org>
Subject: Re: thrift for hbase in CDH3 broken ?

>> There is no firewall. As you can see, on the same client machine, I am 
>> able to get the ruby version of the code to work.
>> This confirms that the thrift server is not the problem. Basically I am 
>> just trying to fetch the same row of data
>> as that of the ruby program.
>>
>> I am not running thrift server on the same regionserver. I am running the 
>> thrift server on a standalone machine
>> that is configured to point to the zookeeper for the hbase cluster.
>>
>> since the ruby version of the client code works,  I would assume that the 
>> thrift server is not the problem.
>> I also tried java version and it doesn't work either. in the previous 
>> post somebody asked why I use java.
>> The reason is because I want to test and see if the thrift server works. 
>> I never managed to get java working,
>> even until now.
>>
>> Have you gotten the perl version to work ? Have you been able to read a 
>> row of data using perl ?
>
> It works for me. I run cdh3 on a single test node in pseudo-distributed 
> mode. You mentioned that you get an exception when fetching a row. What is 
> the exception? Try
>
> my $row;
> eval {
>     $row = $client->getRow('table', 'row');
> };
>
> if ($@) {
>     print Dumper($@);
> }
>
> i.
>
>> Jimmy.
>>
>> --------------------------------------------------
>> From: "Igor Ranitovic" <ir...@gmail.com>
>> Sent: Tuesday, September 07, 2010 8:18 PM
>> To: <us...@hbase.apache.org>
>> Subject: Re: thrift for hbase in CDH3 broken ?
>>
>>> Jinsong Hu wrote:
>>>> I tried, this doesn't work. I noticed
>>>> $transport->open();
>>>> is missing in this code. so I added it.
>>>
>>> Yup. Sorry about that. Copy and paste error :(
>>>
>>>> following code first successfully print all tables, then in the line 
>>>> getRow(), it throws exception, even with ruby client, the row data is 
>>>> there
>>>>
>>>>
>>>>
>>>>   $transport->open();
>>>>
>>>>
>>>>   my @names=$client->getTableNames();
>>>>
>>>>   print  Dumper(@names);
>>>>   print "\n";
>>>>
>>>> my $row = $client->getRow('table12345', "key123");
>>>>
>>>>   print  Dumper($row);
>>>>   print "\n";
>>>>
>>>>
>>>>
>>>>   $transport->close();
>>>
>>>
>>> So you can scan META table on the master, but can fetch a row from a RS.
>>> Are there any firewalls in place ? Are you running thrift servers on the 
>>> same nodes as region servers? What kind of exception do you get?
>>>
>>> i.
>>>
>>>> --------------------------------------------------
>>>> From: "Igor Ranitovic" <ir...@gmail.com>
>>>> Sent: Friday, September 03, 2010 11:45 AM
>>>> To: <us...@hbase.apache.org>
>>>> Subject: Re: thrift for hbase in CDH3 broken ?
>>>>
>>>>> Not should what the test code is...would this test your setup?
>>>>>
>>>>>
>>>>> #!/usr/bin/env perl
>>>>>
>>>>> use strict;
>>>>> use warnings;
>>>>>
>>>>> use Thrift::BinaryProtocol;
>>>>> use Thrift::BufferedTransport;
>>>>> use Thrift::Socket;
>>>>> use Hbase::Hbase;
>>>>> use Data::Dumper;
>>>>>
>>>>> my $sock = Thrift::Socket->new('127.0.0.1', '9090');
>>>>> $sock->setRecvTimeout(60000);
>>>>> my $transport = Thrift::BufferedTransport->new($sock);
>>>>> my $protocol = Thrift::BinaryProtocol->new($transport);
>>>>> my $client = Hbase::HbaseClient->new($protocol);
>>>>>
>>>>> my $row = $client->getRow('table_test', 'row_123');
>>>>> print Dumper($row);
>>>>>
>>>>> $transport->close();
>>>>>
>>>>>
>>>>> BTW, I am not sure why you would want to use java to talk to the HBase 
>>>>> via the thirft server.
>>>>>
>>>>> i.
>>>>>
>>>>>
>>>>> Jinsong Hu wrote:
>>>>>> by the way, does anybody have a perl version of the test code ?
>>>>>>
>>>>>> Jimmy
>>>>>>
>>>>>> --------------------------------------------------
>>>>>> From: "Jinsong Hu" <ji...@hotmail.com>
>>>>>> Sent: Friday, September 03, 2010 11:17 AM
>>>>>> To: <us...@hbase.apache.org>
>>>>>> Subject: Re: thrift for hbase in CDH3 broken ?
>>>>>>
>>>>>>>
>>>>>>> I tried your code and indeed it works. but the java version doesn't 
>>>>>>> work. so it looks like it is a bug
>>>>>>> of the java library supplied by the thrift-0.2.0 version.
>>>>>>>
>>>>>>> Jimmy.
>>>>>>> --------------------------------------------------
>>>>>>> From: "Alexey Kovyrin" <al...@kovyrin.net>
>>>>>>> Sent: Friday, September 03, 2010 12:31 AM
>>>>>>> To: <us...@hbase.apache.org>
>>>>>>> Subject: Re: thrift for hbase in CDH3 broken ?
>>>>>>>
>>>>>>>> yes, Centos 5.5 + CDH3b2
>>>>>>>>
>>>>>>>> On Fri, Sep 3, 2010 at 3:26 AM, Jinsong Hu <ji...@hotmail.com> 
>>>>>>>> wrote:
>>>>>>>>> are you using CDH3 distribution ?
>>>>>>>>>
>>>>>>>>> Jinsong
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --------------------------------------------------
>>>>>>>>> From: "Alexey Kovyrin" <al...@kovyrin.net>
>>>>>>>>> Sent: Friday, September 03, 2010 12:04 AM
>>>>>>>>> To: <us...@hbase.apache.org>
>>>>>>>>> Subject: Re: thrift for hbase in CDH3 broken ?
>>>>>>>>>
>>>>>>>>>> http://github.com/kovyrin/hbase-thrift-client-examples - just 
>>>>>>>>>> wrote
>>>>>>>>>> this example and tested it in our cluster, works as expected.
>>>>>>>>>> For this to work you'd need to install rubygems and thrift gem 
>>>>>>>>>> (gem
>>>>>>>>>> install thrift).
>>>>>>>>>>
>>>>>>>>>> On Fri, Sep 3, 2010 at 12:01 AM, Jinsong Hu 
>>>>>>>>>> <ji...@hotmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>> Can you send me some ruby test code and so I can try against the 
>>>>>>>>>>> latest
>>>>>>>>>>> CDH3
>>>>>>>>>>> ?
>>>>>>>>>>>
>>>>>>>>>>> Jimmy.
>>>>>>>>>>>
>>>>>>>>>>> --------------------------------------------------
>>>>>>>>>>> From: "Alexey Kovyrin" <al...@kovyrin.net>
>>>>>>>>>>> Sent: Thursday, September 02, 2010 8:15 PM
>>>>>>>>>>> To: <us...@hbase.apache.org>
>>>>>>>>>>> Subject: Re: thrift for hbase in CDH3 broken ?
>>>>>>>>>>>
>>>>>>>>>>>> We use it in Scribd.com. All clients are ruby web apps.
>>>>>>>>>>>>
>>>>>>>>>>>> On Thu, Sep 2, 2010 at 10:49 PM, Todd Lipcon 
>>>>>>>>>>>> <to...@cloudera.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Thu, Sep 2, 2010 at 5:35 PM, Jinsong Hu 
>>>>>>>>>>>>> <ji...@hotmail.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Yes, I confirmed that it is indeed thrift server.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> and the fact that the API
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>         List<byte[]> tableNamesList=client.getTableNames();
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>        for (byte [] name : tableNamesList)
>>>>>>>>>>>>>>>>        {
>>>>>>>>>>>>>>>>            System.out.println(new String(name));
>>>>>>>>>>>>>>>>        }
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> successfully printed all table names shows that it is indeed 
>>>>>>>>>>>>>> thrift
>>>>>>>>>>>>>> server.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> if it is hue, it won't print the table names.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Ah, sorry, I missed that in your original message. Not sure 
>>>>>>>>>>>>>> what's up,
>>>>>>>>>>>>>> then
>>>>>>>>>>>>>
>>>>>>>>>>>>> - we don't have any changes in CDH that would affect this. 
>>>>>>>>>>>>> Anyone here
>>>>>>>>>>>>> used
>>>>>>>>>>>>> thrift on 0.89.20100621?
>>>>>>>>>>>>>
>>>>>>>>>>>>> -Todd
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Jimmy.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --------------------------------------------------
>>>>>>>>>>>>>> From: "Todd Lipcon" <to...@cloudera.com>
>>>>>>>>>>>>>> Sent: Thursday, September 02, 2010 5:18 PM
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> To: <us...@hbase.apache.org>
>>>>>>>>>>>>>> Subject: Re: thrift for hbase in CDH3 broken ?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>  Hi Jinsong,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Are you sure that the port you're connecting to is indeed 
>>>>>>>>>>>>>>> the thrift
>>>>>>>>>>>>>>> server?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Unfortunately both the HBase thrift server and the Hue 
>>>>>>>>>>>>>>> namenode
>>>>>>>>>>>>>>> plugin
>>>>>>>>>>>>>>> listen on port 9090, so you might be having an issue where 
>>>>>>>>>>>>>>> your HBase
>>>>>>>>>>>>>>> client
>>>>>>>>>>>>>>> is trying to connect to the Namenode server instead of 
>>>>>>>>>>>>>>> HBase.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> You can verify the ports using a command like 
>>>>>>>>>>>>>>> "/sbin/fuser -n tcp
>>>>>>>>>>>>>>> 9090"
>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>> see which pid has it open, then cross reference against sudo 
>>>>>>>>>>>>>>> jps.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>>> -Todd
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Thu, Sep 2, 2010 at 4:40 PM, Jinsong Hu 
>>>>>>>>>>>>>>> <ji...@hotmail.com>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>  Hi, There,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>  I am trying to test and see if thrift for hbase works. I 
>>>>>>>>>>>>>>>> followed
>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>> example from
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> http://www.workhabit.com/labs/centos-55-and-thriftscribe
>>>>>>>>>>>>>>>> http://incubator.apache.org/thrift/
>>>>>>>>>>>>>>>> http://wiki.apache.org/hadoop/Hbase/ThriftApi
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> and wrote test code: I found that client.getTableNames();
>>>>>>>>>>>>>>>> returns all table names successfully, but the scanner never 
>>>>>>>>>>>>>>>> returned
>>>>>>>>>>>>>>>> any record. and it even throws exception:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> org.apache.thrift.TApplicationException: get failed: 
>>>>>>>>>>>>>>>> unknown result
>>>>>>>>>>>>>>>>     at
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> org.apache.hadoop.hbase.thrift.generated.Hbase$Client.recv_get(Hbase.java:785)
>>>>>>>>>>>>>>>>     at
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> org.apache.hadoop.hbase.thrift.generated.Hbase$Client.get(Hbase.java:750)
>>>>>>>>>>>>>>>>     at
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> org.apache.hadoop.hbase.thrift.HbaseThriftTest.testThriftAPI(HbaseThriftTest.java:73)
>>>>>>>>>>>>>>>>     at
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> org.apache.hadoop.hbase.thrift.HbaseThriftTest.main(HbaseThriftTest.java:128)
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I checked the syntax and the table and confirmed that the 
>>>>>>>>>>>>>>>> table does
>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>> record and it should
>>>>>>>>>>>>>>>> return result, but it doesn't. Can anybody tell me what is 
>>>>>>>>>>>>>>>> wrong ?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> At this time, I am suspecting the thrift server shipped 
>>>>>>>>>>>>>>>> with CDH3
>>>>>>>>>>>>>>>> may
>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>> problem. if there anyway
>>>>>>>>>>>>>>>> I can get this test code working ?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I have to use thrift-0.2.0-incubating.tar.gz , as I found 
>>>>>>>>>>>>>>>> that the
>>>>>>>>>>>>>>>> code
>>>>>>>>>>>>>>>> generated from hbase-thrift project
>>>>>>>>>>>>>>>> doesn't even compile with 0.3.0 and 0.4.0 thrift.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Jimmy.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>  public void testThriftAPI()
>>>>>>>>>>>>>>>>  {
>>>>>>>>>>>>>>>>    // Make socket
>>>>>>>>>>>>>>>>    TSocket transport = new TSocket("10.20.12.13", 9090);
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>    // Wrap in a protocol
>>>>>>>>>>>>>>>>    TBinaryProtocol protocol = new 
>>>>>>>>>>>>>>>> TBinaryProtocol(transport);
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>    Hbase.Client client = new Hbase.Client(protocol);
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>    try {
>>>>>>>>>>>>>>>>        transport.open();
>>>>>>>>>>>>>>>>        List<byte[]> tableNamesList=client.getTableNames();
>>>>>>>>>>>>>>>>        for (byte [] name : tableNamesList)
>>>>>>>>>>>>>>>>        {
>>>>>>>>>>>>>>>>            System.out.println(new String(name));
>>>>>>>>>>>>>>>>        }
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>        String tableName="HEARTBEAT_CLUSTER";
>>>>>>>>>>>>>>>>        String startRow="";
>>>>>>>>>>>>>>>>        List<byte[]> columns= new ArrayList<byte[]>();
>>>>>>>>>>>>>>>>        columns.add("fields:time_format".getBytes());
>>>>>>>>>>>>>>>>        columns.add("fields:customer_id".getBytes());
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>        int aScannerId=0;
>>>>>>>>>>>>>>>>        try {
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>            TCell cell=client.get(tableName.getBytes(),
>>>>>>>>>>>>>>>> "2010-08\tproduction-2".getBytes(),
>>>>>>>>>>>>>>>> "fields:customer_id".getBytes());
>>>>>>>>>>>>>>>>            if (cell != null)
>>>>>>>>>>>>>>>>            {
>>>>>>>>>>>>>>>>               String value= new String( cell.value);
>>>>>>>>>>>>>>>>               System.out.println(value);
>>>>>>>>>>>>>>>>            }
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> aScannerId=client.scannerOpen(tableName.getBytes(),
>>>>>>>>>>>>>>>> startRow.getBytes(), columns);
>>>>>>>>>>>>>>>>            TRowResult result=null;
>>>>>>>>>>>>>>>>            for (int i=0; i< 10; i++)
>>>>>>>>>>>>>>>>            {
>>>>>>>>>>>>>>>>                try {
>>>>>>>>>>>>>>>>                result=client.scannerGet(aScannerId);
>>>>>>>>>>>>>>>>                } catch (Exception ex1)
>>>>>>>>>>>>>>>>                {
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>                }
>>>>>>>>>>>>>>>>                if (result ==null) break;
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>                String 
>>>>>>>>>>>>>>>> timeFormat=result.getFieldValue(0).toString();
>>>>>>>>>>>>>>>>                String 
>>>>>>>>>>>>>>>> customerId=result.getFieldValue(1).toString();
>>>>>>>>>>>>>>>>                System.out.println(timeFormat + "\t" + 
>>>>>>>>>>>>>>>> customerId);
>>>>>>>>>>>>>>>>            }
>>>>>>>>>>>>>>>>        } catch (Exception ex)
>>>>>>>>>>>>>>>>        {
>>>>>>>>>>>>>>>>            ex.printStackTrace();
>>>>>>>>>>>>>>>>        }
>>>>>>>>>>>>>>>>        finally {
>>>>>>>>>>>>>>>>            if (aScannerId >0 ) 
>>>>>>>>>>>>>>>> client.scannerClose(aScannerId);
>>>>>>>>>>>>>>>>        }
>>>>>>>>>>>>>>>>    } catch (Exception exp)
>>>>>>>>>>>>>>>>    {
>>>>>>>>>>>>>>>>        exp.printStackTrace();
>>>>>>>>>>>>>>>>    }
>>>>>>>>>>>>>>>>    finally{
>>>>>>>>>>>>>>>>        transport.close();
>>>>>>>>>>>>>>>>    }
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>  }
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> -- 
>>>>>>>>>>>>>>> Todd Lipcon
>>>>>>>>>>>>>>> Software Engineer, Cloudera
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> -- 
>>>>>>>>>>>>> Todd Lipcon
>>>>>>>>>>>>> Software Engineer, Cloudera
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> -- 
>>>>>>>>>>>> Alexey Kovyrin
>>>>>>>>>>>> http://kovyrin.net/
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> -- 
>>>>>>>>>> Alexey Kovyrin
>>>>>>>>>> http://kovyrin.net/
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> -- 
>>>>>>>> Alexey Kovyrin
>>>>>>>> http://kovyrin.net/
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>>
>>
>
>

Re: thrift for hbase in CDH3 broken ?

Posted by Igor Ranitovic <ir...@gmail.com>.
> There is no firewall. As you can see, on the same client machine, I am 
> able to get the ruby version of the code to work.
> This confirms that the thrift server is not the problem. Basically I am 
> just trying to fetch the same row of data
> as that of the ruby program.
> 
> I am not running thrift server on the same regionserver. I am running 
> the thrift server on a standalone machine
> that is configured to point to the zookeeper for the hbase cluster.
> 
> since the ruby version of the client code works,  I would assume that 
> the thrift server is not the problem.
> I also tried java version and it doesn't work either. in the previous 
> post somebody asked why I use java.
> The reason is because I want to test and see if the thrift server works. 
> I never managed to get java working,
> even until now.
> 
> Have you gotten the perl version to work ? Have you been able to read a 
> row of data using perl ?

It works for me. I run cdh3 on a single test node in pseudo-distributed 
mode. You mentioned that you get an exception when fetching a row. What 
is the exception? Try

my $row;
eval {
     $row = $client->getRow('table', 'row');
};

if ($@) {
     print Dumper($@);
}

i.

> Jimmy.
> 
> --------------------------------------------------
> From: "Igor Ranitovic" <ir...@gmail.com>
> Sent: Tuesday, September 07, 2010 8:18 PM
> To: <us...@hbase.apache.org>
> Subject: Re: thrift for hbase in CDH3 broken ?
> 
>> Jinsong Hu wrote:
>>> I tried, this doesn't work. I noticed
>>> $transport->open();
>>> is missing in this code. so I added it.
>>
>> Yup. Sorry about that. Copy and paste error :(
>>
>>> following code first successfully print all tables, then in the line 
>>> getRow(), it throws exception, even with ruby client, the row data is 
>>> there
>>>
>>>
>>>
>>>   $transport->open();
>>>
>>>
>>>   my @names=$client->getTableNames();
>>>
>>>   print  Dumper(@names);
>>>   print "\n";
>>>
>>> my $row = $client->getRow('table12345', "key123");
>>>
>>>   print  Dumper($row);
>>>   print "\n";
>>>
>>>
>>>
>>>   $transport->close();
>>
>>
>> So you can scan META table on the master, but can fetch a row from a RS.
>> Are there any firewalls in place ? Are you running thrift servers on 
>> the same nodes as region servers? What kind of exception do you get?
>>
>> i.
>>
>>> --------------------------------------------------
>>> From: "Igor Ranitovic" <ir...@gmail.com>
>>> Sent: Friday, September 03, 2010 11:45 AM
>>> To: <us...@hbase.apache.org>
>>> Subject: Re: thrift for hbase in CDH3 broken ?
>>>
>>>> Not should what the test code is...would this test your setup?
>>>>
>>>>
>>>> #!/usr/bin/env perl
>>>>
>>>> use strict;
>>>> use warnings;
>>>>
>>>> use Thrift::BinaryProtocol;
>>>> use Thrift::BufferedTransport;
>>>> use Thrift::Socket;
>>>> use Hbase::Hbase;
>>>> use Data::Dumper;
>>>>
>>>> my $sock = Thrift::Socket->new('127.0.0.1', '9090');
>>>> $sock->setRecvTimeout(60000);
>>>> my $transport = Thrift::BufferedTransport->new($sock);
>>>> my $protocol = Thrift::BinaryProtocol->new($transport);
>>>> my $client = Hbase::HbaseClient->new($protocol);
>>>>
>>>> my $row = $client->getRow('table_test', 'row_123');
>>>> print Dumper($row);
>>>>
>>>> $transport->close();
>>>>
>>>>
>>>> BTW, I am not sure why you would want to use java to talk to the 
>>>> HBase via the thirft server.
>>>>
>>>> i.
>>>>
>>>>
>>>> Jinsong Hu wrote:
>>>>> by the way, does anybody have a perl version of the test code ?
>>>>>
>>>>> Jimmy
>>>>>
>>>>> --------------------------------------------------
>>>>> From: "Jinsong Hu" <ji...@hotmail.com>
>>>>> Sent: Friday, September 03, 2010 11:17 AM
>>>>> To: <us...@hbase.apache.org>
>>>>> Subject: Re: thrift for hbase in CDH3 broken ?
>>>>>
>>>>>>
>>>>>> I tried your code and indeed it works. but the java version 
>>>>>> doesn't work. so it looks like it is a bug
>>>>>> of the java library supplied by the thrift-0.2.0 version.
>>>>>>
>>>>>> Jimmy.
>>>>>> --------------------------------------------------
>>>>>> From: "Alexey Kovyrin" <al...@kovyrin.net>
>>>>>> Sent: Friday, September 03, 2010 12:31 AM
>>>>>> To: <us...@hbase.apache.org>
>>>>>> Subject: Re: thrift for hbase in CDH3 broken ?
>>>>>>
>>>>>>> yes, Centos 5.5 + CDH3b2
>>>>>>>
>>>>>>> On Fri, Sep 3, 2010 at 3:26 AM, Jinsong Hu 
>>>>>>> <ji...@hotmail.com> wrote:
>>>>>>>> are you using CDH3 distribution ?
>>>>>>>>
>>>>>>>> Jinsong
>>>>>>>>
>>>>>>>>
>>>>>>>> --------------------------------------------------
>>>>>>>> From: "Alexey Kovyrin" <al...@kovyrin.net>
>>>>>>>> Sent: Friday, September 03, 2010 12:04 AM
>>>>>>>> To: <us...@hbase.apache.org>
>>>>>>>> Subject: Re: thrift for hbase in CDH3 broken ?
>>>>>>>>
>>>>>>>>> http://github.com/kovyrin/hbase-thrift-client-examples - just 
>>>>>>>>> wrote
>>>>>>>>> this example and tested it in our cluster, works as expected.
>>>>>>>>> For this to work you'd need to install rubygems and thrift gem 
>>>>>>>>> (gem
>>>>>>>>> install thrift).
>>>>>>>>>
>>>>>>>>> On Fri, Sep 3, 2010 at 12:01 AM, Jinsong Hu 
>>>>>>>>> <ji...@hotmail.com>
>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> Can you send me some ruby test code and so I can try against 
>>>>>>>>>> the latest
>>>>>>>>>> CDH3
>>>>>>>>>> ?
>>>>>>>>>>
>>>>>>>>>> Jimmy.
>>>>>>>>>>
>>>>>>>>>> --------------------------------------------------
>>>>>>>>>> From: "Alexey Kovyrin" <al...@kovyrin.net>
>>>>>>>>>> Sent: Thursday, September 02, 2010 8:15 PM
>>>>>>>>>> To: <us...@hbase.apache.org>
>>>>>>>>>> Subject: Re: thrift for hbase in CDH3 broken ?
>>>>>>>>>>
>>>>>>>>>>> We use it in Scribd.com. All clients are ruby web apps.
>>>>>>>>>>>
>>>>>>>>>>> On Thu, Sep 2, 2010 at 10:49 PM, Todd Lipcon 
>>>>>>>>>>> <to...@cloudera.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> On Thu, Sep 2, 2010 at 5:35 PM, Jinsong Hu 
>>>>>>>>>>>> <ji...@hotmail.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Yes, I confirmed that it is indeed thrift server.
>>>>>>>>>>>>>
>>>>>>>>>>>>> and the fact that the API
>>>>>>>>>>>>>
>>>>>>>>>>>>>         List<byte[]> tableNamesList=client.getTableNames();
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>        for (byte [] name : tableNamesList)
>>>>>>>>>>>>>>>        {
>>>>>>>>>>>>>>>            System.out.println(new String(name));
>>>>>>>>>>>>>>>        }
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>> successfully printed all table names shows that it is 
>>>>>>>>>>>>> indeed thrift
>>>>>>>>>>>>> server.
>>>>>>>>>>>>>
>>>>>>>>>>>>> if it is hue, it won't print the table names.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Ah, sorry, I missed that in your original message. Not sure 
>>>>>>>>>>>>> what's up,
>>>>>>>>>>>>> then
>>>>>>>>>>>>
>>>>>>>>>>>> - we don't have any changes in CDH that would affect this. 
>>>>>>>>>>>> Anyone here
>>>>>>>>>>>> used
>>>>>>>>>>>> thrift on 0.89.20100621?
>>>>>>>>>>>>
>>>>>>>>>>>> -Todd
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>> Jimmy.
>>>>>>>>>>>>>
>>>>>>>>>>>>> --------------------------------------------------
>>>>>>>>>>>>> From: "Todd Lipcon" <to...@cloudera.com>
>>>>>>>>>>>>> Sent: Thursday, September 02, 2010 5:18 PM
>>>>>>>>>>>>>
>>>>>>>>>>>>> To: <us...@hbase.apache.org>
>>>>>>>>>>>>> Subject: Re: thrift for hbase in CDH3 broken ?
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>  Hi Jinsong,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Are you sure that the port you're connecting to is indeed 
>>>>>>>>>>>>>> the thrift
>>>>>>>>>>>>>> server?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Unfortunately both the HBase thrift server and the Hue 
>>>>>>>>>>>>>> namenode
>>>>>>>>>>>>>> plugin
>>>>>>>>>>>>>> listen on port 9090, so you might be having an issue where 
>>>>>>>>>>>>>> your HBase
>>>>>>>>>>>>>> client
>>>>>>>>>>>>>> is trying to connect to the Namenode server instead of HBase.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> You can verify the ports using a command like "/sbin/fuser 
>>>>>>>>>>>>>> -n tcp
>>>>>>>>>>>>>> 9090"
>>>>>>>>>>>>>> to
>>>>>>>>>>>>>> see which pid has it open, then cross reference against 
>>>>>>>>>>>>>> sudo jps.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>> -Todd
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Thu, Sep 2, 2010 at 4:40 PM, Jinsong Hu 
>>>>>>>>>>>>>> <ji...@hotmail.com>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>  Hi, There,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>  I am trying to test and see if thrift for hbase works. I 
>>>>>>>>>>>>>>> followed
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> example from
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> http://www.workhabit.com/labs/centos-55-and-thriftscribe
>>>>>>>>>>>>>>> http://incubator.apache.org/thrift/
>>>>>>>>>>>>>>> http://wiki.apache.org/hadoop/Hbase/ThriftApi
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> and wrote test code: I found that client.getTableNames();
>>>>>>>>>>>>>>> returns all table names successfully, but the scanner 
>>>>>>>>>>>>>>> never returned
>>>>>>>>>>>>>>> any record. and it even throws exception:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> org.apache.thrift.TApplicationException: get failed: 
>>>>>>>>>>>>>>> unknown result
>>>>>>>>>>>>>>>     at
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> org.apache.hadoop.hbase.thrift.generated.Hbase$Client.recv_get(Hbase.java:785) 
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>     at
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> org.apache.hadoop.hbase.thrift.generated.Hbase$Client.get(Hbase.java:750) 
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>     at
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> org.apache.hadoop.hbase.thrift.HbaseThriftTest.testThriftAPI(HbaseThriftTest.java:73) 
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>     at
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> org.apache.hadoop.hbase.thrift.HbaseThriftTest.main(HbaseThriftTest.java:128) 
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I checked the syntax and the table and confirmed that the 
>>>>>>>>>>>>>>> table does
>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>> record and it should
>>>>>>>>>>>>>>> return result, but it doesn't. Can anybody tell me what 
>>>>>>>>>>>>>>> is wrong ?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> At this time, I am suspecting the thrift server shipped 
>>>>>>>>>>>>>>> with CDH3
>>>>>>>>>>>>>>> may
>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>> problem. if there anyway
>>>>>>>>>>>>>>> I can get this test code working ?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I have to use thrift-0.2.0-incubating.tar.gz , as I found 
>>>>>>>>>>>>>>> that the
>>>>>>>>>>>>>>> code
>>>>>>>>>>>>>>> generated from hbase-thrift project
>>>>>>>>>>>>>>> doesn't even compile with 0.3.0 and 0.4.0 thrift.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Jimmy.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>  public void testThriftAPI()
>>>>>>>>>>>>>>>  {
>>>>>>>>>>>>>>>    // Make socket
>>>>>>>>>>>>>>>    TSocket transport = new TSocket("10.20.12.13", 9090);
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>    // Wrap in a protocol
>>>>>>>>>>>>>>>    TBinaryProtocol protocol = new 
>>>>>>>>>>>>>>> TBinaryProtocol(transport);
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>    Hbase.Client client = new Hbase.Client(protocol);
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>    try {
>>>>>>>>>>>>>>>        transport.open();
>>>>>>>>>>>>>>>        List<byte[]> tableNamesList=client.getTableNames();
>>>>>>>>>>>>>>>        for (byte [] name : tableNamesList)
>>>>>>>>>>>>>>>        {
>>>>>>>>>>>>>>>            System.out.println(new String(name));
>>>>>>>>>>>>>>>        }
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>        String tableName="HEARTBEAT_CLUSTER";
>>>>>>>>>>>>>>>        String startRow="";
>>>>>>>>>>>>>>>        List<byte[]> columns= new ArrayList<byte[]>();
>>>>>>>>>>>>>>>        columns.add("fields:time_format".getBytes());
>>>>>>>>>>>>>>>        columns.add("fields:customer_id".getBytes());
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>        int aScannerId=0;
>>>>>>>>>>>>>>>        try {
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>            TCell cell=client.get(tableName.getBytes(),
>>>>>>>>>>>>>>> "2010-08\tproduction-2".getBytes(),
>>>>>>>>>>>>>>> "fields:customer_id".getBytes());
>>>>>>>>>>>>>>>            if (cell != null)
>>>>>>>>>>>>>>>            {
>>>>>>>>>>>>>>>               String value= new String( cell.value);
>>>>>>>>>>>>>>>               System.out.println(value);
>>>>>>>>>>>>>>>            }
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> aScannerId=client.scannerOpen(tableName.getBytes(),
>>>>>>>>>>>>>>> startRow.getBytes(), columns);
>>>>>>>>>>>>>>>            TRowResult result=null;
>>>>>>>>>>>>>>>            for (int i=0; i< 10; i++)
>>>>>>>>>>>>>>>            {
>>>>>>>>>>>>>>>                try {
>>>>>>>>>>>>>>>                result=client.scannerGet(aScannerId);
>>>>>>>>>>>>>>>                } catch (Exception ex1)
>>>>>>>>>>>>>>>                {
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>                }
>>>>>>>>>>>>>>>                if (result ==null) break;
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>                String 
>>>>>>>>>>>>>>> timeFormat=result.getFieldValue(0).toString();
>>>>>>>>>>>>>>>                String 
>>>>>>>>>>>>>>> customerId=result.getFieldValue(1).toString();
>>>>>>>>>>>>>>>                System.out.println(timeFormat + "\t" + 
>>>>>>>>>>>>>>> customerId);
>>>>>>>>>>>>>>>            }
>>>>>>>>>>>>>>>        } catch (Exception ex)
>>>>>>>>>>>>>>>        {
>>>>>>>>>>>>>>>            ex.printStackTrace();
>>>>>>>>>>>>>>>        }
>>>>>>>>>>>>>>>        finally {
>>>>>>>>>>>>>>>            if (aScannerId >0 ) 
>>>>>>>>>>>>>>> client.scannerClose(aScannerId);
>>>>>>>>>>>>>>>        }
>>>>>>>>>>>>>>>    } catch (Exception exp)
>>>>>>>>>>>>>>>    {
>>>>>>>>>>>>>>>        exp.printStackTrace();
>>>>>>>>>>>>>>>    }
>>>>>>>>>>>>>>>    finally{
>>>>>>>>>>>>>>>        transport.close();
>>>>>>>>>>>>>>>    }
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>  }
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> -- 
>>>>>>>>>>>>>> Todd Lipcon
>>>>>>>>>>>>>> Software Engineer, Cloudera
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> -- 
>>>>>>>>>>>> Todd Lipcon
>>>>>>>>>>>> Software Engineer, Cloudera
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> -- 
>>>>>>>>>>> Alexey Kovyrin
>>>>>>>>>>> http://kovyrin.net/
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> -- 
>>>>>>>>> Alexey Kovyrin
>>>>>>>>> http://kovyrin.net/
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> -- 
>>>>>>> Alexey Kovyrin
>>>>>>> http://kovyrin.net/
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>
>>
>>
>

Re: thrift for hbase in CDH3 broken ?

Posted by Jinsong Hu <ji...@hotmail.com>.
There is no firewall. As you can see, on the same client machine, I am able 
to get the ruby version of the code to work.
This confirms that the thrift server is not the problem. Basically I am just 
trying to fetch the same row of data
as that of the ruby program.

I am not running thrift server on the same regionserver. I am running the 
thrift server on a standalone machine
that is configured to point to the zookeeper for the hbase cluster.

since the ruby version of the client code works,  I would assume that the 
thrift server is not the problem.
I also tried java version and it doesn't work either. in the previous post 
somebody asked why I use java.
The reason is because I want to test and see if the thrift server works. I 
never managed to get java working,
even until now.

Have you gotten the perl version to work ? Have you been able to read a row 
of data using perl ?

Jimmy.

--------------------------------------------------
From: "Igor Ranitovic" <ir...@gmail.com>
Sent: Tuesday, September 07, 2010 8:18 PM
To: <us...@hbase.apache.org>
Subject: Re: thrift for hbase in CDH3 broken ?

> Jinsong Hu wrote:
>> I tried, this doesn't work. I noticed
>> $transport->open();
>> is missing in this code. so I added it.
>
> Yup. Sorry about that. Copy and paste error :(
>
>> following code first successfully print all tables, then in the line 
>> getRow(), it throws exception, even with ruby client, the row data is 
>> there
>>
>>
>>
>>   $transport->open();
>>
>>
>>   my @names=$client->getTableNames();
>>
>>   print  Dumper(@names);
>>   print "\n";
>>
>> my $row = $client->getRow('table12345', "key123");
>>
>>   print  Dumper($row);
>>   print "\n";
>>
>>
>>
>>   $transport->close();
>
>
> So you can scan META table on the master, but can fetch a row from a RS.
> Are there any firewalls in place ? Are you running thrift servers on the 
> same nodes as region servers? What kind of exception do you get?
>
> i.
>
>> --------------------------------------------------
>> From: "Igor Ranitovic" <ir...@gmail.com>
>> Sent: Friday, September 03, 2010 11:45 AM
>> To: <us...@hbase.apache.org>
>> Subject: Re: thrift for hbase in CDH3 broken ?
>>
>>> Not should what the test code is...would this test your setup?
>>>
>>>
>>> #!/usr/bin/env perl
>>>
>>> use strict;
>>> use warnings;
>>>
>>> use Thrift::BinaryProtocol;
>>> use Thrift::BufferedTransport;
>>> use Thrift::Socket;
>>> use Hbase::Hbase;
>>> use Data::Dumper;
>>>
>>> my $sock = Thrift::Socket->new('127.0.0.1', '9090');
>>> $sock->setRecvTimeout(60000);
>>> my $transport = Thrift::BufferedTransport->new($sock);
>>> my $protocol = Thrift::BinaryProtocol->new($transport);
>>> my $client = Hbase::HbaseClient->new($protocol);
>>>
>>> my $row = $client->getRow('table_test', 'row_123');
>>> print Dumper($row);
>>>
>>> $transport->close();
>>>
>>>
>>> BTW, I am not sure why you would want to use java to talk to the HBase 
>>> via the thirft server.
>>>
>>> i.
>>>
>>>
>>> Jinsong Hu wrote:
>>>> by the way, does anybody have a perl version of the test code ?
>>>>
>>>> Jimmy
>>>>
>>>> --------------------------------------------------
>>>> From: "Jinsong Hu" <ji...@hotmail.com>
>>>> Sent: Friday, September 03, 2010 11:17 AM
>>>> To: <us...@hbase.apache.org>
>>>> Subject: Re: thrift for hbase in CDH3 broken ?
>>>>
>>>>>
>>>>> I tried your code and indeed it works. but the java version doesn't 
>>>>> work. so it looks like it is a bug
>>>>> of the java library supplied by the thrift-0.2.0 version.
>>>>>
>>>>> Jimmy.
>>>>> --------------------------------------------------
>>>>> From: "Alexey Kovyrin" <al...@kovyrin.net>
>>>>> Sent: Friday, September 03, 2010 12:31 AM
>>>>> To: <us...@hbase.apache.org>
>>>>> Subject: Re: thrift for hbase in CDH3 broken ?
>>>>>
>>>>>> yes, Centos 5.5 + CDH3b2
>>>>>>
>>>>>> On Fri, Sep 3, 2010 at 3:26 AM, Jinsong Hu <ji...@hotmail.com> 
>>>>>> wrote:
>>>>>>> are you using CDH3 distribution ?
>>>>>>>
>>>>>>> Jinsong
>>>>>>>
>>>>>>>
>>>>>>> --------------------------------------------------
>>>>>>> From: "Alexey Kovyrin" <al...@kovyrin.net>
>>>>>>> Sent: Friday, September 03, 2010 12:04 AM
>>>>>>> To: <us...@hbase.apache.org>
>>>>>>> Subject: Re: thrift for hbase in CDH3 broken ?
>>>>>>>
>>>>>>>> http://github.com/kovyrin/hbase-thrift-client-examples - just wrote
>>>>>>>> this example and tested it in our cluster, works as expected.
>>>>>>>> For this to work you'd need to install rubygems and thrift gem (gem
>>>>>>>> install thrift).
>>>>>>>>
>>>>>>>> On Fri, Sep 3, 2010 at 12:01 AM, Jinsong Hu 
>>>>>>>> <ji...@hotmail.com>
>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> Can you send me some ruby test code and so I can try against the 
>>>>>>>>> latest
>>>>>>>>> CDH3
>>>>>>>>> ?
>>>>>>>>>
>>>>>>>>> Jimmy.
>>>>>>>>>
>>>>>>>>> --------------------------------------------------
>>>>>>>>> From: "Alexey Kovyrin" <al...@kovyrin.net>
>>>>>>>>> Sent: Thursday, September 02, 2010 8:15 PM
>>>>>>>>> To: <us...@hbase.apache.org>
>>>>>>>>> Subject: Re: thrift for hbase in CDH3 broken ?
>>>>>>>>>
>>>>>>>>>> We use it in Scribd.com. All clients are ruby web apps.
>>>>>>>>>>
>>>>>>>>>> On Thu, Sep 2, 2010 at 10:49 PM, Todd Lipcon <to...@cloudera.com> 
>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>> On Thu, Sep 2, 2010 at 5:35 PM, Jinsong Hu 
>>>>>>>>>>> <ji...@hotmail.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Yes, I confirmed that it is indeed thrift server.
>>>>>>>>>>>>
>>>>>>>>>>>> and the fact that the API
>>>>>>>>>>>>
>>>>>>>>>>>>         List<byte[]> tableNamesList=client.getTableNames();
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>        for (byte [] name : tableNamesList)
>>>>>>>>>>>>>>        {
>>>>>>>>>>>>>>            System.out.println(new String(name));
>>>>>>>>>>>>>>        }
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>> successfully printed all table names shows that it is indeed 
>>>>>>>>>>>> thrift
>>>>>>>>>>>> server.
>>>>>>>>>>>>
>>>>>>>>>>>> if it is hue, it won't print the table names.
>>>>>>>>>>>>
>>>>>>>>>>>> Ah, sorry, I missed that in your original message. Not sure 
>>>>>>>>>>>> what's up,
>>>>>>>>>>>> then
>>>>>>>>>>>
>>>>>>>>>>> - we don't have any changes in CDH that would affect this. 
>>>>>>>>>>> Anyone here
>>>>>>>>>>> used
>>>>>>>>>>> thrift on 0.89.20100621?
>>>>>>>>>>>
>>>>>>>>>>> -Todd
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> Jimmy.
>>>>>>>>>>>>
>>>>>>>>>>>> --------------------------------------------------
>>>>>>>>>>>> From: "Todd Lipcon" <to...@cloudera.com>
>>>>>>>>>>>> Sent: Thursday, September 02, 2010 5:18 PM
>>>>>>>>>>>>
>>>>>>>>>>>> To: <us...@hbase.apache.org>
>>>>>>>>>>>> Subject: Re: thrift for hbase in CDH3 broken ?
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>  Hi Jinsong,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Are you sure that the port you're connecting to is indeed the 
>>>>>>>>>>>>> thrift
>>>>>>>>>>>>> server?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Unfortunately both the HBase thrift server and the Hue 
>>>>>>>>>>>>> namenode
>>>>>>>>>>>>> plugin
>>>>>>>>>>>>> listen on port 9090, so you might be having an issue where 
>>>>>>>>>>>>> your HBase
>>>>>>>>>>>>> client
>>>>>>>>>>>>> is trying to connect to the Namenode server instead of HBase.
>>>>>>>>>>>>>
>>>>>>>>>>>>> You can verify the ports using a command like "/sbin/fuser -n 
>>>>>>>>>>>>> tcp
>>>>>>>>>>>>> 9090"
>>>>>>>>>>>>> to
>>>>>>>>>>>>> see which pid has it open, then cross reference against sudo 
>>>>>>>>>>>>> jps.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>> -Todd
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Thu, Sep 2, 2010 at 4:40 PM, Jinsong Hu 
>>>>>>>>>>>>> <ji...@hotmail.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>  Hi, There,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>  I am trying to test and see if thrift for hbase works. I 
>>>>>>>>>>>>>> followed
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>> example from
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> http://www.workhabit.com/labs/centos-55-and-thriftscribe
>>>>>>>>>>>>>> http://incubator.apache.org/thrift/
>>>>>>>>>>>>>> http://wiki.apache.org/hadoop/Hbase/ThriftApi
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> and wrote test code: I found that client.getTableNames();
>>>>>>>>>>>>>> returns all table names successfully, but the scanner never 
>>>>>>>>>>>>>> returned
>>>>>>>>>>>>>> any record. and it even throws exception:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> org.apache.thrift.TApplicationException: get failed: unknown 
>>>>>>>>>>>>>> result
>>>>>>>>>>>>>>     at
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> org.apache.hadoop.hbase.thrift.generated.Hbase$Client.recv_get(Hbase.java:785)
>>>>>>>>>>>>>>     at
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> org.apache.hadoop.hbase.thrift.generated.Hbase$Client.get(Hbase.java:750)
>>>>>>>>>>>>>>     at
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> org.apache.hadoop.hbase.thrift.HbaseThriftTest.testThriftAPI(HbaseThriftTest.java:73)
>>>>>>>>>>>>>>     at
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> org.apache.hadoop.hbase.thrift.HbaseThriftTest.main(HbaseThriftTest.java:128)
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I checked the syntax and the table and confirmed that the 
>>>>>>>>>>>>>> table does
>>>>>>>>>>>>>> have
>>>>>>>>>>>>>> record and it should
>>>>>>>>>>>>>> return result, but it doesn't. Can anybody tell me what is 
>>>>>>>>>>>>>> wrong ?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> At this time, I am suspecting the thrift server shipped with 
>>>>>>>>>>>>>> CDH3
>>>>>>>>>>>>>> may
>>>>>>>>>>>>>> have
>>>>>>>>>>>>>> problem. if there anyway
>>>>>>>>>>>>>> I can get this test code working ?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I have to use thrift-0.2.0-incubating.tar.gz , as I found 
>>>>>>>>>>>>>> that the
>>>>>>>>>>>>>> code
>>>>>>>>>>>>>> generated from hbase-thrift project
>>>>>>>>>>>>>> doesn't even compile with 0.3.0 and 0.4.0 thrift.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Jimmy.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>  public void testThriftAPI()
>>>>>>>>>>>>>>  {
>>>>>>>>>>>>>>    // Make socket
>>>>>>>>>>>>>>    TSocket transport = new TSocket("10.20.12.13", 9090);
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>    // Wrap in a protocol
>>>>>>>>>>>>>>    TBinaryProtocol protocol = new 
>>>>>>>>>>>>>> TBinaryProtocol(transport);
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>    Hbase.Client client = new Hbase.Client(protocol);
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>    try {
>>>>>>>>>>>>>>        transport.open();
>>>>>>>>>>>>>>        List<byte[]> tableNamesList=client.getTableNames();
>>>>>>>>>>>>>>        for (byte [] name : tableNamesList)
>>>>>>>>>>>>>>        {
>>>>>>>>>>>>>>            System.out.println(new String(name));
>>>>>>>>>>>>>>        }
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>        String tableName="HEARTBEAT_CLUSTER";
>>>>>>>>>>>>>>        String startRow="";
>>>>>>>>>>>>>>        List<byte[]> columns= new ArrayList<byte[]>();
>>>>>>>>>>>>>>        columns.add("fields:time_format".getBytes());
>>>>>>>>>>>>>>        columns.add("fields:customer_id".getBytes());
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>        int aScannerId=0;
>>>>>>>>>>>>>>        try {
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>            TCell cell=client.get(tableName.getBytes(),
>>>>>>>>>>>>>> "2010-08\tproduction-2".getBytes(),
>>>>>>>>>>>>>> "fields:customer_id".getBytes());
>>>>>>>>>>>>>>            if (cell != null)
>>>>>>>>>>>>>>            {
>>>>>>>>>>>>>>               String value= new String( cell.value);
>>>>>>>>>>>>>>               System.out.println(value);
>>>>>>>>>>>>>>            }
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> aScannerId=client.scannerOpen(tableName.getBytes(),
>>>>>>>>>>>>>> startRow.getBytes(), columns);
>>>>>>>>>>>>>>            TRowResult result=null;
>>>>>>>>>>>>>>            for (int i=0; i< 10; i++)
>>>>>>>>>>>>>>            {
>>>>>>>>>>>>>>                try {
>>>>>>>>>>>>>>                result=client.scannerGet(aScannerId);
>>>>>>>>>>>>>>                } catch (Exception ex1)
>>>>>>>>>>>>>>                {
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>                }
>>>>>>>>>>>>>>                if (result ==null) break;
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>                String 
>>>>>>>>>>>>>> timeFormat=result.getFieldValue(0).toString();
>>>>>>>>>>>>>>                String 
>>>>>>>>>>>>>> customerId=result.getFieldValue(1).toString();
>>>>>>>>>>>>>>                System.out.println(timeFormat + "\t" + 
>>>>>>>>>>>>>> customerId);
>>>>>>>>>>>>>>            }
>>>>>>>>>>>>>>        } catch (Exception ex)
>>>>>>>>>>>>>>        {
>>>>>>>>>>>>>>            ex.printStackTrace();
>>>>>>>>>>>>>>        }
>>>>>>>>>>>>>>        finally {
>>>>>>>>>>>>>>            if (aScannerId >0 ) 
>>>>>>>>>>>>>> client.scannerClose(aScannerId);
>>>>>>>>>>>>>>        }
>>>>>>>>>>>>>>    } catch (Exception exp)
>>>>>>>>>>>>>>    {
>>>>>>>>>>>>>>        exp.printStackTrace();
>>>>>>>>>>>>>>    }
>>>>>>>>>>>>>>    finally{
>>>>>>>>>>>>>>        transport.close();
>>>>>>>>>>>>>>    }
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>  }
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> -- 
>>>>>>>>>>>>> Todd Lipcon
>>>>>>>>>>>>> Software Engineer, Cloudera
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> -- 
>>>>>>>>>>> Todd Lipcon
>>>>>>>>>>> Software Engineer, Cloudera
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> -- 
>>>>>>>>>> Alexey Kovyrin
>>>>>>>>>> http://kovyrin.net/
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> -- 
>>>>>>>> Alexey Kovyrin
>>>>>>>> http://kovyrin.net/
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> -- 
>>>>>> Alexey Kovyrin
>>>>>> http://kovyrin.net/
>>>>>>
>>>>>
>>>>
>>>
>>>
>>
>
>

Re: thrift for hbase in CDH3 broken ?

Posted by Igor Ranitovic <ir...@gmail.com>.
Jinsong Hu wrote:
> I tried, this doesn't work. I noticed
> $transport->open();
> is missing in this code. so I added it.

Yup. Sorry about that. Copy and paste error :(

> following code first successfully print all tables, then in the line 
> getRow(), it throws exception, even with ruby client, the row data is there
> 
> 
> 
>   $transport->open();
> 
> 
>   my @names=$client->getTableNames();
> 
>   print  Dumper(@names);
>   print "\n";
> 
> my $row = $client->getRow('table12345', "key123");
> 
>   print  Dumper($row);
>   print "\n";
> 
> 
> 
>   $transport->close();


So you can scan META table on the master, but can fetch a row from a RS.
Are there any firewalls in place ? Are you running thrift servers on the 
same nodes as region servers? What kind of exception do you get?

i.

> --------------------------------------------------
> From: "Igor Ranitovic" <ir...@gmail.com>
> Sent: Friday, September 03, 2010 11:45 AM
> To: <us...@hbase.apache.org>
> Subject: Re: thrift for hbase in CDH3 broken ?
> 
>> Not should what the test code is...would this test your setup?
>>
>>
>> #!/usr/bin/env perl
>>
>> use strict;
>> use warnings;
>>
>> use Thrift::BinaryProtocol;
>> use Thrift::BufferedTransport;
>> use Thrift::Socket;
>> use Hbase::Hbase;
>> use Data::Dumper;
>>
>> my $sock = Thrift::Socket->new('127.0.0.1', '9090');
>> $sock->setRecvTimeout(60000);
>> my $transport = Thrift::BufferedTransport->new($sock);
>> my $protocol = Thrift::BinaryProtocol->new($transport);
>> my $client = Hbase::HbaseClient->new($protocol);
>>
>> my $row = $client->getRow('table_test', 'row_123');
>> print Dumper($row);
>>
>> $transport->close();
>>
>>
>> BTW, I am not sure why you would want to use java to talk to the HBase 
>> via the thirft server.
>>
>> i.
>>
>>
>> Jinsong Hu wrote:
>>> by the way, does anybody have a perl version of the test code ?
>>>
>>> Jimmy
>>>
>>> --------------------------------------------------
>>> From: "Jinsong Hu" <ji...@hotmail.com>
>>> Sent: Friday, September 03, 2010 11:17 AM
>>> To: <us...@hbase.apache.org>
>>> Subject: Re: thrift for hbase in CDH3 broken ?
>>>
>>>>
>>>> I tried your code and indeed it works. but the java version doesn't 
>>>> work. so it looks like it is a bug
>>>> of the java library supplied by the thrift-0.2.0 version.
>>>>
>>>> Jimmy.
>>>> --------------------------------------------------
>>>> From: "Alexey Kovyrin" <al...@kovyrin.net>
>>>> Sent: Friday, September 03, 2010 12:31 AM
>>>> To: <us...@hbase.apache.org>
>>>> Subject: Re: thrift for hbase in CDH3 broken ?
>>>>
>>>>> yes, Centos 5.5 + CDH3b2
>>>>>
>>>>> On Fri, Sep 3, 2010 at 3:26 AM, Jinsong Hu <ji...@hotmail.com> 
>>>>> wrote:
>>>>>> are you using CDH3 distribution ?
>>>>>>
>>>>>> Jinsong
>>>>>>
>>>>>>
>>>>>> --------------------------------------------------
>>>>>> From: "Alexey Kovyrin" <al...@kovyrin.net>
>>>>>> Sent: Friday, September 03, 2010 12:04 AM
>>>>>> To: <us...@hbase.apache.org>
>>>>>> Subject: Re: thrift for hbase in CDH3 broken ?
>>>>>>
>>>>>>> http://github.com/kovyrin/hbase-thrift-client-examples - just wrote
>>>>>>> this example and tested it in our cluster, works as expected.
>>>>>>> For this to work you'd need to install rubygems and thrift gem (gem
>>>>>>> install thrift).
>>>>>>>
>>>>>>> On Fri, Sep 3, 2010 at 12:01 AM, Jinsong Hu <ji...@hotmail.com>
>>>>>>> wrote:
>>>>>>>>
>>>>>>>> Can you send me some ruby test code and so I can try against the 
>>>>>>>> latest
>>>>>>>> CDH3
>>>>>>>> ?
>>>>>>>>
>>>>>>>> Jimmy.
>>>>>>>>
>>>>>>>> --------------------------------------------------
>>>>>>>> From: "Alexey Kovyrin" <al...@kovyrin.net>
>>>>>>>> Sent: Thursday, September 02, 2010 8:15 PM
>>>>>>>> To: <us...@hbase.apache.org>
>>>>>>>> Subject: Re: thrift for hbase in CDH3 broken ?
>>>>>>>>
>>>>>>>>> We use it in Scribd.com. All clients are ruby web apps.
>>>>>>>>>
>>>>>>>>> On Thu, Sep 2, 2010 at 10:49 PM, Todd Lipcon 
>>>>>>>>> <to...@cloudera.com> wrote:
>>>>>>>>>>
>>>>>>>>>> On Thu, Sep 2, 2010 at 5:35 PM, Jinsong Hu 
>>>>>>>>>> <ji...@hotmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Yes, I confirmed that it is indeed thrift server.
>>>>>>>>>>>
>>>>>>>>>>> and the fact that the API
>>>>>>>>>>>
>>>>>>>>>>>         List<byte[]> tableNamesList=client.getTableNames();
>>>>>>>>>>>>>
>>>>>>>>>>>>>        for (byte [] name : tableNamesList)
>>>>>>>>>>>>>        {
>>>>>>>>>>>>>            System.out.println(new String(name));
>>>>>>>>>>>>>        }
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> successfully printed all table names shows that it is indeed 
>>>>>>>>>>> thrift
>>>>>>>>>>> server.
>>>>>>>>>>>
>>>>>>>>>>> if it is hue, it won't print the table names.
>>>>>>>>>>>
>>>>>>>>>>> Ah, sorry, I missed that in your original message. Not sure 
>>>>>>>>>>> what's up,
>>>>>>>>>>> then
>>>>>>>>>>
>>>>>>>>>> - we don't have any changes in CDH that would affect this. 
>>>>>>>>>> Anyone here
>>>>>>>>>> used
>>>>>>>>>> thrift on 0.89.20100621?
>>>>>>>>>>
>>>>>>>>>> -Todd
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> Jimmy.
>>>>>>>>>>>
>>>>>>>>>>> --------------------------------------------------
>>>>>>>>>>> From: "Todd Lipcon" <to...@cloudera.com>
>>>>>>>>>>> Sent: Thursday, September 02, 2010 5:18 PM
>>>>>>>>>>>
>>>>>>>>>>> To: <us...@hbase.apache.org>
>>>>>>>>>>> Subject: Re: thrift for hbase in CDH3 broken ?
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>  Hi Jinsong,
>>>>>>>>>>>>
>>>>>>>>>>>> Are you sure that the port you're connecting to is indeed 
>>>>>>>>>>>> the thrift
>>>>>>>>>>>> server?
>>>>>>>>>>>>
>>>>>>>>>>>> Unfortunately both the HBase thrift server and the Hue namenode
>>>>>>>>>>>> plugin
>>>>>>>>>>>> listen on port 9090, so you might be having an issue where 
>>>>>>>>>>>> your HBase
>>>>>>>>>>>> client
>>>>>>>>>>>> is trying to connect to the Namenode server instead of HBase.
>>>>>>>>>>>>
>>>>>>>>>>>> You can verify the ports using a command like "/sbin/fuser 
>>>>>>>>>>>> -n tcp
>>>>>>>>>>>> 9090"
>>>>>>>>>>>> to
>>>>>>>>>>>> see which pid has it open, then cross reference against sudo 
>>>>>>>>>>>> jps.
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks
>>>>>>>>>>>> -Todd
>>>>>>>>>>>>
>>>>>>>>>>>> On Thu, Sep 2, 2010 at 4:40 PM, Jinsong Hu 
>>>>>>>>>>>> <ji...@hotmail.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>  Hi, There,
>>>>>>>>>>>>>
>>>>>>>>>>>>>  I am trying to test and see if thrift for hbase works. I 
>>>>>>>>>>>>> followed
>>>>>>>>>>>>> the
>>>>>>>>>>>>> example from
>>>>>>>>>>>>>
>>>>>>>>>>>>> http://www.workhabit.com/labs/centos-55-and-thriftscribe
>>>>>>>>>>>>> http://incubator.apache.org/thrift/
>>>>>>>>>>>>> http://wiki.apache.org/hadoop/Hbase/ThriftApi
>>>>>>>>>>>>>
>>>>>>>>>>>>> and wrote test code: I found that client.getTableNames();
>>>>>>>>>>>>> returns all table names successfully, but the scanner never 
>>>>>>>>>>>>> returned
>>>>>>>>>>>>> any record. and it even throws exception:
>>>>>>>>>>>>>
>>>>>>>>>>>>> org.apache.thrift.TApplicationException: get failed: 
>>>>>>>>>>>>> unknown result
>>>>>>>>>>>>>     at
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> org.apache.hadoop.hbase.thrift.generated.Hbase$Client.recv_get(Hbase.java:785) 
>>>>>>>>>>>>>
>>>>>>>>>>>>>     at
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> org.apache.hadoop.hbase.thrift.generated.Hbase$Client.get(Hbase.java:750) 
>>>>>>>>>>>>>
>>>>>>>>>>>>>     at
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> org.apache.hadoop.hbase.thrift.HbaseThriftTest.testThriftAPI(HbaseThriftTest.java:73) 
>>>>>>>>>>>>>
>>>>>>>>>>>>>     at
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> org.apache.hadoop.hbase.thrift.HbaseThriftTest.main(HbaseThriftTest.java:128) 
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> I checked the syntax and the table and confirmed that the 
>>>>>>>>>>>>> table does
>>>>>>>>>>>>> have
>>>>>>>>>>>>> record and it should
>>>>>>>>>>>>> return result, but it doesn't. Can anybody tell me what is 
>>>>>>>>>>>>> wrong ?
>>>>>>>>>>>>>
>>>>>>>>>>>>> At this time, I am suspecting the thrift server shipped 
>>>>>>>>>>>>> with CDH3
>>>>>>>>>>>>> may
>>>>>>>>>>>>> have
>>>>>>>>>>>>> problem. if there anyway
>>>>>>>>>>>>> I can get this test code working ?
>>>>>>>>>>>>>
>>>>>>>>>>>>> I have to use thrift-0.2.0-incubating.tar.gz , as I found 
>>>>>>>>>>>>> that the
>>>>>>>>>>>>> code
>>>>>>>>>>>>> generated from hbase-thrift project
>>>>>>>>>>>>> doesn't even compile with 0.3.0 and 0.4.0 thrift.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Jimmy.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>  public void testThriftAPI()
>>>>>>>>>>>>>  {
>>>>>>>>>>>>>    // Make socket
>>>>>>>>>>>>>    TSocket transport = new TSocket("10.20.12.13", 9090);
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>    // Wrap in a protocol
>>>>>>>>>>>>>    TBinaryProtocol protocol = new  TBinaryProtocol(transport);
>>>>>>>>>>>>>
>>>>>>>>>>>>>    Hbase.Client client = new Hbase.Client(protocol);
>>>>>>>>>>>>>
>>>>>>>>>>>>>    try {
>>>>>>>>>>>>>        transport.open();
>>>>>>>>>>>>>        List<byte[]> tableNamesList=client.getTableNames();
>>>>>>>>>>>>>        for (byte [] name : tableNamesList)
>>>>>>>>>>>>>        {
>>>>>>>>>>>>>            System.out.println(new String(name));
>>>>>>>>>>>>>        }
>>>>>>>>>>>>>
>>>>>>>>>>>>>        String tableName="HEARTBEAT_CLUSTER";
>>>>>>>>>>>>>        String startRow="";
>>>>>>>>>>>>>        List<byte[]> columns= new ArrayList<byte[]>();
>>>>>>>>>>>>>        columns.add("fields:time_format".getBytes());
>>>>>>>>>>>>>        columns.add("fields:customer_id".getBytes());
>>>>>>>>>>>>>
>>>>>>>>>>>>>        int aScannerId=0;
>>>>>>>>>>>>>        try {
>>>>>>>>>>>>>
>>>>>>>>>>>>>            TCell cell=client.get(tableName.getBytes(),
>>>>>>>>>>>>> "2010-08\tproduction-2".getBytes(),
>>>>>>>>>>>>> "fields:customer_id".getBytes());
>>>>>>>>>>>>>            if (cell != null)
>>>>>>>>>>>>>            {
>>>>>>>>>>>>>               String value= new String( cell.value);
>>>>>>>>>>>>>               System.out.println(value);
>>>>>>>>>>>>>            }
>>>>>>>>>>>>>            aScannerId=client.scannerOpen(tableName.getBytes(),
>>>>>>>>>>>>> startRow.getBytes(), columns);
>>>>>>>>>>>>>            TRowResult result=null;
>>>>>>>>>>>>>            for (int i=0; i< 10; i++)
>>>>>>>>>>>>>            {
>>>>>>>>>>>>>                try {
>>>>>>>>>>>>>                result=client.scannerGet(aScannerId);
>>>>>>>>>>>>>                } catch (Exception ex1)
>>>>>>>>>>>>>                {
>>>>>>>>>>>>>
>>>>>>>>>>>>>                }
>>>>>>>>>>>>>                if (result ==null) break;
>>>>>>>>>>>>>
>>>>>>>>>>>>>                String 
>>>>>>>>>>>>> timeFormat=result.getFieldValue(0).toString();
>>>>>>>>>>>>>                String 
>>>>>>>>>>>>> customerId=result.getFieldValue(1).toString();
>>>>>>>>>>>>>                System.out.println(timeFormat + "\t" + 
>>>>>>>>>>>>> customerId);
>>>>>>>>>>>>>            }
>>>>>>>>>>>>>        } catch (Exception ex)
>>>>>>>>>>>>>        {
>>>>>>>>>>>>>            ex.printStackTrace();
>>>>>>>>>>>>>        }
>>>>>>>>>>>>>        finally {
>>>>>>>>>>>>>            if (aScannerId >0 ) 
>>>>>>>>>>>>> client.scannerClose(aScannerId);
>>>>>>>>>>>>>        }
>>>>>>>>>>>>>    } catch (Exception exp)
>>>>>>>>>>>>>    {
>>>>>>>>>>>>>        exp.printStackTrace();
>>>>>>>>>>>>>    }
>>>>>>>>>>>>>    finally{
>>>>>>>>>>>>>        transport.close();
>>>>>>>>>>>>>    }
>>>>>>>>>>>>>
>>>>>>>>>>>>>  }
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> -- 
>>>>>>>>>>>> Todd Lipcon
>>>>>>>>>>>> Software Engineer, Cloudera
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> -- 
>>>>>>>>>> Todd Lipcon
>>>>>>>>>> Software Engineer, Cloudera
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> -- 
>>>>>>>>> Alexey Kovyrin
>>>>>>>>> http://kovyrin.net/
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> -- 
>>>>>>> Alexey Kovyrin
>>>>>>> http://kovyrin.net/
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> -- 
>>>>> Alexey Kovyrin
>>>>> http://kovyrin.net/
>>>>>
>>>>
>>>
>>
>>
>

Re: thrift for hbase in CDH3 broken ?

Posted by Jinsong Hu <ji...@hotmail.com>.
anybody has a fix for the perl version of the hbase thrift API yet ?

Jimmy.

--------------------------------------------------
From: "Jinsong Hu" <ji...@hotmail.com>
Sent: Friday, September 03, 2010 12:16 PM
To: <us...@hbase.apache.org>
Subject: Re: thrift for hbase in CDH3 broken ?

> I tried, this doesn't work. I noticed
> $transport->open();
> is missing in this code. so I added it.
>
> following code first successfully print all tables, then in the line 
> getRow(), it throws exception, even with ruby client, the row data is 
> there
>
>
>
>   $transport->open();
>
>
>   my @names=$client->getTableNames();
>
>   print  Dumper(@names);
>   print "\n";
>
> my $row = $client->getRow('table12345', "key123");
>
>   print  Dumper($row);
>   print "\n";
>
>
>
>   $transport->close();
>
>
>
>
> --------------------------------------------------
> From: "Igor Ranitovic" <ir...@gmail.com>
> Sent: Friday, September 03, 2010 11:45 AM
> To: <us...@hbase.apache.org>
> Subject: Re: thrift for hbase in CDH3 broken ?
>
>> Not should what the test code is...would this test your setup?
>>
>>
>> #!/usr/bin/env perl
>>
>> use strict;
>> use warnings;
>>
>> use Thrift::BinaryProtocol;
>> use Thrift::BufferedTransport;
>> use Thrift::Socket;
>> use Hbase::Hbase;
>> use Data::Dumper;
>>
>> my $sock = Thrift::Socket->new('127.0.0.1', '9090');
>> $sock->setRecvTimeout(60000);
>> my $transport = Thrift::BufferedTransport->new($sock);
>> my $protocol = Thrift::BinaryProtocol->new($transport);
>> my $client = Hbase::HbaseClient->new($protocol);
>>
>> my $row = $client->getRow('table_test', 'row_123');
>> print Dumper($row);
>>
>> $transport->close();
>>
>>
>> BTW, I am not sure why you would want to use java to talk to the HBase 
>> via the thirft server.
>>
>> i.
>>
>>
>> Jinsong Hu wrote:
>>> by the way, does anybody have a perl version of the test code ?
>>>
>>> Jimmy
>>>
>>> --------------------------------------------------
>>> From: "Jinsong Hu" <ji...@hotmail.com>
>>> Sent: Friday, September 03, 2010 11:17 AM
>>> To: <us...@hbase.apache.org>
>>> Subject: Re: thrift for hbase in CDH3 broken ?
>>>
>>>>
>>>> I tried your code and indeed it works. but the java version doesn't 
>>>> work. so it looks like it is a bug
>>>> of the java library supplied by the thrift-0.2.0 version.
>>>>
>>>> Jimmy.
>>>> --------------------------------------------------
>>>> From: "Alexey Kovyrin" <al...@kovyrin.net>
>>>> Sent: Friday, September 03, 2010 12:31 AM
>>>> To: <us...@hbase.apache.org>
>>>> Subject: Re: thrift for hbase in CDH3 broken ?
>>>>
>>>>> yes, Centos 5.5 + CDH3b2
>>>>>
>>>>> On Fri, Sep 3, 2010 at 3:26 AM, Jinsong Hu <ji...@hotmail.com> 
>>>>> wrote:
>>>>>> are you using CDH3 distribution ?
>>>>>>
>>>>>> Jinsong
>>>>>>
>>>>>>
>>>>>> --------------------------------------------------
>>>>>> From: "Alexey Kovyrin" <al...@kovyrin.net>
>>>>>> Sent: Friday, September 03, 2010 12:04 AM
>>>>>> To: <us...@hbase.apache.org>
>>>>>> Subject: Re: thrift for hbase in CDH3 broken ?
>>>>>>
>>>>>>> http://github.com/kovyrin/hbase-thrift-client-examples - just wrote
>>>>>>> this example and tested it in our cluster, works as expected.
>>>>>>> For this to work you'd need to install rubygems and thrift gem (gem
>>>>>>> install thrift).
>>>>>>>
>>>>>>> On Fri, Sep 3, 2010 at 12:01 AM, Jinsong Hu <ji...@hotmail.com>
>>>>>>> wrote:
>>>>>>>>
>>>>>>>> Can you send me some ruby test code and so I can try against the 
>>>>>>>> latest
>>>>>>>> CDH3
>>>>>>>> ?
>>>>>>>>
>>>>>>>> Jimmy.
>>>>>>>>
>>>>>>>> --------------------------------------------------
>>>>>>>> From: "Alexey Kovyrin" <al...@kovyrin.net>
>>>>>>>> Sent: Thursday, September 02, 2010 8:15 PM
>>>>>>>> To: <us...@hbase.apache.org>
>>>>>>>> Subject: Re: thrift for hbase in CDH3 broken ?
>>>>>>>>
>>>>>>>>> We use it in Scribd.com. All clients are ruby web apps.
>>>>>>>>>
>>>>>>>>> On Thu, Sep 2, 2010 at 10:49 PM, Todd Lipcon <to...@cloudera.com> 
>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> On Thu, Sep 2, 2010 at 5:35 PM, Jinsong Hu 
>>>>>>>>>> <ji...@hotmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Yes, I confirmed that it is indeed thrift server.
>>>>>>>>>>>
>>>>>>>>>>> and the fact that the API
>>>>>>>>>>>
>>>>>>>>>>>         List<byte[]> tableNamesList=client.getTableNames();
>>>>>>>>>>>>>
>>>>>>>>>>>>>        for (byte [] name : tableNamesList)
>>>>>>>>>>>>>        {
>>>>>>>>>>>>>            System.out.println(new String(name));
>>>>>>>>>>>>>        }
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> successfully printed all table names shows that it is indeed 
>>>>>>>>>>> thrift
>>>>>>>>>>> server.
>>>>>>>>>>>
>>>>>>>>>>> if it is hue, it won't print the table names.
>>>>>>>>>>>
>>>>>>>>>>> Ah, sorry, I missed that in your original message. Not sure 
>>>>>>>>>>> what's up,
>>>>>>>>>>> then
>>>>>>>>>>
>>>>>>>>>> - we don't have any changes in CDH that would affect this. Anyone 
>>>>>>>>>> here
>>>>>>>>>> used
>>>>>>>>>> thrift on 0.89.20100621?
>>>>>>>>>>
>>>>>>>>>> -Todd
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> Jimmy.
>>>>>>>>>>>
>>>>>>>>>>> --------------------------------------------------
>>>>>>>>>>> From: "Todd Lipcon" <to...@cloudera.com>
>>>>>>>>>>> Sent: Thursday, September 02, 2010 5:18 PM
>>>>>>>>>>>
>>>>>>>>>>> To: <us...@hbase.apache.org>
>>>>>>>>>>> Subject: Re: thrift for hbase in CDH3 broken ?
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>  Hi Jinsong,
>>>>>>>>>>>>
>>>>>>>>>>>> Are you sure that the port you're connecting to is indeed the 
>>>>>>>>>>>> thrift
>>>>>>>>>>>> server?
>>>>>>>>>>>>
>>>>>>>>>>>> Unfortunately both the HBase thrift server and the Hue namenode
>>>>>>>>>>>> plugin
>>>>>>>>>>>> listen on port 9090, so you might be having an issue where your 
>>>>>>>>>>>> HBase
>>>>>>>>>>>> client
>>>>>>>>>>>> is trying to connect to the Namenode server instead of HBase.
>>>>>>>>>>>>
>>>>>>>>>>>> You can verify the ports using a command like "/sbin/fuser -n 
>>>>>>>>>>>> tcp
>>>>>>>>>>>> 9090"
>>>>>>>>>>>> to
>>>>>>>>>>>> see which pid has it open, then cross reference against sudo 
>>>>>>>>>>>> jps.
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks
>>>>>>>>>>>> -Todd
>>>>>>>>>>>>
>>>>>>>>>>>> On Thu, Sep 2, 2010 at 4:40 PM, Jinsong Hu 
>>>>>>>>>>>> <ji...@hotmail.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>  Hi, There,
>>>>>>>>>>>>>
>>>>>>>>>>>>>  I am trying to test and see if thrift for hbase works. I 
>>>>>>>>>>>>> followed
>>>>>>>>>>>>> the
>>>>>>>>>>>>> example from
>>>>>>>>>>>>>
>>>>>>>>>>>>> http://www.workhabit.com/labs/centos-55-and-thriftscribe
>>>>>>>>>>>>> http://incubator.apache.org/thrift/
>>>>>>>>>>>>> http://wiki.apache.org/hadoop/Hbase/ThriftApi
>>>>>>>>>>>>>
>>>>>>>>>>>>> and wrote test code: I found that client.getTableNames();
>>>>>>>>>>>>> returns all table names successfully, but the scanner never 
>>>>>>>>>>>>> returned
>>>>>>>>>>>>> any record. and it even throws exception:
>>>>>>>>>>>>>
>>>>>>>>>>>>> org.apache.thrift.TApplicationException: get failed: unknown 
>>>>>>>>>>>>> result
>>>>>>>>>>>>>     at
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> org.apache.hadoop.hbase.thrift.generated.Hbase$Client.recv_get(Hbase.java:785)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> org.apache.hadoop.hbase.thrift.generated.Hbase$Client.get(Hbase.java:750)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> org.apache.hadoop.hbase.thrift.HbaseThriftTest.testThriftAPI(HbaseThriftTest.java:73)
>>>>>>>>>>>>>     at
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> org.apache.hadoop.hbase.thrift.HbaseThriftTest.main(HbaseThriftTest.java:128)
>>>>>>>>>>>>>
>>>>>>>>>>>>> I checked the syntax and the table and confirmed that the 
>>>>>>>>>>>>> table does
>>>>>>>>>>>>> have
>>>>>>>>>>>>> record and it should
>>>>>>>>>>>>> return result, but it doesn't. Can anybody tell me what is 
>>>>>>>>>>>>> wrong ?
>>>>>>>>>>>>>
>>>>>>>>>>>>> At this time, I am suspecting the thrift server shipped with 
>>>>>>>>>>>>> CDH3
>>>>>>>>>>>>> may
>>>>>>>>>>>>> have
>>>>>>>>>>>>> problem. if there anyway
>>>>>>>>>>>>> I can get this test code working ?
>>>>>>>>>>>>>
>>>>>>>>>>>>> I have to use thrift-0.2.0-incubating.tar.gz , as I found that 
>>>>>>>>>>>>> the
>>>>>>>>>>>>> code
>>>>>>>>>>>>> generated from hbase-thrift project
>>>>>>>>>>>>> doesn't even compile with 0.3.0 and 0.4.0 thrift.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Jimmy.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>  public void testThriftAPI()
>>>>>>>>>>>>>  {
>>>>>>>>>>>>>    // Make socket
>>>>>>>>>>>>>    TSocket transport = new TSocket("10.20.12.13", 9090);
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>    // Wrap in a protocol
>>>>>>>>>>>>>    TBinaryProtocol protocol = new  TBinaryProtocol(transport);
>>>>>>>>>>>>>
>>>>>>>>>>>>>    Hbase.Client client = new Hbase.Client(protocol);
>>>>>>>>>>>>>
>>>>>>>>>>>>>    try {
>>>>>>>>>>>>>        transport.open();
>>>>>>>>>>>>>        List<byte[]> tableNamesList=client.getTableNames();
>>>>>>>>>>>>>        for (byte [] name : tableNamesList)
>>>>>>>>>>>>>        {
>>>>>>>>>>>>>            System.out.println(new String(name));
>>>>>>>>>>>>>        }
>>>>>>>>>>>>>
>>>>>>>>>>>>>        String tableName="HEARTBEAT_CLUSTER";
>>>>>>>>>>>>>        String startRow="";
>>>>>>>>>>>>>        List<byte[]> columns= new ArrayList<byte[]>();
>>>>>>>>>>>>>        columns.add("fields:time_format".getBytes());
>>>>>>>>>>>>>        columns.add("fields:customer_id".getBytes());
>>>>>>>>>>>>>
>>>>>>>>>>>>>        int aScannerId=0;
>>>>>>>>>>>>>        try {
>>>>>>>>>>>>>
>>>>>>>>>>>>>            TCell cell=client.get(tableName.getBytes(),
>>>>>>>>>>>>> "2010-08\tproduction-2".getBytes(),
>>>>>>>>>>>>> "fields:customer_id".getBytes());
>>>>>>>>>>>>>            if (cell != null)
>>>>>>>>>>>>>            {
>>>>>>>>>>>>>               String value= new String( cell.value);
>>>>>>>>>>>>>               System.out.println(value);
>>>>>>>>>>>>>            }
>>>>>>>>>>>>>            aScannerId=client.scannerOpen(tableName.getBytes(),
>>>>>>>>>>>>> startRow.getBytes(), columns);
>>>>>>>>>>>>>            TRowResult result=null;
>>>>>>>>>>>>>            for (int i=0; i< 10; i++)
>>>>>>>>>>>>>            {
>>>>>>>>>>>>>                try {
>>>>>>>>>>>>>                result=client.scannerGet(aScannerId);
>>>>>>>>>>>>>                } catch (Exception ex1)
>>>>>>>>>>>>>                {
>>>>>>>>>>>>>
>>>>>>>>>>>>>                }
>>>>>>>>>>>>>                if (result ==null) break;
>>>>>>>>>>>>>
>>>>>>>>>>>>>                String 
>>>>>>>>>>>>> timeFormat=result.getFieldValue(0).toString();
>>>>>>>>>>>>>                String 
>>>>>>>>>>>>> customerId=result.getFieldValue(1).toString();
>>>>>>>>>>>>>                System.out.println(timeFormat + "\t" + 
>>>>>>>>>>>>> customerId);
>>>>>>>>>>>>>            }
>>>>>>>>>>>>>        } catch (Exception ex)
>>>>>>>>>>>>>        {
>>>>>>>>>>>>>            ex.printStackTrace();
>>>>>>>>>>>>>        }
>>>>>>>>>>>>>        finally {
>>>>>>>>>>>>>            if (aScannerId >0 ) 
>>>>>>>>>>>>> client.scannerClose(aScannerId);
>>>>>>>>>>>>>        }
>>>>>>>>>>>>>    } catch (Exception exp)
>>>>>>>>>>>>>    {
>>>>>>>>>>>>>        exp.printStackTrace();
>>>>>>>>>>>>>    }
>>>>>>>>>>>>>    finally{
>>>>>>>>>>>>>        transport.close();
>>>>>>>>>>>>>    }
>>>>>>>>>>>>>
>>>>>>>>>>>>>  }
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> -- 
>>>>>>>>>>>> Todd Lipcon
>>>>>>>>>>>> Software Engineer, Cloudera
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> -- 
>>>>>>>>>> Todd Lipcon
>>>>>>>>>> Software Engineer, Cloudera
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> -- 
>>>>>>>>> Alexey Kovyrin
>>>>>>>>> http://kovyrin.net/
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> -- 
>>>>>>> Alexey Kovyrin
>>>>>>> http://kovyrin.net/
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> -- 
>>>>> Alexey Kovyrin
>>>>> http://kovyrin.net/
>>>>>
>>>>
>>>
>>
>>
>

Re: thrift for hbase in CDH3 broken ?

Posted by Jinsong Hu <ji...@hotmail.com>.
I tried, this doesn't work. I noticed
$transport->open();
is missing in this code. so I added it.

 following code first successfully print all tables, then in the line 
getRow(), it throws exception, even with ruby client, the row data is there



   $transport->open();


   my @names=$client->getTableNames();

   print  Dumper(@names);
   print "\n";

 my $row = $client->getRow('table12345', "key123");

   print  Dumper($row);
   print "\n";



   $transport->close();




--------------------------------------------------
From: "Igor Ranitovic" <ir...@gmail.com>
Sent: Friday, September 03, 2010 11:45 AM
To: <us...@hbase.apache.org>
Subject: Re: thrift for hbase in CDH3 broken ?

> Not should what the test code is...would this test your setup?
>
>
> #!/usr/bin/env perl
>
> use strict;
> use warnings;
>
> use Thrift::BinaryProtocol;
> use Thrift::BufferedTransport;
> use Thrift::Socket;
> use Hbase::Hbase;
> use Data::Dumper;
>
> my $sock = Thrift::Socket->new('127.0.0.1', '9090');
> $sock->setRecvTimeout(60000);
> my $transport = Thrift::BufferedTransport->new($sock);
> my $protocol = Thrift::BinaryProtocol->new($transport);
> my $client = Hbase::HbaseClient->new($protocol);
>
> my $row = $client->getRow('table_test', 'row_123');
> print Dumper($row);
>
> $transport->close();
>
>
> BTW, I am not sure why you would want to use java to talk to the HBase via 
> the thirft server.
>
> i.
>
>
> Jinsong Hu wrote:
>> by the way, does anybody have a perl version of the test code ?
>>
>> Jimmy
>>
>> --------------------------------------------------
>> From: "Jinsong Hu" <ji...@hotmail.com>
>> Sent: Friday, September 03, 2010 11:17 AM
>> To: <us...@hbase.apache.org>
>> Subject: Re: thrift for hbase in CDH3 broken ?
>>
>>>
>>> I tried your code and indeed it works. but the java version doesn't 
>>> work. so it looks like it is a bug
>>> of the java library supplied by the thrift-0.2.0 version.
>>>
>>> Jimmy.
>>> --------------------------------------------------
>>> From: "Alexey Kovyrin" <al...@kovyrin.net>
>>> Sent: Friday, September 03, 2010 12:31 AM
>>> To: <us...@hbase.apache.org>
>>> Subject: Re: thrift for hbase in CDH3 broken ?
>>>
>>>> yes, Centos 5.5 + CDH3b2
>>>>
>>>> On Fri, Sep 3, 2010 at 3:26 AM, Jinsong Hu <ji...@hotmail.com> 
>>>> wrote:
>>>>> are you using CDH3 distribution ?
>>>>>
>>>>> Jinsong
>>>>>
>>>>>
>>>>> --------------------------------------------------
>>>>> From: "Alexey Kovyrin" <al...@kovyrin.net>
>>>>> Sent: Friday, September 03, 2010 12:04 AM
>>>>> To: <us...@hbase.apache.org>
>>>>> Subject: Re: thrift for hbase in CDH3 broken ?
>>>>>
>>>>>> http://github.com/kovyrin/hbase-thrift-client-examples - just wrote
>>>>>> this example and tested it in our cluster, works as expected.
>>>>>> For this to work you'd need to install rubygems and thrift gem (gem
>>>>>> install thrift).
>>>>>>
>>>>>> On Fri, Sep 3, 2010 at 12:01 AM, Jinsong Hu <ji...@hotmail.com>
>>>>>> wrote:
>>>>>>>
>>>>>>> Can you send me some ruby test code and so I can try against the 
>>>>>>> latest
>>>>>>> CDH3
>>>>>>> ?
>>>>>>>
>>>>>>> Jimmy.
>>>>>>>
>>>>>>> --------------------------------------------------
>>>>>>> From: "Alexey Kovyrin" <al...@kovyrin.net>
>>>>>>> Sent: Thursday, September 02, 2010 8:15 PM
>>>>>>> To: <us...@hbase.apache.org>
>>>>>>> Subject: Re: thrift for hbase in CDH3 broken ?
>>>>>>>
>>>>>>>> We use it in Scribd.com. All clients are ruby web apps.
>>>>>>>>
>>>>>>>> On Thu, Sep 2, 2010 at 10:49 PM, Todd Lipcon <to...@cloudera.com> 
>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> On Thu, Sep 2, 2010 at 5:35 PM, Jinsong Hu 
>>>>>>>>> <ji...@hotmail.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Yes, I confirmed that it is indeed thrift server.
>>>>>>>>>>
>>>>>>>>>> and the fact that the API
>>>>>>>>>>
>>>>>>>>>>         List<byte[]> tableNamesList=client.getTableNames();
>>>>>>>>>>>>
>>>>>>>>>>>>        for (byte [] name : tableNamesList)
>>>>>>>>>>>>        {
>>>>>>>>>>>>            System.out.println(new String(name));
>>>>>>>>>>>>        }
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>> successfully printed all table names shows that it is indeed 
>>>>>>>>>> thrift
>>>>>>>>>> server.
>>>>>>>>>>
>>>>>>>>>> if it is hue, it won't print the table names.
>>>>>>>>>>
>>>>>>>>>> Ah, sorry, I missed that in your original message. Not sure 
>>>>>>>>>> what's up,
>>>>>>>>>> then
>>>>>>>>>
>>>>>>>>> - we don't have any changes in CDH that would affect this. Anyone 
>>>>>>>>> here
>>>>>>>>> used
>>>>>>>>> thrift on 0.89.20100621?
>>>>>>>>>
>>>>>>>>> -Todd
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> Jimmy.
>>>>>>>>>>
>>>>>>>>>> --------------------------------------------------
>>>>>>>>>> From: "Todd Lipcon" <to...@cloudera.com>
>>>>>>>>>> Sent: Thursday, September 02, 2010 5:18 PM
>>>>>>>>>>
>>>>>>>>>> To: <us...@hbase.apache.org>
>>>>>>>>>> Subject: Re: thrift for hbase in CDH3 broken ?
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>  Hi Jinsong,
>>>>>>>>>>>
>>>>>>>>>>> Are you sure that the port you're connecting to is indeed the 
>>>>>>>>>>> thrift
>>>>>>>>>>> server?
>>>>>>>>>>>
>>>>>>>>>>> Unfortunately both the HBase thrift server and the Hue namenode
>>>>>>>>>>> plugin
>>>>>>>>>>> listen on port 9090, so you might be having an issue where your 
>>>>>>>>>>> HBase
>>>>>>>>>>> client
>>>>>>>>>>> is trying to connect to the Namenode server instead of HBase.
>>>>>>>>>>>
>>>>>>>>>>> You can verify the ports using a command like "/sbin/fuser -n 
>>>>>>>>>>> tcp
>>>>>>>>>>> 9090"
>>>>>>>>>>> to
>>>>>>>>>>> see which pid has it open, then cross reference against sudo 
>>>>>>>>>>> jps.
>>>>>>>>>>>
>>>>>>>>>>> Thanks
>>>>>>>>>>> -Todd
>>>>>>>>>>>
>>>>>>>>>>> On Thu, Sep 2, 2010 at 4:40 PM, Jinsong Hu 
>>>>>>>>>>> <ji...@hotmail.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>  Hi, There,
>>>>>>>>>>>>
>>>>>>>>>>>>  I am trying to test and see if thrift for hbase works. I 
>>>>>>>>>>>> followed
>>>>>>>>>>>> the
>>>>>>>>>>>> example from
>>>>>>>>>>>>
>>>>>>>>>>>> http://www.workhabit.com/labs/centos-55-and-thriftscribe
>>>>>>>>>>>> http://incubator.apache.org/thrift/
>>>>>>>>>>>> http://wiki.apache.org/hadoop/Hbase/ThriftApi
>>>>>>>>>>>>
>>>>>>>>>>>> and wrote test code: I found that client.getTableNames();
>>>>>>>>>>>> returns all table names successfully, but the scanner never 
>>>>>>>>>>>> returned
>>>>>>>>>>>> any record. and it even throws exception:
>>>>>>>>>>>>
>>>>>>>>>>>> org.apache.thrift.TApplicationException: get failed: unknown 
>>>>>>>>>>>> result
>>>>>>>>>>>>     at
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> org.apache.hadoop.hbase.thrift.generated.Hbase$Client.recv_get(Hbase.java:785)
>>>>>>>>>>>>     at
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> org.apache.hadoop.hbase.thrift.generated.Hbase$Client.get(Hbase.java:750)
>>>>>>>>>>>>     at
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> org.apache.hadoop.hbase.thrift.HbaseThriftTest.testThriftAPI(HbaseThriftTest.java:73)
>>>>>>>>>>>>     at
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> org.apache.hadoop.hbase.thrift.HbaseThriftTest.main(HbaseThriftTest.java:128)
>>>>>>>>>>>>
>>>>>>>>>>>> I checked the syntax and the table and confirmed that the table 
>>>>>>>>>>>> does
>>>>>>>>>>>> have
>>>>>>>>>>>> record and it should
>>>>>>>>>>>> return result, but it doesn't. Can anybody tell me what is 
>>>>>>>>>>>> wrong ?
>>>>>>>>>>>>
>>>>>>>>>>>> At this time, I am suspecting the thrift server shipped with 
>>>>>>>>>>>> CDH3
>>>>>>>>>>>> may
>>>>>>>>>>>> have
>>>>>>>>>>>> problem. if there anyway
>>>>>>>>>>>> I can get this test code working ?
>>>>>>>>>>>>
>>>>>>>>>>>> I have to use thrift-0.2.0-incubating.tar.gz , as I found that 
>>>>>>>>>>>> the
>>>>>>>>>>>> code
>>>>>>>>>>>> generated from hbase-thrift project
>>>>>>>>>>>> doesn't even compile with 0.3.0 and 0.4.0 thrift.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Jimmy.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>  public void testThriftAPI()
>>>>>>>>>>>>  {
>>>>>>>>>>>>    // Make socket
>>>>>>>>>>>>    TSocket transport = new TSocket("10.20.12.13", 9090);
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>    // Wrap in a protocol
>>>>>>>>>>>>    TBinaryProtocol protocol = new  TBinaryProtocol(transport);
>>>>>>>>>>>>
>>>>>>>>>>>>    Hbase.Client client = new Hbase.Client(protocol);
>>>>>>>>>>>>
>>>>>>>>>>>>    try {
>>>>>>>>>>>>        transport.open();
>>>>>>>>>>>>        List<byte[]> tableNamesList=client.getTableNames();
>>>>>>>>>>>>        for (byte [] name : tableNamesList)
>>>>>>>>>>>>        {
>>>>>>>>>>>>            System.out.println(new String(name));
>>>>>>>>>>>>        }
>>>>>>>>>>>>
>>>>>>>>>>>>        String tableName="HEARTBEAT_CLUSTER";
>>>>>>>>>>>>        String startRow="";
>>>>>>>>>>>>        List<byte[]> columns= new ArrayList<byte[]>();
>>>>>>>>>>>>        columns.add("fields:time_format".getBytes());
>>>>>>>>>>>>        columns.add("fields:customer_id".getBytes());
>>>>>>>>>>>>
>>>>>>>>>>>>        int aScannerId=0;
>>>>>>>>>>>>        try {
>>>>>>>>>>>>
>>>>>>>>>>>>            TCell cell=client.get(tableName.getBytes(),
>>>>>>>>>>>> "2010-08\tproduction-2".getBytes(),
>>>>>>>>>>>> "fields:customer_id".getBytes());
>>>>>>>>>>>>            if (cell != null)
>>>>>>>>>>>>            {
>>>>>>>>>>>>               String value= new String( cell.value);
>>>>>>>>>>>>               System.out.println(value);
>>>>>>>>>>>>            }
>>>>>>>>>>>>            aScannerId=client.scannerOpen(tableName.getBytes(),
>>>>>>>>>>>> startRow.getBytes(), columns);
>>>>>>>>>>>>            TRowResult result=null;
>>>>>>>>>>>>            for (int i=0; i< 10; i++)
>>>>>>>>>>>>            {
>>>>>>>>>>>>                try {
>>>>>>>>>>>>                result=client.scannerGet(aScannerId);
>>>>>>>>>>>>                } catch (Exception ex1)
>>>>>>>>>>>>                {
>>>>>>>>>>>>
>>>>>>>>>>>>                }
>>>>>>>>>>>>                if (result ==null) break;
>>>>>>>>>>>>
>>>>>>>>>>>>                String 
>>>>>>>>>>>> timeFormat=result.getFieldValue(0).toString();
>>>>>>>>>>>>                String 
>>>>>>>>>>>> customerId=result.getFieldValue(1).toString();
>>>>>>>>>>>>                System.out.println(timeFormat + "\t" + 
>>>>>>>>>>>> customerId);
>>>>>>>>>>>>            }
>>>>>>>>>>>>        } catch (Exception ex)
>>>>>>>>>>>>        {
>>>>>>>>>>>>            ex.printStackTrace();
>>>>>>>>>>>>        }
>>>>>>>>>>>>        finally {
>>>>>>>>>>>>            if (aScannerId >0 ) client.scannerClose(aScannerId);
>>>>>>>>>>>>        }
>>>>>>>>>>>>    } catch (Exception exp)
>>>>>>>>>>>>    {
>>>>>>>>>>>>        exp.printStackTrace();
>>>>>>>>>>>>    }
>>>>>>>>>>>>    finally{
>>>>>>>>>>>>        transport.close();
>>>>>>>>>>>>    }
>>>>>>>>>>>>
>>>>>>>>>>>>  }
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> -- 
>>>>>>>>>>> Todd Lipcon
>>>>>>>>>>> Software Engineer, Cloudera
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> -- 
>>>>>>>>> Todd Lipcon
>>>>>>>>> Software Engineer, Cloudera
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> -- 
>>>>>>>> Alexey Kovyrin
>>>>>>>> http://kovyrin.net/
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> -- 
>>>>>> Alexey Kovyrin
>>>>>> http://kovyrin.net/
>>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> -- 
>>>> Alexey Kovyrin
>>>> http://kovyrin.net/
>>>>
>>>
>>
>
>

Re: thrift for hbase in CDH3 broken ?

Posted by Igor Ranitovic <ir...@gmail.com>.
Not should what the test code is...would this test your setup?


#!/usr/bin/env perl

use strict;
use warnings;

use Thrift::BinaryProtocol;
use Thrift::BufferedTransport;
use Thrift::Socket;
use Hbase::Hbase;
use Data::Dumper;

my $sock = Thrift::Socket->new('127.0.0.1', '9090');
$sock->setRecvTimeout(60000);
my $transport = Thrift::BufferedTransport->new($sock);
my $protocol = Thrift::BinaryProtocol->new($transport);
my $client = Hbase::HbaseClient->new($protocol);

my $row = $client->getRow('table_test', 'row_123');
print Dumper($row);

$transport->close();


BTW, I am not sure why you would want to use java to talk to the HBase 
via the thirft server.

i.


Jinsong Hu wrote:
> by the way, does anybody have a perl version of the test code ?
> 
> Jimmy
> 
> --------------------------------------------------
> From: "Jinsong Hu" <ji...@hotmail.com>
> Sent: Friday, September 03, 2010 11:17 AM
> To: <us...@hbase.apache.org>
> Subject: Re: thrift for hbase in CDH3 broken ?
> 
>>
>> I tried your code and indeed it works. but the java version doesn't 
>> work. so it looks like it is a bug
>> of the java library supplied by the thrift-0.2.0 version.
>>
>> Jimmy.
>> --------------------------------------------------
>> From: "Alexey Kovyrin" <al...@kovyrin.net>
>> Sent: Friday, September 03, 2010 12:31 AM
>> To: <us...@hbase.apache.org>
>> Subject: Re: thrift for hbase in CDH3 broken ?
>>
>>> yes, Centos 5.5 + CDH3b2
>>>
>>> On Fri, Sep 3, 2010 at 3:26 AM, Jinsong Hu <ji...@hotmail.com> 
>>> wrote:
>>>> are you using CDH3 distribution ?
>>>>
>>>> Jinsong
>>>>
>>>>
>>>> --------------------------------------------------
>>>> From: "Alexey Kovyrin" <al...@kovyrin.net>
>>>> Sent: Friday, September 03, 2010 12:04 AM
>>>> To: <us...@hbase.apache.org>
>>>> Subject: Re: thrift for hbase in CDH3 broken ?
>>>>
>>>>> http://github.com/kovyrin/hbase-thrift-client-examples - just wrote
>>>>> this example and tested it in our cluster, works as expected.
>>>>> For this to work you'd need to install rubygems and thrift gem (gem
>>>>> install thrift).
>>>>>
>>>>> On Fri, Sep 3, 2010 at 12:01 AM, Jinsong Hu <ji...@hotmail.com>
>>>>> wrote:
>>>>>>
>>>>>> Can you send me some ruby test code and so I can try against the 
>>>>>> latest
>>>>>> CDH3
>>>>>> ?
>>>>>>
>>>>>> Jimmy.
>>>>>>
>>>>>> --------------------------------------------------
>>>>>> From: "Alexey Kovyrin" <al...@kovyrin.net>
>>>>>> Sent: Thursday, September 02, 2010 8:15 PM
>>>>>> To: <us...@hbase.apache.org>
>>>>>> Subject: Re: thrift for hbase in CDH3 broken ?
>>>>>>
>>>>>>> We use it in Scribd.com. All clients are ruby web apps.
>>>>>>>
>>>>>>> On Thu, Sep 2, 2010 at 10:49 PM, Todd Lipcon <to...@cloudera.com> 
>>>>>>> wrote:
>>>>>>>>
>>>>>>>> On Thu, Sep 2, 2010 at 5:35 PM, Jinsong Hu <ji...@hotmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Yes, I confirmed that it is indeed thrift server.
>>>>>>>>>
>>>>>>>>> and the fact that the API
>>>>>>>>>
>>>>>>>>>         List<byte[]> tableNamesList=client.getTableNames();
>>>>>>>>>>>
>>>>>>>>>>>        for (byte [] name : tableNamesList)
>>>>>>>>>>>        {
>>>>>>>>>>>            System.out.println(new String(name));
>>>>>>>>>>>        }
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>> successfully printed all table names shows that it is indeed 
>>>>>>>>> thrift
>>>>>>>>> server.
>>>>>>>>>
>>>>>>>>> if it is hue, it won't print the table names.
>>>>>>>>>
>>>>>>>>> Ah, sorry, I missed that in your original message. Not sure 
>>>>>>>>> what's up,
>>>>>>>>> then
>>>>>>>>
>>>>>>>> - we don't have any changes in CDH that would affect this. 
>>>>>>>> Anyone here
>>>>>>>> used
>>>>>>>> thrift on 0.89.20100621?
>>>>>>>>
>>>>>>>> -Todd
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> Jimmy.
>>>>>>>>>
>>>>>>>>> --------------------------------------------------
>>>>>>>>> From: "Todd Lipcon" <to...@cloudera.com>
>>>>>>>>> Sent: Thursday, September 02, 2010 5:18 PM
>>>>>>>>>
>>>>>>>>> To: <us...@hbase.apache.org>
>>>>>>>>> Subject: Re: thrift for hbase in CDH3 broken ?
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>  Hi Jinsong,
>>>>>>>>>>
>>>>>>>>>> Are you sure that the port you're connecting to is indeed the 
>>>>>>>>>> thrift
>>>>>>>>>> server?
>>>>>>>>>>
>>>>>>>>>> Unfortunately both the HBase thrift server and the Hue namenode
>>>>>>>>>> plugin
>>>>>>>>>> listen on port 9090, so you might be having an issue where 
>>>>>>>>>> your HBase
>>>>>>>>>> client
>>>>>>>>>> is trying to connect to the Namenode server instead of HBase.
>>>>>>>>>>
>>>>>>>>>> You can verify the ports using a command like "/sbin/fuser -n tcp
>>>>>>>>>> 9090"
>>>>>>>>>> to
>>>>>>>>>> see which pid has it open, then cross reference against sudo jps.
>>>>>>>>>>
>>>>>>>>>> Thanks
>>>>>>>>>> -Todd
>>>>>>>>>>
>>>>>>>>>> On Thu, Sep 2, 2010 at 4:40 PM, Jinsong Hu 
>>>>>>>>>> <ji...@hotmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>  Hi, There,
>>>>>>>>>>>
>>>>>>>>>>>  I am trying to test and see if thrift for hbase works. I 
>>>>>>>>>>> followed
>>>>>>>>>>> the
>>>>>>>>>>> example from
>>>>>>>>>>>
>>>>>>>>>>> http://www.workhabit.com/labs/centos-55-and-thriftscribe
>>>>>>>>>>> http://incubator.apache.org/thrift/
>>>>>>>>>>> http://wiki.apache.org/hadoop/Hbase/ThriftApi
>>>>>>>>>>>
>>>>>>>>>>> and wrote test code: I found that client.getTableNames();
>>>>>>>>>>> returns all table names successfully, but the scanner never 
>>>>>>>>>>> returned
>>>>>>>>>>> any record. and it even throws exception:
>>>>>>>>>>>
>>>>>>>>>>> org.apache.thrift.TApplicationException: get failed: unknown 
>>>>>>>>>>> result
>>>>>>>>>>>     at
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> org.apache.hadoop.hbase.thrift.generated.Hbase$Client.recv_get(Hbase.java:785) 
>>>>>>>>>>>
>>>>>>>>>>>     at
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> org.apache.hadoop.hbase.thrift.generated.Hbase$Client.get(Hbase.java:750) 
>>>>>>>>>>>
>>>>>>>>>>>     at
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> org.apache.hadoop.hbase.thrift.HbaseThriftTest.testThriftAPI(HbaseThriftTest.java:73) 
>>>>>>>>>>>
>>>>>>>>>>>     at
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> org.apache.hadoop.hbase.thrift.HbaseThriftTest.main(HbaseThriftTest.java:128) 
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> I checked the syntax and the table and confirmed that the 
>>>>>>>>>>> table does
>>>>>>>>>>> have
>>>>>>>>>>> record and it should
>>>>>>>>>>> return result, but it doesn't. Can anybody tell me what is 
>>>>>>>>>>> wrong ?
>>>>>>>>>>>
>>>>>>>>>>> At this time, I am suspecting the thrift server shipped with 
>>>>>>>>>>> CDH3
>>>>>>>>>>> may
>>>>>>>>>>> have
>>>>>>>>>>> problem. if there anyway
>>>>>>>>>>> I can get this test code working ?
>>>>>>>>>>>
>>>>>>>>>>> I have to use thrift-0.2.0-incubating.tar.gz , as I found 
>>>>>>>>>>> that the
>>>>>>>>>>> code
>>>>>>>>>>> generated from hbase-thrift project
>>>>>>>>>>> doesn't even compile with 0.3.0 and 0.4.0 thrift.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Jimmy.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>  public void testThriftAPI()
>>>>>>>>>>>  {
>>>>>>>>>>>    // Make socket
>>>>>>>>>>>    TSocket transport = new TSocket("10.20.12.13", 9090);
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>    // Wrap in a protocol
>>>>>>>>>>>    TBinaryProtocol protocol = new  TBinaryProtocol(transport);
>>>>>>>>>>>
>>>>>>>>>>>    Hbase.Client client = new Hbase.Client(protocol);
>>>>>>>>>>>
>>>>>>>>>>>    try {
>>>>>>>>>>>        transport.open();
>>>>>>>>>>>        List<byte[]> tableNamesList=client.getTableNames();
>>>>>>>>>>>        for (byte [] name : tableNamesList)
>>>>>>>>>>>        {
>>>>>>>>>>>            System.out.println(new String(name));
>>>>>>>>>>>        }
>>>>>>>>>>>
>>>>>>>>>>>        String tableName="HEARTBEAT_CLUSTER";
>>>>>>>>>>>        String startRow="";
>>>>>>>>>>>        List<byte[]> columns= new ArrayList<byte[]>();
>>>>>>>>>>>        columns.add("fields:time_format".getBytes());
>>>>>>>>>>>        columns.add("fields:customer_id".getBytes());
>>>>>>>>>>>
>>>>>>>>>>>        int aScannerId=0;
>>>>>>>>>>>        try {
>>>>>>>>>>>
>>>>>>>>>>>            TCell cell=client.get(tableName.getBytes(),
>>>>>>>>>>> "2010-08\tproduction-2".getBytes(),
>>>>>>>>>>> "fields:customer_id".getBytes());
>>>>>>>>>>>            if (cell != null)
>>>>>>>>>>>            {
>>>>>>>>>>>               String value= new String( cell.value);
>>>>>>>>>>>               System.out.println(value);
>>>>>>>>>>>            }
>>>>>>>>>>>            aScannerId=client.scannerOpen(tableName.getBytes(),
>>>>>>>>>>> startRow.getBytes(), columns);
>>>>>>>>>>>            TRowResult result=null;
>>>>>>>>>>>            for (int i=0; i< 10; i++)
>>>>>>>>>>>            {
>>>>>>>>>>>                try {
>>>>>>>>>>>                result=client.scannerGet(aScannerId);
>>>>>>>>>>>                } catch (Exception ex1)
>>>>>>>>>>>                {
>>>>>>>>>>>
>>>>>>>>>>>                }
>>>>>>>>>>>                if (result ==null) break;
>>>>>>>>>>>
>>>>>>>>>>>                String 
>>>>>>>>>>> timeFormat=result.getFieldValue(0).toString();
>>>>>>>>>>>                String 
>>>>>>>>>>> customerId=result.getFieldValue(1).toString();
>>>>>>>>>>>                System.out.println(timeFormat + "\t" + 
>>>>>>>>>>> customerId);
>>>>>>>>>>>            }
>>>>>>>>>>>        } catch (Exception ex)
>>>>>>>>>>>        {
>>>>>>>>>>>            ex.printStackTrace();
>>>>>>>>>>>        }
>>>>>>>>>>>        finally {
>>>>>>>>>>>            if (aScannerId >0 ) client.scannerClose(aScannerId);
>>>>>>>>>>>        }
>>>>>>>>>>>    } catch (Exception exp)
>>>>>>>>>>>    {
>>>>>>>>>>>        exp.printStackTrace();
>>>>>>>>>>>    }
>>>>>>>>>>>    finally{
>>>>>>>>>>>        transport.close();
>>>>>>>>>>>    }
>>>>>>>>>>>
>>>>>>>>>>>  }
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> -- 
>>>>>>>>>> Todd Lipcon
>>>>>>>>>> Software Engineer, Cloudera
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> -- 
>>>>>>>> Todd Lipcon
>>>>>>>> Software Engineer, Cloudera
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> -- 
>>>>>>> Alexey Kovyrin
>>>>>>> http://kovyrin.net/
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> -- 
>>>>> Alexey Kovyrin
>>>>> http://kovyrin.net/
>>>>>
>>>>
>>>
>>>
>>>
>>> -- 
>>> Alexey Kovyrin
>>> http://kovyrin.net/
>>>
>>
>

Re: thrift for hbase in CDH3 broken ?

Posted by Jinsong Hu <ji...@hotmail.com>.
by the way, does anybody have a perl version of the test code ?

Jimmy

--------------------------------------------------
From: "Jinsong Hu" <ji...@hotmail.com>
Sent: Friday, September 03, 2010 11:17 AM
To: <us...@hbase.apache.org>
Subject: Re: thrift for hbase in CDH3 broken ?

>
> I tried your code and indeed it works. but the java version doesn't work. 
> so it looks like it is a bug
> of the java library supplied by the thrift-0.2.0 version.
>
> Jimmy.
> --------------------------------------------------
> From: "Alexey Kovyrin" <al...@kovyrin.net>
> Sent: Friday, September 03, 2010 12:31 AM
> To: <us...@hbase.apache.org>
> Subject: Re: thrift for hbase in CDH3 broken ?
>
>> yes, Centos 5.5 + CDH3b2
>>
>> On Fri, Sep 3, 2010 at 3:26 AM, Jinsong Hu <ji...@hotmail.com> 
>> wrote:
>>> are you using CDH3 distribution ?
>>>
>>> Jinsong
>>>
>>>
>>> --------------------------------------------------
>>> From: "Alexey Kovyrin" <al...@kovyrin.net>
>>> Sent: Friday, September 03, 2010 12:04 AM
>>> To: <us...@hbase.apache.org>
>>> Subject: Re: thrift for hbase in CDH3 broken ?
>>>
>>>> http://github.com/kovyrin/hbase-thrift-client-examples - just wrote
>>>> this example and tested it in our cluster, works as expected.
>>>> For this to work you'd need to install rubygems and thrift gem (gem
>>>> install thrift).
>>>>
>>>> On Fri, Sep 3, 2010 at 12:01 AM, Jinsong Hu <ji...@hotmail.com>
>>>> wrote:
>>>>>
>>>>> Can you send me some ruby test code and so I can try against the 
>>>>> latest
>>>>> CDH3
>>>>> ?
>>>>>
>>>>> Jimmy.
>>>>>
>>>>> --------------------------------------------------
>>>>> From: "Alexey Kovyrin" <al...@kovyrin.net>
>>>>> Sent: Thursday, September 02, 2010 8:15 PM
>>>>> To: <us...@hbase.apache.org>
>>>>> Subject: Re: thrift for hbase in CDH3 broken ?
>>>>>
>>>>>> We use it in Scribd.com. All clients are ruby web apps.
>>>>>>
>>>>>> On Thu, Sep 2, 2010 at 10:49 PM, Todd Lipcon <to...@cloudera.com> 
>>>>>> wrote:
>>>>>>>
>>>>>>> On Thu, Sep 2, 2010 at 5:35 PM, Jinsong Hu <ji...@hotmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Yes, I confirmed that it is indeed thrift server.
>>>>>>>>
>>>>>>>> and the fact that the API
>>>>>>>>
>>>>>>>>         List<byte[]> tableNamesList=client.getTableNames();
>>>>>>>>>>
>>>>>>>>>>        for (byte [] name : tableNamesList)
>>>>>>>>>>        {
>>>>>>>>>>            System.out.println(new String(name));
>>>>>>>>>>        }
>>>>>>>>>>
>>>>>>>>>
>>>>>>>> successfully printed all table names shows that it is indeed thrift
>>>>>>>> server.
>>>>>>>>
>>>>>>>> if it is hue, it won't print the table names.
>>>>>>>>
>>>>>>>> Ah, sorry, I missed that in your original message. Not sure what's 
>>>>>>>> up,
>>>>>>>> then
>>>>>>>
>>>>>>> - we don't have any changes in CDH that would affect this. Anyone 
>>>>>>> here
>>>>>>> used
>>>>>>> thrift on 0.89.20100621?
>>>>>>>
>>>>>>> -Todd
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> Jimmy.
>>>>>>>>
>>>>>>>> --------------------------------------------------
>>>>>>>> From: "Todd Lipcon" <to...@cloudera.com>
>>>>>>>> Sent: Thursday, September 02, 2010 5:18 PM
>>>>>>>>
>>>>>>>> To: <us...@hbase.apache.org>
>>>>>>>> Subject: Re: thrift for hbase in CDH3 broken ?
>>>>>>>>
>>>>>>>>
>>>>>>>>  Hi Jinsong,
>>>>>>>>>
>>>>>>>>> Are you sure that the port you're connecting to is indeed the 
>>>>>>>>> thrift
>>>>>>>>> server?
>>>>>>>>>
>>>>>>>>> Unfortunately both the HBase thrift server and the Hue namenode
>>>>>>>>> plugin
>>>>>>>>> listen on port 9090, so you might be having an issue where your 
>>>>>>>>> HBase
>>>>>>>>> client
>>>>>>>>> is trying to connect to the Namenode server instead of HBase.
>>>>>>>>>
>>>>>>>>> You can verify the ports using a command like "/sbin/fuser -n tcp
>>>>>>>>> 9090"
>>>>>>>>> to
>>>>>>>>> see which pid has it open, then cross reference against sudo jps.
>>>>>>>>>
>>>>>>>>> Thanks
>>>>>>>>> -Todd
>>>>>>>>>
>>>>>>>>> On Thu, Sep 2, 2010 at 4:40 PM, Jinsong Hu 
>>>>>>>>> <ji...@hotmail.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>  Hi, There,
>>>>>>>>>>
>>>>>>>>>>  I am trying to test and see if thrift for hbase works. I 
>>>>>>>>>> followed
>>>>>>>>>> the
>>>>>>>>>> example from
>>>>>>>>>>
>>>>>>>>>> http://www.workhabit.com/labs/centos-55-and-thriftscribe
>>>>>>>>>> http://incubator.apache.org/thrift/
>>>>>>>>>> http://wiki.apache.org/hadoop/Hbase/ThriftApi
>>>>>>>>>>
>>>>>>>>>> and wrote test code: I found that client.getTableNames();
>>>>>>>>>> returns all table names successfully, but the scanner never 
>>>>>>>>>> returned
>>>>>>>>>> any record. and it even throws exception:
>>>>>>>>>>
>>>>>>>>>> org.apache.thrift.TApplicationException: get failed: unknown 
>>>>>>>>>> result
>>>>>>>>>>     at
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> org.apache.hadoop.hbase.thrift.generated.Hbase$Client.recv_get(Hbase.java:785)
>>>>>>>>>>     at
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> org.apache.hadoop.hbase.thrift.generated.Hbase$Client.get(Hbase.java:750)
>>>>>>>>>>     at
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> org.apache.hadoop.hbase.thrift.HbaseThriftTest.testThriftAPI(HbaseThriftTest.java:73)
>>>>>>>>>>     at
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> org.apache.hadoop.hbase.thrift.HbaseThriftTest.main(HbaseThriftTest.java:128)
>>>>>>>>>>
>>>>>>>>>> I checked the syntax and the table and confirmed that the table 
>>>>>>>>>> does
>>>>>>>>>> have
>>>>>>>>>> record and it should
>>>>>>>>>> return result, but it doesn't. Can anybody tell me what is wrong 
>>>>>>>>>> ?
>>>>>>>>>>
>>>>>>>>>> At this time, I am suspecting the thrift server shipped with CDH3
>>>>>>>>>> may
>>>>>>>>>> have
>>>>>>>>>> problem. if there anyway
>>>>>>>>>> I can get this test code working ?
>>>>>>>>>>
>>>>>>>>>> I have to use thrift-0.2.0-incubating.tar.gz , as I found that 
>>>>>>>>>> the
>>>>>>>>>> code
>>>>>>>>>> generated from hbase-thrift project
>>>>>>>>>> doesn't even compile with 0.3.0 and 0.4.0 thrift.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Jimmy.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>  public void testThriftAPI()
>>>>>>>>>>  {
>>>>>>>>>>    // Make socket
>>>>>>>>>>    TSocket transport = new TSocket("10.20.12.13", 9090);
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>    // Wrap in a protocol
>>>>>>>>>>    TBinaryProtocol protocol = new  TBinaryProtocol(transport);
>>>>>>>>>>
>>>>>>>>>>    Hbase.Client client = new Hbase.Client(protocol);
>>>>>>>>>>
>>>>>>>>>>    try {
>>>>>>>>>>        transport.open();
>>>>>>>>>>        List<byte[]> tableNamesList=client.getTableNames();
>>>>>>>>>>        for (byte [] name : tableNamesList)
>>>>>>>>>>        {
>>>>>>>>>>            System.out.println(new String(name));
>>>>>>>>>>        }
>>>>>>>>>>
>>>>>>>>>>        String tableName="HEARTBEAT_CLUSTER";
>>>>>>>>>>        String startRow="";
>>>>>>>>>>        List<byte[]> columns= new ArrayList<byte[]>();
>>>>>>>>>>        columns.add("fields:time_format".getBytes());
>>>>>>>>>>        columns.add("fields:customer_id".getBytes());
>>>>>>>>>>
>>>>>>>>>>        int aScannerId=0;
>>>>>>>>>>        try {
>>>>>>>>>>
>>>>>>>>>>            TCell cell=client.get(tableName.getBytes(),
>>>>>>>>>> "2010-08\tproduction-2".getBytes(),
>>>>>>>>>> "fields:customer_id".getBytes());
>>>>>>>>>>            if (cell != null)
>>>>>>>>>>            {
>>>>>>>>>>               String value= new String( cell.value);
>>>>>>>>>>               System.out.println(value);
>>>>>>>>>>            }
>>>>>>>>>>            aScannerId=client.scannerOpen(tableName.getBytes(),
>>>>>>>>>> startRow.getBytes(), columns);
>>>>>>>>>>            TRowResult result=null;
>>>>>>>>>>            for (int i=0; i< 10; i++)
>>>>>>>>>>            {
>>>>>>>>>>                try {
>>>>>>>>>>                result=client.scannerGet(aScannerId);
>>>>>>>>>>                } catch (Exception ex1)
>>>>>>>>>>                {
>>>>>>>>>>
>>>>>>>>>>                }
>>>>>>>>>>                if (result ==null) break;
>>>>>>>>>>
>>>>>>>>>>                String 
>>>>>>>>>> timeFormat=result.getFieldValue(0).toString();
>>>>>>>>>>                String 
>>>>>>>>>> customerId=result.getFieldValue(1).toString();
>>>>>>>>>>                System.out.println(timeFormat + "\t" + 
>>>>>>>>>> customerId);
>>>>>>>>>>            }
>>>>>>>>>>        } catch (Exception ex)
>>>>>>>>>>        {
>>>>>>>>>>            ex.printStackTrace();
>>>>>>>>>>        }
>>>>>>>>>>        finally {
>>>>>>>>>>            if (aScannerId >0 ) client.scannerClose(aScannerId);
>>>>>>>>>>        }
>>>>>>>>>>    } catch (Exception exp)
>>>>>>>>>>    {
>>>>>>>>>>        exp.printStackTrace();
>>>>>>>>>>    }
>>>>>>>>>>    finally{
>>>>>>>>>>        transport.close();
>>>>>>>>>>    }
>>>>>>>>>>
>>>>>>>>>>  }
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Todd Lipcon
>>>>>>>>> Software Engineer, Cloudera
>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Todd Lipcon
>>>>>>> Software Engineer, Cloudera
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Alexey Kovyrin
>>>>>> http://kovyrin.net/
>>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Alexey Kovyrin
>>>> http://kovyrin.net/
>>>>
>>>
>>
>>
>>
>> -- 
>> Alexey Kovyrin
>> http://kovyrin.net/
>>
>

Re: thrift for hbase in CDH3 broken ?

Posted by Jinsong Hu <ji...@hotmail.com>.
I tried your code and indeed it works. but the java version doesn't work. so 
it looks like it is a bug
of the java library supplied by the thrift-0.2.0 version.

Jimmy.
--------------------------------------------------
From: "Alexey Kovyrin" <al...@kovyrin.net>
Sent: Friday, September 03, 2010 12:31 AM
To: <us...@hbase.apache.org>
Subject: Re: thrift for hbase in CDH3 broken ?

> yes, Centos 5.5 + CDH3b2
>
> On Fri, Sep 3, 2010 at 3:26 AM, Jinsong Hu <ji...@hotmail.com> wrote:
>> are you using CDH3 distribution ?
>>
>> Jinsong
>>
>>
>> --------------------------------------------------
>> From: "Alexey Kovyrin" <al...@kovyrin.net>
>> Sent: Friday, September 03, 2010 12:04 AM
>> To: <us...@hbase.apache.org>
>> Subject: Re: thrift for hbase in CDH3 broken ?
>>
>>> http://github.com/kovyrin/hbase-thrift-client-examples - just wrote
>>> this example and tested it in our cluster, works as expected.
>>> For this to work you'd need to install rubygems and thrift gem (gem
>>> install thrift).
>>>
>>> On Fri, Sep 3, 2010 at 12:01 AM, Jinsong Hu <ji...@hotmail.com>
>>> wrote:
>>>>
>>>> Can you send me some ruby test code and so I can try against the latest
>>>> CDH3
>>>> ?
>>>>
>>>> Jimmy.
>>>>
>>>> --------------------------------------------------
>>>> From: "Alexey Kovyrin" <al...@kovyrin.net>
>>>> Sent: Thursday, September 02, 2010 8:15 PM
>>>> To: <us...@hbase.apache.org>
>>>> Subject: Re: thrift for hbase in CDH3 broken ?
>>>>
>>>>> We use it in Scribd.com. All clients are ruby web apps.
>>>>>
>>>>> On Thu, Sep 2, 2010 at 10:49 PM, Todd Lipcon <to...@cloudera.com> 
>>>>> wrote:
>>>>>>
>>>>>> On Thu, Sep 2, 2010 at 5:35 PM, Jinsong Hu <ji...@hotmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Yes, I confirmed that it is indeed thrift server.
>>>>>>>
>>>>>>> and the fact that the API
>>>>>>>
>>>>>>>         List<byte[]> tableNamesList=client.getTableNames();
>>>>>>>>>
>>>>>>>>>        for (byte [] name : tableNamesList)
>>>>>>>>>        {
>>>>>>>>>            System.out.println(new String(name));
>>>>>>>>>        }
>>>>>>>>>
>>>>>>>>
>>>>>>> successfully printed all table names shows that it is indeed thrift
>>>>>>> server.
>>>>>>>
>>>>>>> if it is hue, it won't print the table names.
>>>>>>>
>>>>>>> Ah, sorry, I missed that in your original message. Not sure what's 
>>>>>>> up,
>>>>>>> then
>>>>>>
>>>>>> - we don't have any changes in CDH that would affect this. Anyone 
>>>>>> here
>>>>>> used
>>>>>> thrift on 0.89.20100621?
>>>>>>
>>>>>> -Todd
>>>>>>
>>>>>>
>>>>>>
>>>>>>> Jimmy.
>>>>>>>
>>>>>>> --------------------------------------------------
>>>>>>> From: "Todd Lipcon" <to...@cloudera.com>
>>>>>>> Sent: Thursday, September 02, 2010 5:18 PM
>>>>>>>
>>>>>>> To: <us...@hbase.apache.org>
>>>>>>> Subject: Re: thrift for hbase in CDH3 broken ?
>>>>>>>
>>>>>>>
>>>>>>>  Hi Jinsong,
>>>>>>>>
>>>>>>>> Are you sure that the port you're connecting to is indeed the 
>>>>>>>> thrift
>>>>>>>> server?
>>>>>>>>
>>>>>>>> Unfortunately both the HBase thrift server and the Hue namenode
>>>>>>>> plugin
>>>>>>>> listen on port 9090, so you might be having an issue where your 
>>>>>>>> HBase
>>>>>>>> client
>>>>>>>> is trying to connect to the Namenode server instead of HBase.
>>>>>>>>
>>>>>>>> You can verify the ports using a command like "/sbin/fuser -n tcp
>>>>>>>> 9090"
>>>>>>>> to
>>>>>>>> see which pid has it open, then cross reference against sudo jps.
>>>>>>>>
>>>>>>>> Thanks
>>>>>>>> -Todd
>>>>>>>>
>>>>>>>> On Thu, Sep 2, 2010 at 4:40 PM, Jinsong Hu <ji...@hotmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>  Hi, There,
>>>>>>>>>
>>>>>>>>>  I am trying to test and see if thrift for hbase works. I followed
>>>>>>>>> the
>>>>>>>>> example from
>>>>>>>>>
>>>>>>>>> http://www.workhabit.com/labs/centos-55-and-thriftscribe
>>>>>>>>> http://incubator.apache.org/thrift/
>>>>>>>>> http://wiki.apache.org/hadoop/Hbase/ThriftApi
>>>>>>>>>
>>>>>>>>> and wrote test code: I found that client.getTableNames();
>>>>>>>>> returns all table names successfully, but the scanner never 
>>>>>>>>> returned
>>>>>>>>> any record. and it even throws exception:
>>>>>>>>>
>>>>>>>>> org.apache.thrift.TApplicationException: get failed: unknown 
>>>>>>>>> result
>>>>>>>>>     at
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> org.apache.hadoop.hbase.thrift.generated.Hbase$Client.recv_get(Hbase.java:785)
>>>>>>>>>     at
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> org.apache.hadoop.hbase.thrift.generated.Hbase$Client.get(Hbase.java:750)
>>>>>>>>>     at
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> org.apache.hadoop.hbase.thrift.HbaseThriftTest.testThriftAPI(HbaseThriftTest.java:73)
>>>>>>>>>     at
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> org.apache.hadoop.hbase.thrift.HbaseThriftTest.main(HbaseThriftTest.java:128)
>>>>>>>>>
>>>>>>>>> I checked the syntax and the table and confirmed that the table 
>>>>>>>>> does
>>>>>>>>> have
>>>>>>>>> record and it should
>>>>>>>>> return result, but it doesn't. Can anybody tell me what is wrong ?
>>>>>>>>>
>>>>>>>>> At this time, I am suspecting the thrift server shipped with CDH3
>>>>>>>>> may
>>>>>>>>> have
>>>>>>>>> problem. if there anyway
>>>>>>>>> I can get this test code working ?
>>>>>>>>>
>>>>>>>>> I have to use thrift-0.2.0-incubating.tar.gz , as I found that the
>>>>>>>>> code
>>>>>>>>> generated from hbase-thrift project
>>>>>>>>> doesn't even compile with 0.3.0 and 0.4.0 thrift.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Jimmy.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>  public void testThriftAPI()
>>>>>>>>>  {
>>>>>>>>>    // Make socket
>>>>>>>>>    TSocket transport = new TSocket("10.20.12.13", 9090);
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>    // Wrap in a protocol
>>>>>>>>>    TBinaryProtocol protocol = new  TBinaryProtocol(transport);
>>>>>>>>>
>>>>>>>>>    Hbase.Client client = new Hbase.Client(protocol);
>>>>>>>>>
>>>>>>>>>    try {
>>>>>>>>>        transport.open();
>>>>>>>>>        List<byte[]> tableNamesList=client.getTableNames();
>>>>>>>>>        for (byte [] name : tableNamesList)
>>>>>>>>>        {
>>>>>>>>>            System.out.println(new String(name));
>>>>>>>>>        }
>>>>>>>>>
>>>>>>>>>        String tableName="HEARTBEAT_CLUSTER";
>>>>>>>>>        String startRow="";
>>>>>>>>>        List<byte[]> columns= new ArrayList<byte[]>();
>>>>>>>>>        columns.add("fields:time_format".getBytes());
>>>>>>>>>        columns.add("fields:customer_id".getBytes());
>>>>>>>>>
>>>>>>>>>        int aScannerId=0;
>>>>>>>>>        try {
>>>>>>>>>
>>>>>>>>>            TCell cell=client.get(tableName.getBytes(),
>>>>>>>>> "2010-08\tproduction-2".getBytes(),
>>>>>>>>> "fields:customer_id".getBytes());
>>>>>>>>>            if (cell != null)
>>>>>>>>>            {
>>>>>>>>>               String value= new String( cell.value);
>>>>>>>>>               System.out.println(value);
>>>>>>>>>            }
>>>>>>>>>            aScannerId=client.scannerOpen(tableName.getBytes(),
>>>>>>>>> startRow.getBytes(), columns);
>>>>>>>>>            TRowResult result=null;
>>>>>>>>>            for (int i=0; i< 10; i++)
>>>>>>>>>            {
>>>>>>>>>                try {
>>>>>>>>>                result=client.scannerGet(aScannerId);
>>>>>>>>>                } catch (Exception ex1)
>>>>>>>>>                {
>>>>>>>>>
>>>>>>>>>                }
>>>>>>>>>                if (result ==null) break;
>>>>>>>>>
>>>>>>>>>                String 
>>>>>>>>> timeFormat=result.getFieldValue(0).toString();
>>>>>>>>>                String 
>>>>>>>>> customerId=result.getFieldValue(1).toString();
>>>>>>>>>                System.out.println(timeFormat + "\t" + customerId);
>>>>>>>>>            }
>>>>>>>>>        } catch (Exception ex)
>>>>>>>>>        {
>>>>>>>>>            ex.printStackTrace();
>>>>>>>>>        }
>>>>>>>>>        finally {
>>>>>>>>>            if (aScannerId >0 ) client.scannerClose(aScannerId);
>>>>>>>>>        }
>>>>>>>>>    } catch (Exception exp)
>>>>>>>>>    {
>>>>>>>>>        exp.printStackTrace();
>>>>>>>>>    }
>>>>>>>>>    finally{
>>>>>>>>>        transport.close();
>>>>>>>>>    }
>>>>>>>>>
>>>>>>>>>  }
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Todd Lipcon
>>>>>>>> Software Engineer, Cloudera
>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Todd Lipcon
>>>>>> Software Engineer, Cloudera
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Alexey Kovyrin
>>>>> http://kovyrin.net/
>>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Alexey Kovyrin
>>> http://kovyrin.net/
>>>
>>
>
>
>
> -- 
> Alexey Kovyrin
> http://kovyrin.net/
>

Re: thrift for hbase in CDH3 broken ?

Posted by Alexey Kovyrin <al...@kovyrin.net>.
yes, Centos 5.5 + CDH3b2

On Fri, Sep 3, 2010 at 3:26 AM, Jinsong Hu <ji...@hotmail.com> wrote:
> are you using CDH3 distribution ?
>
> Jinsong
>
>
> --------------------------------------------------
> From: "Alexey Kovyrin" <al...@kovyrin.net>
> Sent: Friday, September 03, 2010 12:04 AM
> To: <us...@hbase.apache.org>
> Subject: Re: thrift for hbase in CDH3 broken ?
>
>> http://github.com/kovyrin/hbase-thrift-client-examples - just wrote
>> this example and tested it in our cluster, works as expected.
>> For this to work you'd need to install rubygems and thrift gem (gem
>> install thrift).
>>
>> On Fri, Sep 3, 2010 at 12:01 AM, Jinsong Hu <ji...@hotmail.com>
>> wrote:
>>>
>>> Can you send me some ruby test code and so I can try against the latest
>>> CDH3
>>> ?
>>>
>>> Jimmy.
>>>
>>> --------------------------------------------------
>>> From: "Alexey Kovyrin" <al...@kovyrin.net>
>>> Sent: Thursday, September 02, 2010 8:15 PM
>>> To: <us...@hbase.apache.org>
>>> Subject: Re: thrift for hbase in CDH3 broken ?
>>>
>>>> We use it in Scribd.com. All clients are ruby web apps.
>>>>
>>>> On Thu, Sep 2, 2010 at 10:49 PM, Todd Lipcon <to...@cloudera.com> wrote:
>>>>>
>>>>> On Thu, Sep 2, 2010 at 5:35 PM, Jinsong Hu <ji...@hotmail.com>
>>>>> wrote:
>>>>>
>>>>>> Yes, I confirmed that it is indeed thrift server.
>>>>>>
>>>>>> and the fact that the API
>>>>>>
>>>>>>         List<byte[]> tableNamesList=client.getTableNames();
>>>>>>>>
>>>>>>>>        for (byte [] name : tableNamesList)
>>>>>>>>        {
>>>>>>>>            System.out.println(new String(name));
>>>>>>>>        }
>>>>>>>>
>>>>>>>
>>>>>> successfully printed all table names shows that it is indeed thrift
>>>>>> server.
>>>>>>
>>>>>> if it is hue, it won't print the table names.
>>>>>>
>>>>>> Ah, sorry, I missed that in your original message. Not sure what's up,
>>>>>> then
>>>>>
>>>>> - we don't have any changes in CDH that would affect this. Anyone here
>>>>> used
>>>>> thrift on 0.89.20100621?
>>>>>
>>>>> -Todd
>>>>>
>>>>>
>>>>>
>>>>>> Jimmy.
>>>>>>
>>>>>> --------------------------------------------------
>>>>>> From: "Todd Lipcon" <to...@cloudera.com>
>>>>>> Sent: Thursday, September 02, 2010 5:18 PM
>>>>>>
>>>>>> To: <us...@hbase.apache.org>
>>>>>> Subject: Re: thrift for hbase in CDH3 broken ?
>>>>>>
>>>>>>
>>>>>>  Hi Jinsong,
>>>>>>>
>>>>>>> Are you sure that the port you're connecting to is indeed the thrift
>>>>>>> server?
>>>>>>>
>>>>>>> Unfortunately both the HBase thrift server and the Hue namenode
>>>>>>> plugin
>>>>>>> listen on port 9090, so you might be having an issue where your HBase
>>>>>>> client
>>>>>>> is trying to connect to the Namenode server instead of HBase.
>>>>>>>
>>>>>>> You can verify the ports using a command like "/sbin/fuser -n tcp
>>>>>>> 9090"
>>>>>>> to
>>>>>>> see which pid has it open, then cross reference against sudo jps.
>>>>>>>
>>>>>>> Thanks
>>>>>>> -Todd
>>>>>>>
>>>>>>> On Thu, Sep 2, 2010 at 4:40 PM, Jinsong Hu <ji...@hotmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>  Hi, There,
>>>>>>>>
>>>>>>>>  I am trying to test and see if thrift for hbase works. I followed
>>>>>>>> the
>>>>>>>> example from
>>>>>>>>
>>>>>>>> http://www.workhabit.com/labs/centos-55-and-thriftscribe
>>>>>>>> http://incubator.apache.org/thrift/
>>>>>>>> http://wiki.apache.org/hadoop/Hbase/ThriftApi
>>>>>>>>
>>>>>>>> and wrote test code: I found that client.getTableNames();
>>>>>>>> returns all table names successfully, but the scanner never returned
>>>>>>>> any record. and it even throws exception:
>>>>>>>>
>>>>>>>> org.apache.thrift.TApplicationException: get failed: unknown result
>>>>>>>>     at
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> org.apache.hadoop.hbase.thrift.generated.Hbase$Client.recv_get(Hbase.java:785)
>>>>>>>>     at
>>>>>>>>
>>>>>>>>
>>>>>>>> org.apache.hadoop.hbase.thrift.generated.Hbase$Client.get(Hbase.java:750)
>>>>>>>>     at
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> org.apache.hadoop.hbase.thrift.HbaseThriftTest.testThriftAPI(HbaseThriftTest.java:73)
>>>>>>>>     at
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> org.apache.hadoop.hbase.thrift.HbaseThriftTest.main(HbaseThriftTest.java:128)
>>>>>>>>
>>>>>>>> I checked the syntax and the table and confirmed that the table does
>>>>>>>> have
>>>>>>>> record and it should
>>>>>>>> return result, but it doesn't. Can anybody tell me what is wrong ?
>>>>>>>>
>>>>>>>> At this time, I am suspecting the thrift server shipped with CDH3
>>>>>>>> may
>>>>>>>> have
>>>>>>>> problem. if there anyway
>>>>>>>> I can get this test code working ?
>>>>>>>>
>>>>>>>> I have to use thrift-0.2.0-incubating.tar.gz , as I found that the
>>>>>>>> code
>>>>>>>> generated from hbase-thrift project
>>>>>>>> doesn't even compile with 0.3.0 and 0.4.0 thrift.
>>>>>>>>
>>>>>>>>
>>>>>>>> Jimmy.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>  public void testThriftAPI()
>>>>>>>>  {
>>>>>>>>    // Make socket
>>>>>>>>    TSocket transport = new TSocket("10.20.12.13", 9090);
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>    // Wrap in a protocol
>>>>>>>>    TBinaryProtocol protocol = new  TBinaryProtocol(transport);
>>>>>>>>
>>>>>>>>    Hbase.Client client = new Hbase.Client(protocol);
>>>>>>>>
>>>>>>>>    try {
>>>>>>>>        transport.open();
>>>>>>>>        List<byte[]> tableNamesList=client.getTableNames();
>>>>>>>>        for (byte [] name : tableNamesList)
>>>>>>>>        {
>>>>>>>>            System.out.println(new String(name));
>>>>>>>>        }
>>>>>>>>
>>>>>>>>        String tableName="HEARTBEAT_CLUSTER";
>>>>>>>>        String startRow="";
>>>>>>>>        List<byte[]> columns= new ArrayList<byte[]>();
>>>>>>>>        columns.add("fields:time_format".getBytes());
>>>>>>>>        columns.add("fields:customer_id".getBytes());
>>>>>>>>
>>>>>>>>        int aScannerId=0;
>>>>>>>>        try {
>>>>>>>>
>>>>>>>>            TCell cell=client.get(tableName.getBytes(),
>>>>>>>> "2010-08\tproduction-2".getBytes(),
>>>>>>>> "fields:customer_id".getBytes());
>>>>>>>>            if (cell != null)
>>>>>>>>            {
>>>>>>>>               String value= new String( cell.value);
>>>>>>>>               System.out.println(value);
>>>>>>>>            }
>>>>>>>>            aScannerId=client.scannerOpen(tableName.getBytes(),
>>>>>>>> startRow.getBytes(), columns);
>>>>>>>>            TRowResult result=null;
>>>>>>>>            for (int i=0; i< 10; i++)
>>>>>>>>            {
>>>>>>>>                try {
>>>>>>>>                result=client.scannerGet(aScannerId);
>>>>>>>>                } catch (Exception ex1)
>>>>>>>>                {
>>>>>>>>
>>>>>>>>                }
>>>>>>>>                if (result ==null) break;
>>>>>>>>
>>>>>>>>                String timeFormat=result.getFieldValue(0).toString();
>>>>>>>>                String customerId=result.getFieldValue(1).toString();
>>>>>>>>                System.out.println(timeFormat + "\t" + customerId);
>>>>>>>>            }
>>>>>>>>        } catch (Exception ex)
>>>>>>>>        {
>>>>>>>>            ex.printStackTrace();
>>>>>>>>        }
>>>>>>>>        finally {
>>>>>>>>            if (aScannerId >0 ) client.scannerClose(aScannerId);
>>>>>>>>        }
>>>>>>>>    } catch (Exception exp)
>>>>>>>>    {
>>>>>>>>        exp.printStackTrace();
>>>>>>>>    }
>>>>>>>>    finally{
>>>>>>>>        transport.close();
>>>>>>>>    }
>>>>>>>>
>>>>>>>>  }
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Todd Lipcon
>>>>>>> Software Engineer, Cloudera
>>>>>>>
>>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Todd Lipcon
>>>>> Software Engineer, Cloudera
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Alexey Kovyrin
>>>> http://kovyrin.net/
>>>>
>>>
>>
>>
>>
>> --
>> Alexey Kovyrin
>> http://kovyrin.net/
>>
>



-- 
Alexey Kovyrin
http://kovyrin.net/

Re: thrift for hbase in CDH3 broken ?

Posted by Jinsong Hu <ji...@hotmail.com>.
are you using CDH3 distribution ?

Jinsong


--------------------------------------------------
From: "Alexey Kovyrin" <al...@kovyrin.net>
Sent: Friday, September 03, 2010 12:04 AM
To: <us...@hbase.apache.org>
Subject: Re: thrift for hbase in CDH3 broken ?

> http://github.com/kovyrin/hbase-thrift-client-examples - just wrote
> this example and tested it in our cluster, works as expected.
> For this to work you'd need to install rubygems and thrift gem (gem
> install thrift).
>
> On Fri, Sep 3, 2010 at 12:01 AM, Jinsong Hu <ji...@hotmail.com> 
> wrote:
>> Can you send me some ruby test code and so I can try against the latest 
>> CDH3
>> ?
>>
>> Jimmy.
>>
>> --------------------------------------------------
>> From: "Alexey Kovyrin" <al...@kovyrin.net>
>> Sent: Thursday, September 02, 2010 8:15 PM
>> To: <us...@hbase.apache.org>
>> Subject: Re: thrift for hbase in CDH3 broken ?
>>
>>> We use it in Scribd.com. All clients are ruby web apps.
>>>
>>> On Thu, Sep 2, 2010 at 10:49 PM, Todd Lipcon <to...@cloudera.com> wrote:
>>>>
>>>> On Thu, Sep 2, 2010 at 5:35 PM, Jinsong Hu <ji...@hotmail.com>
>>>> wrote:
>>>>
>>>>> Yes, I confirmed that it is indeed thrift server.
>>>>>
>>>>> and the fact that the API
>>>>>
>>>>>          List<byte[]> tableNamesList=client.getTableNames();
>>>>>>>
>>>>>>>         for (byte [] name : tableNamesList)
>>>>>>>         {
>>>>>>>             System.out.println(new String(name));
>>>>>>>         }
>>>>>>>
>>>>>>
>>>>> successfully printed all table names shows that it is indeed thrift
>>>>> server.
>>>>>
>>>>> if it is hue, it won't print the table names.
>>>>>
>>>>> Ah, sorry, I missed that in your original message. Not sure what's up,
>>>>> then
>>>>
>>>> - we don't have any changes in CDH that would affect this. Anyone here
>>>> used
>>>> thrift on 0.89.20100621?
>>>>
>>>> -Todd
>>>>
>>>>
>>>>
>>>>> Jimmy.
>>>>>
>>>>> --------------------------------------------------
>>>>> From: "Todd Lipcon" <to...@cloudera.com>
>>>>> Sent: Thursday, September 02, 2010 5:18 PM
>>>>>
>>>>> To: <us...@hbase.apache.org>
>>>>> Subject: Re: thrift for hbase in CDH3 broken ?
>>>>>
>>>>>
>>>>>  Hi Jinsong,
>>>>>>
>>>>>> Are you sure that the port you're connecting to is indeed the thrift
>>>>>> server?
>>>>>>
>>>>>> Unfortunately both the HBase thrift server and the Hue namenode 
>>>>>> plugin
>>>>>> listen on port 9090, so you might be having an issue where your HBase
>>>>>> client
>>>>>> is trying to connect to the Namenode server instead of HBase.
>>>>>>
>>>>>> You can verify the ports using a command like "/sbin/fuser -n tcp 
>>>>>> 9090"
>>>>>> to
>>>>>> see which pid has it open, then cross reference against sudo jps.
>>>>>>
>>>>>> Thanks
>>>>>> -Todd
>>>>>>
>>>>>> On Thu, Sep 2, 2010 at 4:40 PM, Jinsong Hu <ji...@hotmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>  Hi, There,
>>>>>>>
>>>>>>>  I am trying to test and see if thrift for hbase works. I followed 
>>>>>>> the
>>>>>>> example from
>>>>>>>
>>>>>>> http://www.workhabit.com/labs/centos-55-and-thriftscribe
>>>>>>> http://incubator.apache.org/thrift/
>>>>>>> http://wiki.apache.org/hadoop/Hbase/ThriftApi
>>>>>>>
>>>>>>> and wrote test code: I found that client.getTableNames();
>>>>>>> returns all table names successfully, but the scanner never returned
>>>>>>> any record. and it even throws exception:
>>>>>>>
>>>>>>> org.apache.thrift.TApplicationException: get failed: unknown result
>>>>>>>      at
>>>>>>>
>>>>>>>
>>>>>>> org.apache.hadoop.hbase.thrift.generated.Hbase$Client.recv_get(Hbase.java:785)
>>>>>>>      at
>>>>>>>
>>>>>>> org.apache.hadoop.hbase.thrift.generated.Hbase$Client.get(Hbase.java:750)
>>>>>>>      at
>>>>>>>
>>>>>>>
>>>>>>> org.apache.hadoop.hbase.thrift.HbaseThriftTest.testThriftAPI(HbaseThriftTest.java:73)
>>>>>>>      at
>>>>>>>
>>>>>>>
>>>>>>> org.apache.hadoop.hbase.thrift.HbaseThriftTest.main(HbaseThriftTest.java:128)
>>>>>>>
>>>>>>> I checked the syntax and the table and confirmed that the table does
>>>>>>> have
>>>>>>> record and it should
>>>>>>> return result, but it doesn't. Can anybody tell me what is wrong ?
>>>>>>>
>>>>>>> At this time, I am suspecting the thrift server shipped with CDH3 
>>>>>>> may
>>>>>>> have
>>>>>>> problem. if there anyway
>>>>>>> I can get this test code working ?
>>>>>>>
>>>>>>> I have to use thrift-0.2.0-incubating.tar.gz , as I found that the
>>>>>>> code
>>>>>>> generated from hbase-thrift project
>>>>>>> doesn't even compile with 0.3.0 and 0.4.0 thrift.
>>>>>>>
>>>>>>>
>>>>>>> Jimmy.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>  public void testThriftAPI()
>>>>>>>  {
>>>>>>>     // Make socket
>>>>>>>     TSocket transport = new TSocket("10.20.12.13", 9090);
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>     // Wrap in a protocol
>>>>>>>     TBinaryProtocol protocol = new  TBinaryProtocol(transport);
>>>>>>>
>>>>>>>     Hbase.Client client = new Hbase.Client(protocol);
>>>>>>>
>>>>>>>     try {
>>>>>>>         transport.open();
>>>>>>>         List<byte[]> tableNamesList=client.getTableNames();
>>>>>>>         for (byte [] name : tableNamesList)
>>>>>>>         {
>>>>>>>             System.out.println(new String(name));
>>>>>>>         }
>>>>>>>
>>>>>>>         String tableName="HEARTBEAT_CLUSTER";
>>>>>>>         String startRow="";
>>>>>>>         List<byte[]> columns= new ArrayList<byte[]>();
>>>>>>>         columns.add("fields:time_format".getBytes());
>>>>>>>         columns.add("fields:customer_id".getBytes());
>>>>>>>
>>>>>>>         int aScannerId=0;
>>>>>>>         try {
>>>>>>>
>>>>>>>             TCell cell=client.get(tableName.getBytes(),
>>>>>>> "2010-08\tproduction-2".getBytes(), 
>>>>>>> "fields:customer_id".getBytes());
>>>>>>>             if (cell != null)
>>>>>>>             {
>>>>>>>                String value= new String( cell.value);
>>>>>>>                System.out.println(value);
>>>>>>>             }
>>>>>>>             aScannerId=client.scannerOpen(tableName.getBytes(),
>>>>>>> startRow.getBytes(), columns);
>>>>>>>             TRowResult result=null;
>>>>>>>             for (int i=0; i< 10; i++)
>>>>>>>             {
>>>>>>>                 try {
>>>>>>>                 result=client.scannerGet(aScannerId);
>>>>>>>                 } catch (Exception ex1)
>>>>>>>                 {
>>>>>>>
>>>>>>>                 }
>>>>>>>                 if (result ==null) break;
>>>>>>>
>>>>>>>                 String 
>>>>>>> timeFormat=result.getFieldValue(0).toString();
>>>>>>>                 String 
>>>>>>> customerId=result.getFieldValue(1).toString();
>>>>>>>                 System.out.println(timeFormat + "\t" + customerId);
>>>>>>>             }
>>>>>>>         } catch (Exception ex)
>>>>>>>         {
>>>>>>>             ex.printStackTrace();
>>>>>>>         }
>>>>>>>         finally {
>>>>>>>             if (aScannerId >0 ) client.scannerClose(aScannerId);
>>>>>>>         }
>>>>>>>     } catch (Exception exp)
>>>>>>>     {
>>>>>>>         exp.printStackTrace();
>>>>>>>     }
>>>>>>>     finally{
>>>>>>>         transport.close();
>>>>>>>     }
>>>>>>>
>>>>>>>  }
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Todd Lipcon
>>>>>> Software Engineer, Cloudera
>>>>>>
>>>>>>
>>>>
>>>>
>>>> --
>>>> Todd Lipcon
>>>> Software Engineer, Cloudera
>>>>
>>>
>>>
>>>
>>> --
>>> Alexey Kovyrin
>>> http://kovyrin.net/
>>>
>>
>
>
>
> -- 
> Alexey Kovyrin
> http://kovyrin.net/
>

Re: thrift for hbase in CDH3 broken ?

Posted by Alexey Kovyrin <al...@kovyrin.net>.
http://github.com/kovyrin/hbase-thrift-client-examples - just wrote
this example and tested it in our cluster, works as expected.
For this to work you'd need to install rubygems and thrift gem (gem
install thrift).

On Fri, Sep 3, 2010 at 12:01 AM, Jinsong Hu <ji...@hotmail.com> wrote:
> Can you send me some ruby test code and so I can try against the latest CDH3
> ?
>
> Jimmy.
>
> --------------------------------------------------
> From: "Alexey Kovyrin" <al...@kovyrin.net>
> Sent: Thursday, September 02, 2010 8:15 PM
> To: <us...@hbase.apache.org>
> Subject: Re: thrift for hbase in CDH3 broken ?
>
>> We use it in Scribd.com. All clients are ruby web apps.
>>
>> On Thu, Sep 2, 2010 at 10:49 PM, Todd Lipcon <to...@cloudera.com> wrote:
>>>
>>> On Thu, Sep 2, 2010 at 5:35 PM, Jinsong Hu <ji...@hotmail.com>
>>> wrote:
>>>
>>>> Yes, I confirmed that it is indeed thrift server.
>>>>
>>>> and the fact that the API
>>>>
>>>>          List<byte[]> tableNamesList=client.getTableNames();
>>>>>>
>>>>>>         for (byte [] name : tableNamesList)
>>>>>>         {
>>>>>>             System.out.println(new String(name));
>>>>>>         }
>>>>>>
>>>>>
>>>> successfully printed all table names shows that it is indeed thrift
>>>> server.
>>>>
>>>> if it is hue, it won't print the table names.
>>>>
>>>> Ah, sorry, I missed that in your original message. Not sure what's up,
>>>> then
>>>
>>> - we don't have any changes in CDH that would affect this. Anyone here
>>> used
>>> thrift on 0.89.20100621?
>>>
>>> -Todd
>>>
>>>
>>>
>>>> Jimmy.
>>>>
>>>> --------------------------------------------------
>>>> From: "Todd Lipcon" <to...@cloudera.com>
>>>> Sent: Thursday, September 02, 2010 5:18 PM
>>>>
>>>> To: <us...@hbase.apache.org>
>>>> Subject: Re: thrift for hbase in CDH3 broken ?
>>>>
>>>>
>>>>  Hi Jinsong,
>>>>>
>>>>> Are you sure that the port you're connecting to is indeed the thrift
>>>>> server?
>>>>>
>>>>> Unfortunately both the HBase thrift server and the Hue namenode plugin
>>>>> listen on port 9090, so you might be having an issue where your HBase
>>>>> client
>>>>> is trying to connect to the Namenode server instead of HBase.
>>>>>
>>>>> You can verify the ports using a command like "/sbin/fuser -n tcp 9090"
>>>>> to
>>>>> see which pid has it open, then cross reference against sudo jps.
>>>>>
>>>>> Thanks
>>>>> -Todd
>>>>>
>>>>> On Thu, Sep 2, 2010 at 4:40 PM, Jinsong Hu <ji...@hotmail.com>
>>>>> wrote:
>>>>>
>>>>>  Hi, There,
>>>>>>
>>>>>>  I am trying to test and see if thrift for hbase works. I followed the
>>>>>> example from
>>>>>>
>>>>>> http://www.workhabit.com/labs/centos-55-and-thriftscribe
>>>>>> http://incubator.apache.org/thrift/
>>>>>> http://wiki.apache.org/hadoop/Hbase/ThriftApi
>>>>>>
>>>>>> and wrote test code: I found that client.getTableNames();
>>>>>> returns all table names successfully, but the scanner never returned
>>>>>> any record. and it even throws exception:
>>>>>>
>>>>>> org.apache.thrift.TApplicationException: get failed: unknown result
>>>>>>      at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.hbase.thrift.generated.Hbase$Client.recv_get(Hbase.java:785)
>>>>>>      at
>>>>>>
>>>>>> org.apache.hadoop.hbase.thrift.generated.Hbase$Client.get(Hbase.java:750)
>>>>>>      at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.hbase.thrift.HbaseThriftTest.testThriftAPI(HbaseThriftTest.java:73)
>>>>>>      at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.hbase.thrift.HbaseThriftTest.main(HbaseThriftTest.java:128)
>>>>>>
>>>>>> I checked the syntax and the table and confirmed that the table does
>>>>>> have
>>>>>> record and it should
>>>>>> return result, but it doesn't. Can anybody tell me what is wrong ?
>>>>>>
>>>>>> At this time, I am suspecting the thrift server shipped with CDH3 may
>>>>>> have
>>>>>> problem. if there anyway
>>>>>> I can get this test code working ?
>>>>>>
>>>>>> I have to use thrift-0.2.0-incubating.tar.gz , as I found that the
>>>>>> code
>>>>>> generated from hbase-thrift project
>>>>>> doesn't even compile with 0.3.0 and 0.4.0 thrift.
>>>>>>
>>>>>>
>>>>>> Jimmy.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>  public void testThriftAPI()
>>>>>>  {
>>>>>>     // Make socket
>>>>>>     TSocket transport = new TSocket("10.20.12.13", 9090);
>>>>>>
>>>>>>
>>>>>>
>>>>>>     // Wrap in a protocol
>>>>>>     TBinaryProtocol protocol = new  TBinaryProtocol(transport);
>>>>>>
>>>>>>     Hbase.Client client = new Hbase.Client(protocol);
>>>>>>
>>>>>>     try {
>>>>>>         transport.open();
>>>>>>         List<byte[]> tableNamesList=client.getTableNames();
>>>>>>         for (byte [] name : tableNamesList)
>>>>>>         {
>>>>>>             System.out.println(new String(name));
>>>>>>         }
>>>>>>
>>>>>>         String tableName="HEARTBEAT_CLUSTER";
>>>>>>         String startRow="";
>>>>>>         List<byte[]> columns= new ArrayList<byte[]>();
>>>>>>         columns.add("fields:time_format".getBytes());
>>>>>>         columns.add("fields:customer_id".getBytes());
>>>>>>
>>>>>>         int aScannerId=0;
>>>>>>         try {
>>>>>>
>>>>>>             TCell cell=client.get(tableName.getBytes(),
>>>>>> "2010-08\tproduction-2".getBytes(), "fields:customer_id".getBytes());
>>>>>>             if (cell != null)
>>>>>>             {
>>>>>>                String value= new String( cell.value);
>>>>>>                System.out.println(value);
>>>>>>             }
>>>>>>             aScannerId=client.scannerOpen(tableName.getBytes(),
>>>>>> startRow.getBytes(), columns);
>>>>>>             TRowResult result=null;
>>>>>>             for (int i=0; i< 10; i++)
>>>>>>             {
>>>>>>                 try {
>>>>>>                 result=client.scannerGet(aScannerId);
>>>>>>                 } catch (Exception ex1)
>>>>>>                 {
>>>>>>
>>>>>>                 }
>>>>>>                 if (result ==null) break;
>>>>>>
>>>>>>                 String timeFormat=result.getFieldValue(0).toString();
>>>>>>                 String customerId=result.getFieldValue(1).toString();
>>>>>>                 System.out.println(timeFormat + "\t" + customerId);
>>>>>>             }
>>>>>>         } catch (Exception ex)
>>>>>>         {
>>>>>>             ex.printStackTrace();
>>>>>>         }
>>>>>>         finally {
>>>>>>             if (aScannerId >0 ) client.scannerClose(aScannerId);
>>>>>>         }
>>>>>>     } catch (Exception exp)
>>>>>>     {
>>>>>>         exp.printStackTrace();
>>>>>>     }
>>>>>>     finally{
>>>>>>         transport.close();
>>>>>>     }
>>>>>>
>>>>>>  }
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Todd Lipcon
>>>>> Software Engineer, Cloudera
>>>>>
>>>>>
>>>
>>>
>>> --
>>> Todd Lipcon
>>> Software Engineer, Cloudera
>>>
>>
>>
>>
>> --
>> Alexey Kovyrin
>> http://kovyrin.net/
>>
>



-- 
Alexey Kovyrin
http://kovyrin.net/

Re: thrift for hbase in CDH3 broken ?

Posted by Jinsong Hu <ji...@hotmail.com>.
Can you send me some ruby test code and so I can try against the latest CDH3 
?

Jimmy.

--------------------------------------------------
From: "Alexey Kovyrin" <al...@kovyrin.net>
Sent: Thursday, September 02, 2010 8:15 PM
To: <us...@hbase.apache.org>
Subject: Re: thrift for hbase in CDH3 broken ?

> We use it in Scribd.com. All clients are ruby web apps.
>
> On Thu, Sep 2, 2010 at 10:49 PM, Todd Lipcon <to...@cloudera.com> wrote:
>> On Thu, Sep 2, 2010 at 5:35 PM, Jinsong Hu <ji...@hotmail.com> 
>> wrote:
>>
>>> Yes, I confirmed that it is indeed thrift server.
>>>
>>> and the fact that the API
>>>
>>>           List<byte[]> tableNamesList=client.getTableNames();
>>>>>          for (byte [] name : tableNamesList)
>>>>>          {
>>>>>              System.out.println(new String(name));
>>>>>          }
>>>>>
>>>>
>>> successfully printed all table names shows that it is indeed thrift 
>>> server.
>>>
>>> if it is hue, it won't print the table names.
>>>
>>> Ah, sorry, I missed that in your original message. Not sure what's up, 
>>> then
>> - we don't have any changes in CDH that would affect this. Anyone here 
>> used
>> thrift on 0.89.20100621?
>>
>> -Todd
>>
>>
>>
>>> Jimmy.
>>>
>>> --------------------------------------------------
>>> From: "Todd Lipcon" <to...@cloudera.com>
>>> Sent: Thursday, September 02, 2010 5:18 PM
>>>
>>> To: <us...@hbase.apache.org>
>>> Subject: Re: thrift for hbase in CDH3 broken ?
>>>
>>>
>>>  Hi Jinsong,
>>>>
>>>> Are you sure that the port you're connecting to is indeed the thrift
>>>> server?
>>>>
>>>> Unfortunately both the HBase thrift server and the Hue namenode plugin
>>>> listen on port 9090, so you might be having an issue where your HBase
>>>> client
>>>> is trying to connect to the Namenode server instead of HBase.
>>>>
>>>> You can verify the ports using a command like "/sbin/fuser -n tcp 9090" 
>>>> to
>>>> see which pid has it open, then cross reference against sudo jps.
>>>>
>>>> Thanks
>>>> -Todd
>>>>
>>>> On Thu, Sep 2, 2010 at 4:40 PM, Jinsong Hu <ji...@hotmail.com>
>>>> wrote:
>>>>
>>>>  Hi, There,
>>>>>  I am trying to test and see if thrift for hbase works. I followed the
>>>>> example from
>>>>>
>>>>> http://www.workhabit.com/labs/centos-55-and-thriftscribe
>>>>> http://incubator.apache.org/thrift/
>>>>> http://wiki.apache.org/hadoop/Hbase/ThriftApi
>>>>>
>>>>> and wrote test code: I found that client.getTableNames();
>>>>> returns all table names successfully, but the scanner never returned
>>>>> any record. and it even throws exception:
>>>>>
>>>>> org.apache.thrift.TApplicationException: get failed: unknown result
>>>>>       at
>>>>>
>>>>> org.apache.hadoop.hbase.thrift.generated.Hbase$Client.recv_get(Hbase.java:785)
>>>>>       at
>>>>> org.apache.hadoop.hbase.thrift.generated.Hbase$Client.get(Hbase.java:750)
>>>>>       at
>>>>>
>>>>> org.apache.hadoop.hbase.thrift.HbaseThriftTest.testThriftAPI(HbaseThriftTest.java:73)
>>>>>       at
>>>>>
>>>>> org.apache.hadoop.hbase.thrift.HbaseThriftTest.main(HbaseThriftTest.java:128)
>>>>>
>>>>> I checked the syntax and the table and confirmed that the table does 
>>>>> have
>>>>> record and it should
>>>>> return result, but it doesn't. Can anybody tell me what is wrong ?
>>>>>
>>>>> At this time, I am suspecting the thrift server shipped with CDH3 may
>>>>> have
>>>>> problem. if there anyway
>>>>> I can get this test code working ?
>>>>>
>>>>> I have to use thrift-0.2.0-incubating.tar.gz , as I found that the 
>>>>> code
>>>>> generated from hbase-thrift project
>>>>> doesn't even compile with 0.3.0 and 0.4.0 thrift.
>>>>>
>>>>>
>>>>> Jimmy.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>  public void testThriftAPI()
>>>>>  {
>>>>>      // Make socket
>>>>>      TSocket transport = new TSocket("10.20.12.13", 9090);
>>>>>
>>>>>
>>>>>
>>>>>      // Wrap in a protocol
>>>>>      TBinaryProtocol protocol = new  TBinaryProtocol(transport);
>>>>>
>>>>>      Hbase.Client client = new Hbase.Client(protocol);
>>>>>
>>>>>      try {
>>>>>          transport.open();
>>>>>          List<byte[]> tableNamesList=client.getTableNames();
>>>>>          for (byte [] name : tableNamesList)
>>>>>          {
>>>>>              System.out.println(new String(name));
>>>>>          }
>>>>>
>>>>>          String tableName="HEARTBEAT_CLUSTER";
>>>>>          String startRow="";
>>>>>          List<byte[]> columns= new ArrayList<byte[]>();
>>>>>          columns.add("fields:time_format".getBytes());
>>>>>          columns.add("fields:customer_id".getBytes());
>>>>>
>>>>>          int aScannerId=0;
>>>>>          try {
>>>>>
>>>>>              TCell cell=client.get(tableName.getBytes(),
>>>>> "2010-08\tproduction-2".getBytes(), "fields:customer_id".getBytes());
>>>>>              if (cell != null)
>>>>>              {
>>>>>                 String value= new String( cell.value);
>>>>>                 System.out.println(value);
>>>>>              }
>>>>>              aScannerId=client.scannerOpen(tableName.getBytes(),
>>>>> startRow.getBytes(), columns);
>>>>>              TRowResult result=null;
>>>>>              for (int i=0; i< 10; i++)
>>>>>              {
>>>>>                  try {
>>>>>                  result=client.scannerGet(aScannerId);
>>>>>                  } catch (Exception ex1)
>>>>>                  {
>>>>>
>>>>>                  }
>>>>>                  if (result ==null) break;
>>>>>
>>>>>                  String timeFormat=result.getFieldValue(0).toString();
>>>>>                  String customerId=result.getFieldValue(1).toString();
>>>>>                  System.out.println(timeFormat + "\t" + customerId);
>>>>>              }
>>>>>          } catch (Exception ex)
>>>>>          {
>>>>>              ex.printStackTrace();
>>>>>          }
>>>>>          finally {
>>>>>              if (aScannerId >0 ) client.scannerClose(aScannerId);
>>>>>          }
>>>>>      } catch (Exception exp)
>>>>>      {
>>>>>          exp.printStackTrace();
>>>>>      }
>>>>>      finally{
>>>>>          transport.close();
>>>>>      }
>>>>>
>>>>>  }
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Todd Lipcon
>>>> Software Engineer, Cloudera
>>>>
>>>>
>>
>>
>> --
>> Todd Lipcon
>> Software Engineer, Cloudera
>>
>
>
>
> -- 
> Alexey Kovyrin
> http://kovyrin.net/
>

Re: thrift for hbase in CDH3 broken ?

Posted by Alexey Kovyrin <al...@kovyrin.net>.
We use it in Scribd.com. All clients are ruby web apps.

On Thu, Sep 2, 2010 at 10:49 PM, Todd Lipcon <to...@cloudera.com> wrote:
> On Thu, Sep 2, 2010 at 5:35 PM, Jinsong Hu <ji...@hotmail.com> wrote:
>
>> Yes, I confirmed that it is indeed thrift server.
>>
>> and the fact that the API
>>
>>           List<byte[]> tableNamesList=client.getTableNames();
>>>>          for (byte [] name : tableNamesList)
>>>>          {
>>>>              System.out.println(new String(name));
>>>>          }
>>>>
>>>
>> successfully printed all table names shows that it is indeed thrift server.
>>
>> if it is hue, it won't print the table names.
>>
>> Ah, sorry, I missed that in your original message. Not sure what's up, then
> - we don't have any changes in CDH that would affect this. Anyone here used
> thrift on 0.89.20100621?
>
> -Todd
>
>
>
>> Jimmy.
>>
>> --------------------------------------------------
>> From: "Todd Lipcon" <to...@cloudera.com>
>> Sent: Thursday, September 02, 2010 5:18 PM
>>
>> To: <us...@hbase.apache.org>
>> Subject: Re: thrift for hbase in CDH3 broken ?
>>
>>
>>  Hi Jinsong,
>>>
>>> Are you sure that the port you're connecting to is indeed the thrift
>>> server?
>>>
>>> Unfortunately both the HBase thrift server and the Hue namenode plugin
>>> listen on port 9090, so you might be having an issue where your HBase
>>> client
>>> is trying to connect to the Namenode server instead of HBase.
>>>
>>> You can verify the ports using a command like "/sbin/fuser -n tcp 9090" to
>>> see which pid has it open, then cross reference against sudo jps.
>>>
>>> Thanks
>>> -Todd
>>>
>>> On Thu, Sep 2, 2010 at 4:40 PM, Jinsong Hu <ji...@hotmail.com>
>>> wrote:
>>>
>>>  Hi, There,
>>>>  I am trying to test and see if thrift for hbase works. I followed the
>>>> example from
>>>>
>>>> http://www.workhabit.com/labs/centos-55-and-thriftscribe
>>>> http://incubator.apache.org/thrift/
>>>> http://wiki.apache.org/hadoop/Hbase/ThriftApi
>>>>
>>>> and wrote test code: I found that client.getTableNames();
>>>> returns all table names successfully, but the scanner never returned
>>>> any record. and it even throws exception:
>>>>
>>>> org.apache.thrift.TApplicationException: get failed: unknown result
>>>>       at
>>>>
>>>> org.apache.hadoop.hbase.thrift.generated.Hbase$Client.recv_get(Hbase.java:785)
>>>>       at
>>>> org.apache.hadoop.hbase.thrift.generated.Hbase$Client.get(Hbase.java:750)
>>>>       at
>>>>
>>>> org.apache.hadoop.hbase.thrift.HbaseThriftTest.testThriftAPI(HbaseThriftTest.java:73)
>>>>       at
>>>>
>>>> org.apache.hadoop.hbase.thrift.HbaseThriftTest.main(HbaseThriftTest.java:128)
>>>>
>>>> I checked the syntax and the table and confirmed that the table does have
>>>> record and it should
>>>> return result, but it doesn't. Can anybody tell me what is wrong ?
>>>>
>>>> At this time, I am suspecting the thrift server shipped with CDH3 may
>>>> have
>>>> problem. if there anyway
>>>> I can get this test code working ?
>>>>
>>>> I have to use thrift-0.2.0-incubating.tar.gz , as I found that the code
>>>> generated from hbase-thrift project
>>>> doesn't even compile with 0.3.0 and 0.4.0 thrift.
>>>>
>>>>
>>>> Jimmy.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>  public void testThriftAPI()
>>>>  {
>>>>      // Make socket
>>>>      TSocket transport = new TSocket("10.20.12.13", 9090);
>>>>
>>>>
>>>>
>>>>      // Wrap in a protocol
>>>>      TBinaryProtocol protocol = new  TBinaryProtocol(transport);
>>>>
>>>>      Hbase.Client client = new Hbase.Client(protocol);
>>>>
>>>>      try {
>>>>          transport.open();
>>>>          List<byte[]> tableNamesList=client.getTableNames();
>>>>          for (byte [] name : tableNamesList)
>>>>          {
>>>>              System.out.println(new String(name));
>>>>          }
>>>>
>>>>          String tableName="HEARTBEAT_CLUSTER";
>>>>          String startRow="";
>>>>          List<byte[]> columns= new ArrayList<byte[]>();
>>>>          columns.add("fields:time_format".getBytes());
>>>>          columns.add("fields:customer_id".getBytes());
>>>>
>>>>          int aScannerId=0;
>>>>          try {
>>>>
>>>>              TCell cell=client.get(tableName.getBytes(),
>>>> "2010-08\tproduction-2".getBytes(), "fields:customer_id".getBytes());
>>>>              if (cell != null)
>>>>              {
>>>>                 String value= new String( cell.value);
>>>>                 System.out.println(value);
>>>>              }
>>>>              aScannerId=client.scannerOpen(tableName.getBytes(),
>>>> startRow.getBytes(), columns);
>>>>              TRowResult result=null;
>>>>              for (int i=0; i< 10; i++)
>>>>              {
>>>>                  try {
>>>>                  result=client.scannerGet(aScannerId);
>>>>                  } catch (Exception ex1)
>>>>                  {
>>>>
>>>>                  }
>>>>                  if (result ==null) break;
>>>>
>>>>                  String timeFormat=result.getFieldValue(0).toString();
>>>>                  String customerId=result.getFieldValue(1).toString();
>>>>                  System.out.println(timeFormat + "\t" + customerId);
>>>>              }
>>>>          } catch (Exception ex)
>>>>          {
>>>>              ex.printStackTrace();
>>>>          }
>>>>          finally {
>>>>              if (aScannerId >0 ) client.scannerClose(aScannerId);
>>>>          }
>>>>      } catch (Exception exp)
>>>>      {
>>>>          exp.printStackTrace();
>>>>      }
>>>>      finally{
>>>>          transport.close();
>>>>      }
>>>>
>>>>  }
>>>>
>>>>
>>>
>>>
>>> --
>>> Todd Lipcon
>>> Software Engineer, Cloudera
>>>
>>>
>
>
> --
> Todd Lipcon
> Software Engineer, Cloudera
>



-- 
Alexey Kovyrin
http://kovyrin.net/

Re: thrift for hbase in CDH3 broken ?

Posted by Todd Lipcon <to...@cloudera.com>.
On Thu, Sep 2, 2010 at 5:35 PM, Jinsong Hu <ji...@hotmail.com> wrote:

> Yes, I confirmed that it is indeed thrift server.
>
> and the fact that the API
>
>           List<byte[]> tableNamesList=client.getTableNames();
>>>          for (byte [] name : tableNamesList)
>>>          {
>>>              System.out.println(new String(name));
>>>          }
>>>
>>
> successfully printed all table names shows that it is indeed thrift server.
>
> if it is hue, it won't print the table names.
>
> Ah, sorry, I missed that in your original message. Not sure what's up, then
- we don't have any changes in CDH that would affect this. Anyone here used
thrift on 0.89.20100621?

-Todd



> Jimmy.
>
> --------------------------------------------------
> From: "Todd Lipcon" <to...@cloudera.com>
> Sent: Thursday, September 02, 2010 5:18 PM
>
> To: <us...@hbase.apache.org>
> Subject: Re: thrift for hbase in CDH3 broken ?
>
>
>  Hi Jinsong,
>>
>> Are you sure that the port you're connecting to is indeed the thrift
>> server?
>>
>> Unfortunately both the HBase thrift server and the Hue namenode plugin
>> listen on port 9090, so you might be having an issue where your HBase
>> client
>> is trying to connect to the Namenode server instead of HBase.
>>
>> You can verify the ports using a command like "/sbin/fuser -n tcp 9090" to
>> see which pid has it open, then cross reference against sudo jps.
>>
>> Thanks
>> -Todd
>>
>> On Thu, Sep 2, 2010 at 4:40 PM, Jinsong Hu <ji...@hotmail.com>
>> wrote:
>>
>>  Hi, There,
>>>  I am trying to test and see if thrift for hbase works. I followed the
>>> example from
>>>
>>> http://www.workhabit.com/labs/centos-55-and-thriftscribe
>>> http://incubator.apache.org/thrift/
>>> http://wiki.apache.org/hadoop/Hbase/ThriftApi
>>>
>>> and wrote test code: I found that client.getTableNames();
>>> returns all table names successfully, but the scanner never returned
>>> any record. and it even throws exception:
>>>
>>> org.apache.thrift.TApplicationException: get failed: unknown result
>>>       at
>>>
>>> org.apache.hadoop.hbase.thrift.generated.Hbase$Client.recv_get(Hbase.java:785)
>>>       at
>>> org.apache.hadoop.hbase.thrift.generated.Hbase$Client.get(Hbase.java:750)
>>>       at
>>>
>>> org.apache.hadoop.hbase.thrift.HbaseThriftTest.testThriftAPI(HbaseThriftTest.java:73)
>>>       at
>>>
>>> org.apache.hadoop.hbase.thrift.HbaseThriftTest.main(HbaseThriftTest.java:128)
>>>
>>> I checked the syntax and the table and confirmed that the table does have
>>> record and it should
>>> return result, but it doesn't. Can anybody tell me what is wrong ?
>>>
>>> At this time, I am suspecting the thrift server shipped with CDH3 may
>>> have
>>> problem. if there anyway
>>> I can get this test code working ?
>>>
>>> I have to use thrift-0.2.0-incubating.tar.gz , as I found that the code
>>> generated from hbase-thrift project
>>> doesn't even compile with 0.3.0 and 0.4.0 thrift.
>>>
>>>
>>> Jimmy.
>>>
>>>
>>>
>>>
>>>
>>>
>>>  public void testThriftAPI()
>>>  {
>>>      // Make socket
>>>      TSocket transport = new TSocket("10.20.12.13", 9090);
>>>
>>>
>>>
>>>      // Wrap in a protocol
>>>      TBinaryProtocol protocol = new  TBinaryProtocol(transport);
>>>
>>>      Hbase.Client client = new Hbase.Client(protocol);
>>>
>>>      try {
>>>          transport.open();
>>>          List<byte[]> tableNamesList=client.getTableNames();
>>>          for (byte [] name : tableNamesList)
>>>          {
>>>              System.out.println(new String(name));
>>>          }
>>>
>>>          String tableName="HEARTBEAT_CLUSTER";
>>>          String startRow="";
>>>          List<byte[]> columns= new ArrayList<byte[]>();
>>>          columns.add("fields:time_format".getBytes());
>>>          columns.add("fields:customer_id".getBytes());
>>>
>>>          int aScannerId=0;
>>>          try {
>>>
>>>              TCell cell=client.get(tableName.getBytes(),
>>> "2010-08\tproduction-2".getBytes(), "fields:customer_id".getBytes());
>>>              if (cell != null)
>>>              {
>>>                 String value= new String( cell.value);
>>>                 System.out.println(value);
>>>              }
>>>              aScannerId=client.scannerOpen(tableName.getBytes(),
>>> startRow.getBytes(), columns);
>>>              TRowResult result=null;
>>>              for (int i=0; i< 10; i++)
>>>              {
>>>                  try {
>>>                  result=client.scannerGet(aScannerId);
>>>                  } catch (Exception ex1)
>>>                  {
>>>
>>>                  }
>>>                  if (result ==null) break;
>>>
>>>                  String timeFormat=result.getFieldValue(0).toString();
>>>                  String customerId=result.getFieldValue(1).toString();
>>>                  System.out.println(timeFormat + "\t" + customerId);
>>>              }
>>>          } catch (Exception ex)
>>>          {
>>>              ex.printStackTrace();
>>>          }
>>>          finally {
>>>              if (aScannerId >0 ) client.scannerClose(aScannerId);
>>>          }
>>>      } catch (Exception exp)
>>>      {
>>>          exp.printStackTrace();
>>>      }
>>>      finally{
>>>          transport.close();
>>>      }
>>>
>>>  }
>>>
>>>
>>
>>
>> --
>> Todd Lipcon
>> Software Engineer, Cloudera
>>
>>


-- 
Todd Lipcon
Software Engineer, Cloudera

Re: thrift for hbase in CDH3 broken ?

Posted by Jinsong Hu <ji...@hotmail.com>.
Yes, I confirmed that it is indeed thrift server.

and the fact that the API
>>           List<byte[]> tableNamesList=client.getTableNames();
>>           for (byte [] name : tableNamesList)
>>           {
>>               System.out.println(new String(name));
>>           }

successfully printed all table names shows that it is indeed thrift server.

if it is hue, it won't print the table names.

Jimmy.

--------------------------------------------------
From: "Todd Lipcon" <to...@cloudera.com>
Sent: Thursday, September 02, 2010 5:18 PM
To: <us...@hbase.apache.org>
Subject: Re: thrift for hbase in CDH3 broken ?

> Hi Jinsong,
>
> Are you sure that the port you're connecting to is indeed the thrift 
> server?
>
> Unfortunately both the HBase thrift server and the Hue namenode plugin
> listen on port 9090, so you might be having an issue where your HBase 
> client
> is trying to connect to the Namenode server instead of HBase.
>
> You can verify the ports using a command like "/sbin/fuser -n tcp 9090" to
> see which pid has it open, then cross reference against sudo jps.
>
> Thanks
> -Todd
>
> On Thu, Sep 2, 2010 at 4:40 PM, Jinsong Hu <ji...@hotmail.com> wrote:
>
>> Hi, There,
>>  I am trying to test and see if thrift for hbase works. I followed the
>> example from
>>
>> http://www.workhabit.com/labs/centos-55-and-thriftscribe
>> http://incubator.apache.org/thrift/
>> http://wiki.apache.org/hadoop/Hbase/ThriftApi
>>
>> and wrote test code: I found that client.getTableNames();
>> returns all table names successfully, but the scanner never returned
>> any record. and it even throws exception:
>>
>> org.apache.thrift.TApplicationException: get failed: unknown result
>>        at
>> org.apache.hadoop.hbase.thrift.generated.Hbase$Client.recv_get(Hbase.java:785)
>>        at
>> org.apache.hadoop.hbase.thrift.generated.Hbase$Client.get(Hbase.java:750)
>>        at
>> org.apache.hadoop.hbase.thrift.HbaseThriftTest.testThriftAPI(HbaseThriftTest.java:73)
>>        at
>> org.apache.hadoop.hbase.thrift.HbaseThriftTest.main(HbaseThriftTest.java:128)
>>
>> I checked the syntax and the table and confirmed that the table does have
>> record and it should
>> return result, but it doesn't. Can anybody tell me what is wrong ?
>>
>> At this time, I am suspecting the thrift server shipped with CDH3 may 
>> have
>> problem. if there anyway
>> I can get this test code working ?
>>
>> I have to use thrift-0.2.0-incubating.tar.gz , as I found that the code
>> generated from hbase-thrift project
>> doesn't even compile with 0.3.0 and 0.4.0 thrift.
>>
>>
>> Jimmy.
>>
>>
>>
>>
>>
>>
>>   public void testThriftAPI()
>>   {
>>       // Make socket
>>       TSocket transport = new TSocket("10.20.12.13", 9090);
>>
>>
>>
>>       // Wrap in a protocol
>>       TBinaryProtocol protocol = new  TBinaryProtocol(transport);
>>
>>       Hbase.Client client = new Hbase.Client(protocol);
>>
>>       try {
>>           transport.open();
>>           List<byte[]> tableNamesList=client.getTableNames();
>>           for (byte [] name : tableNamesList)
>>           {
>>               System.out.println(new String(name));
>>           }
>>
>>           String tableName="HEARTBEAT_CLUSTER";
>>           String startRow="";
>>           List<byte[]> columns= new ArrayList<byte[]>();
>>           columns.add("fields:time_format".getBytes());
>>           columns.add("fields:customer_id".getBytes());
>>
>>           int aScannerId=0;
>>           try {
>>
>>               TCell cell=client.get(tableName.getBytes(),
>> "2010-08\tproduction-2".getBytes(), "fields:customer_id".getBytes());
>>               if (cell != null)
>>               {
>>                  String value= new String( cell.value);
>>                  System.out.println(value);
>>               }
>>               aScannerId=client.scannerOpen(tableName.getBytes(),
>> startRow.getBytes(), columns);
>>               TRowResult result=null;
>>               for (int i=0; i< 10; i++)
>>               {
>>                   try {
>>                   result=client.scannerGet(aScannerId);
>>                   } catch (Exception ex1)
>>                   {
>>
>>                   }
>>                   if (result ==null) break;
>>
>>                   String timeFormat=result.getFieldValue(0).toString();
>>                   String customerId=result.getFieldValue(1).toString();
>>                   System.out.println(timeFormat + "\t" + customerId);
>>               }
>>           } catch (Exception ex)
>>           {
>>               ex.printStackTrace();
>>           }
>>           finally {
>>               if (aScannerId >0 ) client.scannerClose(aScannerId);
>>           }
>>       } catch (Exception exp)
>>       {
>>           exp.printStackTrace();
>>       }
>>       finally{
>>           transport.close();
>>       }
>>
>>   }
>>
>
>
>
> -- 
> Todd Lipcon
> Software Engineer, Cloudera
>