You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by Pankaj Misra <pa...@impetus.co.in> on 2012/09/19 06:02:11 UTC
HBase Avro Client
Hi All,
I have been trying to find some help around building an Avro client for HBase, but was not able to find much help, as most of the articles were discussing stand-alone avro as a in-memory data format and serialization engine.
I was able to successfully start the avro server with
$bin/hbase avro start
The server starts up without any problems and runs on a port. Now I want to build an avro hbase client to describe table, column families and to write data into habase using this avro client.
I intend to build this client in Java and my hbase version is 0.94.1. I could also see java avro-tools jar version 1.7.1 in avro downloads. Do I need to use this for building my client. Some clues around this would be very helpful. Thanks.
Thanks and Regards
Pankaj Misra
_
________________________________
Impetus Ranked in the Top 50 India’s Best Companies to Work For 2012.
Impetus webcast ‘Designing a Test Automation Framework for Multi-vendor Interoperable Systems’ available at http://lf1.me/0E/.
NOTE: This message may contain information that is confidential, proprietary, privileged or otherwise protected by law. The message is intended solely for the named addressee. If received in error, please destroy and notify the sender. Any use of this email is prohibited when received in error. Impetus does not represent, warrant and/or guarantee, that the integrity of this communication has been maintained nor that the communication is free of errors, virus, interception or interference.
Re: HBase Avro Client
Posted by Harsh J <ha...@cloudera.com>.
Hi Pankaj,
You reached the Hadoop user lists. Not everyone here may be an HBase
user. Moving this to the HBase user lists (user@hbase.apache.org).
Please use the right project's lists for best responses.
My reply inline.
On Wed, Sep 19, 2012 at 12:44 PM, Pankaj Misra
<pa...@impetus.co.in> wrote:
> Hi All,
>
> I have been searching around for the relevant information around building an avro HBase client for the HBase Avro server running on a port(9090). I have come across the HBase Definitive Guide, but even there I only see the the server startup part in the documentation which I have already done.
I won't recommend using HBase Avro Server today, primarily cause it
had been marked for removal, and is removed:
https://issues.apache.org/jira/browse/HBASE-6553
Instead, try the REST or the Thrift server.
If you instead want to persist in using this anyway, read on.
> There are a few things that I do understand reading the avro and hbase documentation. As per the documentation, HBase provides AvroServer.HBaseImpl, which is a server side of the things. This implementation runs as a server listening to a port (9090) and acts as a bridge between the AvroServer and HBase. The operations supported are as described in the Hbase.avpr file.
The HBaseImpl is what you need actually. See the test-case
http://svn.apache.org/repos/asf/hbase/tags/0.94.1/src/test/java/org/apache/hadoop/hbase/avro/TestAvroServer.java
on how to implement an instance of HBaseImpl and use the various
objects such as ATableDescriptor, etc. with it. These generated
classes are already made available in the distribution.
> The doubts however I have are,
> 1. If I have to build a client that could connect to this port and could create table, column families and insert data, what would it take?
See above.
> 2. Will I have to define a custom schema and have to generate classes out of it?
Doubt it. The docs say you do not need to do this cause for Java
things are pre-provided in a distribution.
> 3. How would I connect to the Avro server running with HBase?
Your client should just specify the right server and port address for
the Avro server.
> 4. Since the communication is rpc based, how will I specify what operation is to be performed at the server from the client?
Via the HBaseImpl client API, same as you'd do on a regular client :)
> As I see, the documentation on this issue is very little, hence I am seeking help from the community.
>
> Requesting for help. Thanks.
>
> Regards
> Pankaj Misra
>
> ________________________________________
> From: Pankaj Misra
> Sent: Wednesday, September 19, 2012 9:32 AM
> To: user@hadoop.apache.org
> Subject: HBase Avro Client
>
> Hi All,
>
> I have been trying to find some help around building an Avro client for HBase, but was not able to find much help, as most of the articles were discussing stand-alone avro as a in-memory data format and serialization engine.
>
> I was able to successfully start the avro server with
> $bin/hbase avro start
>
> The server starts up without any problems and runs on a port. Now I want to build an avro hbase client to describe table, column families and to write data into habase using this avro client.
>
> I intend to build this client in Java and my hbase version is 0.94.1. I could also see java avro-tools jar version 1.7.1 in avro downloads. Do I need to use this for building my client. Some clues around this would be very helpful. Thanks.
>
> Thanks and Regards
> Pankaj Misra
>
>
> _
>
> ________________________________
>
> Impetus Ranked in the Top 50 India’s Best Companies to Work For 2012.
>
> Impetus webcast ‘Designing a Test Automation Framework for Multi-vendor Interoperable Systems’ available at http://lf1.me/0E/.
>
>
> NOTE: This message may contain information that is confidential, proprietary, privileged or otherwise protected by law. The message is intended solely for the named addressee. If received in error, please destroy and notify the sender. Any use of this email is prohibited when received in error. Impetus does not represent, warrant and/or guarantee, that the integrity of this communication has been maintained nor that the communication is free of errors, virus, interception or interference.
>
> ________________________________
>
> Impetus Ranked in the Top 50 India’s Best Companies to Work For 2012.
>
> Impetus webcast ‘Designing a Test Automation Framework for Multi-vendor Interoperable Systems’ available at http://lf1.me/0E/.
>
>
> NOTE: This message may contain information that is confidential, proprietary, privileged or otherwise protected by law. The message is intended solely for the named addressee. If received in error, please destroy and notify the sender. Any use of this email is prohibited when received in error. Impetus does not represent, warrant and/or guarantee, that the integrity of this communication has been maintained nor that the communication is free of errors, virus, interception or interference.
--
Harsh J
Re: HBase Avro Client
Posted by Harsh J <ha...@cloudera.com>.
Hi Pankaj,
You reached the Hadoop user lists. Not everyone here may be an HBase
user. Moving this to the HBase user lists (user@hbase.apache.org).
Please use the right project's lists for best responses.
My reply inline.
On Wed, Sep 19, 2012 at 12:44 PM, Pankaj Misra
<pa...@impetus.co.in> wrote:
> Hi All,
>
> I have been searching around for the relevant information around building an avro HBase client for the HBase Avro server running on a port(9090). I have come across the HBase Definitive Guide, but even there I only see the the server startup part in the documentation which I have already done.
I won't recommend using HBase Avro Server today, primarily cause it
had been marked for removal, and is removed:
https://issues.apache.org/jira/browse/HBASE-6553
Instead, try the REST or the Thrift server.
If you instead want to persist in using this anyway, read on.
> There are a few things that I do understand reading the avro and hbase documentation. As per the documentation, HBase provides AvroServer.HBaseImpl, which is a server side of the things. This implementation runs as a server listening to a port (9090) and acts as a bridge between the AvroServer and HBase. The operations supported are as described in the Hbase.avpr file.
The HBaseImpl is what you need actually. See the test-case
http://svn.apache.org/repos/asf/hbase/tags/0.94.1/src/test/java/org/apache/hadoop/hbase/avro/TestAvroServer.java
on how to implement an instance of HBaseImpl and use the various
objects such as ATableDescriptor, etc. with it. These generated
classes are already made available in the distribution.
> The doubts however I have are,
> 1. If I have to build a client that could connect to this port and could create table, column families and insert data, what would it take?
See above.
> 2. Will I have to define a custom schema and have to generate classes out of it?
Doubt it. The docs say you do not need to do this cause for Java
things are pre-provided in a distribution.
> 3. How would I connect to the Avro server running with HBase?
Your client should just specify the right server and port address for
the Avro server.
> 4. Since the communication is rpc based, how will I specify what operation is to be performed at the server from the client?
Via the HBaseImpl client API, same as you'd do on a regular client :)
> As I see, the documentation on this issue is very little, hence I am seeking help from the community.
>
> Requesting for help. Thanks.
>
> Regards
> Pankaj Misra
>
> ________________________________________
> From: Pankaj Misra
> Sent: Wednesday, September 19, 2012 9:32 AM
> To: user@hadoop.apache.org
> Subject: HBase Avro Client
>
> Hi All,
>
> I have been trying to find some help around building an Avro client for HBase, but was not able to find much help, as most of the articles were discussing stand-alone avro as a in-memory data format and serialization engine.
>
> I was able to successfully start the avro server with
> $bin/hbase avro start
>
> The server starts up without any problems and runs on a port. Now I want to build an avro hbase client to describe table, column families and to write data into habase using this avro client.
>
> I intend to build this client in Java and my hbase version is 0.94.1. I could also see java avro-tools jar version 1.7.1 in avro downloads. Do I need to use this for building my client. Some clues around this would be very helpful. Thanks.
>
> Thanks and Regards
> Pankaj Misra
>
>
> _
>
> ________________________________
>
> Impetus Ranked in the Top 50 India’s Best Companies to Work For 2012.
>
> Impetus webcast ‘Designing a Test Automation Framework for Multi-vendor Interoperable Systems’ available at http://lf1.me/0E/.
>
>
> NOTE: This message may contain information that is confidential, proprietary, privileged or otherwise protected by law. The message is intended solely for the named addressee. If received in error, please destroy and notify the sender. Any use of this email is prohibited when received in error. Impetus does not represent, warrant and/or guarantee, that the integrity of this communication has been maintained nor that the communication is free of errors, virus, interception or interference.
>
> ________________________________
>
> Impetus Ranked in the Top 50 India’s Best Companies to Work For 2012.
>
> Impetus webcast ‘Designing a Test Automation Framework for Multi-vendor Interoperable Systems’ available at http://lf1.me/0E/.
>
>
> NOTE: This message may contain information that is confidential, proprietary, privileged or otherwise protected by law. The message is intended solely for the named addressee. If received in error, please destroy and notify the sender. Any use of this email is prohibited when received in error. Impetus does not represent, warrant and/or guarantee, that the integrity of this communication has been maintained nor that the communication is free of errors, virus, interception or interference.
--
Harsh J
Re: HBase Avro Client
Posted by Harsh J <ha...@cloudera.com>.
Hi Pankaj,
You reached the Hadoop user lists. Not everyone here may be an HBase
user. Moving this to the HBase user lists (user@hbase.apache.org).
Please use the right project's lists for best responses.
My reply inline.
On Wed, Sep 19, 2012 at 12:44 PM, Pankaj Misra
<pa...@impetus.co.in> wrote:
> Hi All,
>
> I have been searching around for the relevant information around building an avro HBase client for the HBase Avro server running on a port(9090). I have come across the HBase Definitive Guide, but even there I only see the the server startup part in the documentation which I have already done.
I won't recommend using HBase Avro Server today, primarily cause it
had been marked for removal, and is removed:
https://issues.apache.org/jira/browse/HBASE-6553
Instead, try the REST or the Thrift server.
If you instead want to persist in using this anyway, read on.
> There are a few things that I do understand reading the avro and hbase documentation. As per the documentation, HBase provides AvroServer.HBaseImpl, which is a server side of the things. This implementation runs as a server listening to a port (9090) and acts as a bridge between the AvroServer and HBase. The operations supported are as described in the Hbase.avpr file.
The HBaseImpl is what you need actually. See the test-case
http://svn.apache.org/repos/asf/hbase/tags/0.94.1/src/test/java/org/apache/hadoop/hbase/avro/TestAvroServer.java
on how to implement an instance of HBaseImpl and use the various
objects such as ATableDescriptor, etc. with it. These generated
classes are already made available in the distribution.
> The doubts however I have are,
> 1. If I have to build a client that could connect to this port and could create table, column families and insert data, what would it take?
See above.
> 2. Will I have to define a custom schema and have to generate classes out of it?
Doubt it. The docs say you do not need to do this cause for Java
things are pre-provided in a distribution.
> 3. How would I connect to the Avro server running with HBase?
Your client should just specify the right server and port address for
the Avro server.
> 4. Since the communication is rpc based, how will I specify what operation is to be performed at the server from the client?
Via the HBaseImpl client API, same as you'd do on a regular client :)
> As I see, the documentation on this issue is very little, hence I am seeking help from the community.
>
> Requesting for help. Thanks.
>
> Regards
> Pankaj Misra
>
> ________________________________________
> From: Pankaj Misra
> Sent: Wednesday, September 19, 2012 9:32 AM
> To: user@hadoop.apache.org
> Subject: HBase Avro Client
>
> Hi All,
>
> I have been trying to find some help around building an Avro client for HBase, but was not able to find much help, as most of the articles were discussing stand-alone avro as a in-memory data format and serialization engine.
>
> I was able to successfully start the avro server with
> $bin/hbase avro start
>
> The server starts up without any problems and runs on a port. Now I want to build an avro hbase client to describe table, column families and to write data into habase using this avro client.
>
> I intend to build this client in Java and my hbase version is 0.94.1. I could also see java avro-tools jar version 1.7.1 in avro downloads. Do I need to use this for building my client. Some clues around this would be very helpful. Thanks.
>
> Thanks and Regards
> Pankaj Misra
>
>
> _
>
> ________________________________
>
> Impetus Ranked in the Top 50 India’s Best Companies to Work For 2012.
>
> Impetus webcast ‘Designing a Test Automation Framework for Multi-vendor Interoperable Systems’ available at http://lf1.me/0E/.
>
>
> NOTE: This message may contain information that is confidential, proprietary, privileged or otherwise protected by law. The message is intended solely for the named addressee. If received in error, please destroy and notify the sender. Any use of this email is prohibited when received in error. Impetus does not represent, warrant and/or guarantee, that the integrity of this communication has been maintained nor that the communication is free of errors, virus, interception or interference.
>
> ________________________________
>
> Impetus Ranked in the Top 50 India’s Best Companies to Work For 2012.
>
> Impetus webcast ‘Designing a Test Automation Framework for Multi-vendor Interoperable Systems’ available at http://lf1.me/0E/.
>
>
> NOTE: This message may contain information that is confidential, proprietary, privileged or otherwise protected by law. The message is intended solely for the named addressee. If received in error, please destroy and notify the sender. Any use of this email is prohibited when received in error. Impetus does not represent, warrant and/or guarantee, that the integrity of this communication has been maintained nor that the communication is free of errors, virus, interception or interference.
--
Harsh J
Re: HBase Avro Client
Posted by Harsh J <ha...@cloudera.com>.
Hi Pankaj,
You reached the Hadoop user lists. Not everyone here may be an HBase
user. Moving this to the HBase user lists (user@hbase.apache.org).
Please use the right project's lists for best responses.
My reply inline.
On Wed, Sep 19, 2012 at 12:44 PM, Pankaj Misra
<pa...@impetus.co.in> wrote:
> Hi All,
>
> I have been searching around for the relevant information around building an avro HBase client for the HBase Avro server running on a port(9090). I have come across the HBase Definitive Guide, but even there I only see the the server startup part in the documentation which I have already done.
I won't recommend using HBase Avro Server today, primarily cause it
had been marked for removal, and is removed:
https://issues.apache.org/jira/browse/HBASE-6553
Instead, try the REST or the Thrift server.
If you instead want to persist in using this anyway, read on.
> There are a few things that I do understand reading the avro and hbase documentation. As per the documentation, HBase provides AvroServer.HBaseImpl, which is a server side of the things. This implementation runs as a server listening to a port (9090) and acts as a bridge between the AvroServer and HBase. The operations supported are as described in the Hbase.avpr file.
The HBaseImpl is what you need actually. See the test-case
http://svn.apache.org/repos/asf/hbase/tags/0.94.1/src/test/java/org/apache/hadoop/hbase/avro/TestAvroServer.java
on how to implement an instance of HBaseImpl and use the various
objects such as ATableDescriptor, etc. with it. These generated
classes are already made available in the distribution.
> The doubts however I have are,
> 1. If I have to build a client that could connect to this port and could create table, column families and insert data, what would it take?
See above.
> 2. Will I have to define a custom schema and have to generate classes out of it?
Doubt it. The docs say you do not need to do this cause for Java
things are pre-provided in a distribution.
> 3. How would I connect to the Avro server running with HBase?
Your client should just specify the right server and port address for
the Avro server.
> 4. Since the communication is rpc based, how will I specify what operation is to be performed at the server from the client?
Via the HBaseImpl client API, same as you'd do on a regular client :)
> As I see, the documentation on this issue is very little, hence I am seeking help from the community.
>
> Requesting for help. Thanks.
>
> Regards
> Pankaj Misra
>
> ________________________________________
> From: Pankaj Misra
> Sent: Wednesday, September 19, 2012 9:32 AM
> To: user@hadoop.apache.org
> Subject: HBase Avro Client
>
> Hi All,
>
> I have been trying to find some help around building an Avro client for HBase, but was not able to find much help, as most of the articles were discussing stand-alone avro as a in-memory data format and serialization engine.
>
> I was able to successfully start the avro server with
> $bin/hbase avro start
>
> The server starts up without any problems and runs on a port. Now I want to build an avro hbase client to describe table, column families and to write data into habase using this avro client.
>
> I intend to build this client in Java and my hbase version is 0.94.1. I could also see java avro-tools jar version 1.7.1 in avro downloads. Do I need to use this for building my client. Some clues around this would be very helpful. Thanks.
>
> Thanks and Regards
> Pankaj Misra
>
>
> _
>
> ________________________________
>
> Impetus Ranked in the Top 50 India’s Best Companies to Work For 2012.
>
> Impetus webcast ‘Designing a Test Automation Framework for Multi-vendor Interoperable Systems’ available at http://lf1.me/0E/.
>
>
> NOTE: This message may contain information that is confidential, proprietary, privileged or otherwise protected by law. The message is intended solely for the named addressee. If received in error, please destroy and notify the sender. Any use of this email is prohibited when received in error. Impetus does not represent, warrant and/or guarantee, that the integrity of this communication has been maintained nor that the communication is free of errors, virus, interception or interference.
>
> ________________________________
>
> Impetus Ranked in the Top 50 India’s Best Companies to Work For 2012.
>
> Impetus webcast ‘Designing a Test Automation Framework for Multi-vendor Interoperable Systems’ available at http://lf1.me/0E/.
>
>
> NOTE: This message may contain information that is confidential, proprietary, privileged or otherwise protected by law. The message is intended solely for the named addressee. If received in error, please destroy and notify the sender. Any use of this email is prohibited when received in error. Impetus does not represent, warrant and/or guarantee, that the integrity of this communication has been maintained nor that the communication is free of errors, virus, interception or interference.
--
Harsh J
Re: HBase Avro Client
Posted by Harsh J <ha...@cloudera.com>.
Hi Pankaj,
You reached the Hadoop user lists. Not everyone here may be an HBase
user. Moving this to the HBase user lists (user@hbase.apache.org).
Please use the right project's lists for best responses.
My reply inline.
On Wed, Sep 19, 2012 at 12:44 PM, Pankaj Misra
<pa...@impetus.co.in> wrote:
> Hi All,
>
> I have been searching around for the relevant information around building an avro HBase client for the HBase Avro server running on a port(9090). I have come across the HBase Definitive Guide, but even there I only see the the server startup part in the documentation which I have already done.
I won't recommend using HBase Avro Server today, primarily cause it
had been marked for removal, and is removed:
https://issues.apache.org/jira/browse/HBASE-6553
Instead, try the REST or the Thrift server.
If you instead want to persist in using this anyway, read on.
> There are a few things that I do understand reading the avro and hbase documentation. As per the documentation, HBase provides AvroServer.HBaseImpl, which is a server side of the things. This implementation runs as a server listening to a port (9090) and acts as a bridge between the AvroServer and HBase. The operations supported are as described in the Hbase.avpr file.
The HBaseImpl is what you need actually. See the test-case
http://svn.apache.org/repos/asf/hbase/tags/0.94.1/src/test/java/org/apache/hadoop/hbase/avro/TestAvroServer.java
on how to implement an instance of HBaseImpl and use the various
objects such as ATableDescriptor, etc. with it. These generated
classes are already made available in the distribution.
> The doubts however I have are,
> 1. If I have to build a client that could connect to this port and could create table, column families and insert data, what would it take?
See above.
> 2. Will I have to define a custom schema and have to generate classes out of it?
Doubt it. The docs say you do not need to do this cause for Java
things are pre-provided in a distribution.
> 3. How would I connect to the Avro server running with HBase?
Your client should just specify the right server and port address for
the Avro server.
> 4. Since the communication is rpc based, how will I specify what operation is to be performed at the server from the client?
Via the HBaseImpl client API, same as you'd do on a regular client :)
> As I see, the documentation on this issue is very little, hence I am seeking help from the community.
>
> Requesting for help. Thanks.
>
> Regards
> Pankaj Misra
>
> ________________________________________
> From: Pankaj Misra
> Sent: Wednesday, September 19, 2012 9:32 AM
> To: user@hadoop.apache.org
> Subject: HBase Avro Client
>
> Hi All,
>
> I have been trying to find some help around building an Avro client for HBase, but was not able to find much help, as most of the articles were discussing stand-alone avro as a in-memory data format and serialization engine.
>
> I was able to successfully start the avro server with
> $bin/hbase avro start
>
> The server starts up without any problems and runs on a port. Now I want to build an avro hbase client to describe table, column families and to write data into habase using this avro client.
>
> I intend to build this client in Java and my hbase version is 0.94.1. I could also see java avro-tools jar version 1.7.1 in avro downloads. Do I need to use this for building my client. Some clues around this would be very helpful. Thanks.
>
> Thanks and Regards
> Pankaj Misra
>
>
> _
>
> ________________________________
>
> Impetus Ranked in the Top 50 India’s Best Companies to Work For 2012.
>
> Impetus webcast ‘Designing a Test Automation Framework for Multi-vendor Interoperable Systems’ available at http://lf1.me/0E/.
>
>
> NOTE: This message may contain information that is confidential, proprietary, privileged or otherwise protected by law. The message is intended solely for the named addressee. If received in error, please destroy and notify the sender. Any use of this email is prohibited when received in error. Impetus does not represent, warrant and/or guarantee, that the integrity of this communication has been maintained nor that the communication is free of errors, virus, interception or interference.
>
> ________________________________
>
> Impetus Ranked in the Top 50 India’s Best Companies to Work For 2012.
>
> Impetus webcast ‘Designing a Test Automation Framework for Multi-vendor Interoperable Systems’ available at http://lf1.me/0E/.
>
>
> NOTE: This message may contain information that is confidential, proprietary, privileged or otherwise protected by law. The message is intended solely for the named addressee. If received in error, please destroy and notify the sender. Any use of this email is prohibited when received in error. Impetus does not represent, warrant and/or guarantee, that the integrity of this communication has been maintained nor that the communication is free of errors, virus, interception or interference.
--
Harsh J
RE: HBase Avro Client
Posted by Pankaj Misra <pa...@impetus.co.in>.
Hi All,
I have been searching around for the relevant information around building an avro HBase client for the HBase Avro server running on a port(9090). I have come across the HBase Definitive Guide, but even there I only see the the server startup part in the documentation which I have already done.
There are a few things that I do understand reading the avro and hbase documentation. As per the documentation, HBase provides AvroServer.HBaseImpl, which is a server side of the things. This implementation runs as a server listening to a port (9090) and acts as a bridge between the AvroServer and HBase. The operations supported are as described in the Hbase.avpr file.
The doubts however I have are,
1. If I have to build a client that could connect to this port and could create table, column families and insert data, what would it take?
2. Will I have to define a custom schema and have to generate classes out of it?
3. How would I connect to the Avro server running with HBase?
4. Since the communication is rpc based, how will I specify what operation is to be performed at the server from the client?
As I see, the documentation on this issue is very little, hence I am seeking help from the community.
Requesting for help. Thanks.
Regards
Pankaj Misra
________________________________________
From: Pankaj Misra
Sent: Wednesday, September 19, 2012 9:32 AM
To: user@hadoop.apache.org
Subject: HBase Avro Client
Hi All,
I have been trying to find some help around building an Avro client for HBase, but was not able to find much help, as most of the articles were discussing stand-alone avro as a in-memory data format and serialization engine.
I was able to successfully start the avro server with
$bin/hbase avro start
The server starts up without any problems and runs on a port. Now I want to build an avro hbase client to describe table, column families and to write data into habase using this avro client.
I intend to build this client in Java and my hbase version is 0.94.1. I could also see java avro-tools jar version 1.7.1 in avro downloads. Do I need to use this for building my client. Some clues around this would be very helpful. Thanks.
Thanks and Regards
Pankaj Misra
_
________________________________
Impetus Ranked in the Top 50 India’s Best Companies to Work For 2012.
Impetus webcast ‘Designing a Test Automation Framework for Multi-vendor Interoperable Systems’ available at http://lf1.me/0E/.
NOTE: This message may contain information that is confidential, proprietary, privileged or otherwise protected by law. The message is intended solely for the named addressee. If received in error, please destroy and notify the sender. Any use of this email is prohibited when received in error. Impetus does not represent, warrant and/or guarantee, that the integrity of this communication has been maintained nor that the communication is free of errors, virus, interception or interference.
________________________________
Impetus Ranked in the Top 50 India’s Best Companies to Work For 2012.
Impetus webcast ‘Designing a Test Automation Framework for Multi-vendor Interoperable Systems’ available at http://lf1.me/0E/.
NOTE: This message may contain information that is confidential, proprietary, privileged or otherwise protected by law. The message is intended solely for the named addressee. If received in error, please destroy and notify the sender. Any use of this email is prohibited when received in error. Impetus does not represent, warrant and/or guarantee, that the integrity of this communication has been maintained nor that the communication is free of errors, virus, interception or interference.
RE: HBase Avro Client
Posted by Pankaj Misra <pa...@impetus.co.in>.
Hi All,
I have been searching around for the relevant information around building an avro HBase client for the HBase Avro server running on a port(9090). I have come across the HBase Definitive Guide, but even there I only see the the server startup part in the documentation which I have already done.
There are a few things that I do understand reading the avro and hbase documentation. As per the documentation, HBase provides AvroServer.HBaseImpl, which is a server side of the things. This implementation runs as a server listening to a port (9090) and acts as a bridge between the AvroServer and HBase. The operations supported are as described in the Hbase.avpr file.
The doubts however I have are,
1. If I have to build a client that could connect to this port and could create table, column families and insert data, what would it take?
2. Will I have to define a custom schema and have to generate classes out of it?
3. How would I connect to the Avro server running with HBase?
4. Since the communication is rpc based, how will I specify what operation is to be performed at the server from the client?
As I see, the documentation on this issue is very little, hence I am seeking help from the community.
Requesting for help. Thanks.
Regards
Pankaj Misra
________________________________________
From: Pankaj Misra
Sent: Wednesday, September 19, 2012 9:32 AM
To: user@hadoop.apache.org
Subject: HBase Avro Client
Hi All,
I have been trying to find some help around building an Avro client for HBase, but was not able to find much help, as most of the articles were discussing stand-alone avro as a in-memory data format and serialization engine.
I was able to successfully start the avro server with
$bin/hbase avro start
The server starts up without any problems and runs on a port. Now I want to build an avro hbase client to describe table, column families and to write data into habase using this avro client.
I intend to build this client in Java and my hbase version is 0.94.1. I could also see java avro-tools jar version 1.7.1 in avro downloads. Do I need to use this for building my client. Some clues around this would be very helpful. Thanks.
Thanks and Regards
Pankaj Misra
_
________________________________
Impetus Ranked in the Top 50 India’s Best Companies to Work For 2012.
Impetus webcast ‘Designing a Test Automation Framework for Multi-vendor Interoperable Systems’ available at http://lf1.me/0E/.
NOTE: This message may contain information that is confidential, proprietary, privileged or otherwise protected by law. The message is intended solely for the named addressee. If received in error, please destroy and notify the sender. Any use of this email is prohibited when received in error. Impetus does not represent, warrant and/or guarantee, that the integrity of this communication has been maintained nor that the communication is free of errors, virus, interception or interference.
________________________________
Impetus Ranked in the Top 50 India’s Best Companies to Work For 2012.
Impetus webcast ‘Designing a Test Automation Framework for Multi-vendor Interoperable Systems’ available at http://lf1.me/0E/.
NOTE: This message may contain information that is confidential, proprietary, privileged or otherwise protected by law. The message is intended solely for the named addressee. If received in error, please destroy and notify the sender. Any use of this email is prohibited when received in error. Impetus does not represent, warrant and/or guarantee, that the integrity of this communication has been maintained nor that the communication is free of errors, virus, interception or interference.
RE: HBase Avro Client
Posted by Pankaj Misra <pa...@impetus.co.in>.
Hi All,
I have been searching around for the relevant information around building an avro HBase client for the HBase Avro server running on a port(9090). I have come across the HBase Definitive Guide, but even there I only see the the server startup part in the documentation which I have already done.
There are a few things that I do understand reading the avro and hbase documentation. As per the documentation, HBase provides AvroServer.HBaseImpl, which is a server side of the things. This implementation runs as a server listening to a port (9090) and acts as a bridge between the AvroServer and HBase. The operations supported are as described in the Hbase.avpr file.
The doubts however I have are,
1. If I have to build a client that could connect to this port and could create table, column families and insert data, what would it take?
2. Will I have to define a custom schema and have to generate classes out of it?
3. How would I connect to the Avro server running with HBase?
4. Since the communication is rpc based, how will I specify what operation is to be performed at the server from the client?
As I see, the documentation on this issue is very little, hence I am seeking help from the community.
Requesting for help. Thanks.
Regards
Pankaj Misra
________________________________________
From: Pankaj Misra
Sent: Wednesday, September 19, 2012 9:32 AM
To: user@hadoop.apache.org
Subject: HBase Avro Client
Hi All,
I have been trying to find some help around building an Avro client for HBase, but was not able to find much help, as most of the articles were discussing stand-alone avro as a in-memory data format and serialization engine.
I was able to successfully start the avro server with
$bin/hbase avro start
The server starts up without any problems and runs on a port. Now I want to build an avro hbase client to describe table, column families and to write data into habase using this avro client.
I intend to build this client in Java and my hbase version is 0.94.1. I could also see java avro-tools jar version 1.7.1 in avro downloads. Do I need to use this for building my client. Some clues around this would be very helpful. Thanks.
Thanks and Regards
Pankaj Misra
_
________________________________
Impetus Ranked in the Top 50 India’s Best Companies to Work For 2012.
Impetus webcast ‘Designing a Test Automation Framework for Multi-vendor Interoperable Systems’ available at http://lf1.me/0E/.
NOTE: This message may contain information that is confidential, proprietary, privileged or otherwise protected by law. The message is intended solely for the named addressee. If received in error, please destroy and notify the sender. Any use of this email is prohibited when received in error. Impetus does not represent, warrant and/or guarantee, that the integrity of this communication has been maintained nor that the communication is free of errors, virus, interception or interference.
________________________________
Impetus Ranked in the Top 50 India’s Best Companies to Work For 2012.
Impetus webcast ‘Designing a Test Automation Framework for Multi-vendor Interoperable Systems’ available at http://lf1.me/0E/.
NOTE: This message may contain information that is confidential, proprietary, privileged or otherwise protected by law. The message is intended solely for the named addressee. If received in error, please destroy and notify the sender. Any use of this email is prohibited when received in error. Impetus does not represent, warrant and/or guarantee, that the integrity of this communication has been maintained nor that the communication is free of errors, virus, interception or interference.
RE: HBase Avro Client
Posted by Pankaj Misra <pa...@impetus.co.in>.
Hi All,
I have been searching around for the relevant information around building an avro HBase client for the HBase Avro server running on a port(9090). I have come across the HBase Definitive Guide, but even there I only see the the server startup part in the documentation which I have already done.
There are a few things that I do understand reading the avro and hbase documentation. As per the documentation, HBase provides AvroServer.HBaseImpl, which is a server side of the things. This implementation runs as a server listening to a port (9090) and acts as a bridge between the AvroServer and HBase. The operations supported are as described in the Hbase.avpr file.
The doubts however I have are,
1. If I have to build a client that could connect to this port and could create table, column families and insert data, what would it take?
2. Will I have to define a custom schema and have to generate classes out of it?
3. How would I connect to the Avro server running with HBase?
4. Since the communication is rpc based, how will I specify what operation is to be performed at the server from the client?
As I see, the documentation on this issue is very little, hence I am seeking help from the community.
Requesting for help. Thanks.
Regards
Pankaj Misra
________________________________________
From: Pankaj Misra
Sent: Wednesday, September 19, 2012 9:32 AM
To: user@hadoop.apache.org
Subject: HBase Avro Client
Hi All,
I have been trying to find some help around building an Avro client for HBase, but was not able to find much help, as most of the articles were discussing stand-alone avro as a in-memory data format and serialization engine.
I was able to successfully start the avro server with
$bin/hbase avro start
The server starts up without any problems and runs on a port. Now I want to build an avro hbase client to describe table, column families and to write data into habase using this avro client.
I intend to build this client in Java and my hbase version is 0.94.1. I could also see java avro-tools jar version 1.7.1 in avro downloads. Do I need to use this for building my client. Some clues around this would be very helpful. Thanks.
Thanks and Regards
Pankaj Misra
_
________________________________
Impetus Ranked in the Top 50 India’s Best Companies to Work For 2012.
Impetus webcast ‘Designing a Test Automation Framework for Multi-vendor Interoperable Systems’ available at http://lf1.me/0E/.
NOTE: This message may contain information that is confidential, proprietary, privileged or otherwise protected by law. The message is intended solely for the named addressee. If received in error, please destroy and notify the sender. Any use of this email is prohibited when received in error. Impetus does not represent, warrant and/or guarantee, that the integrity of this communication has been maintained nor that the communication is free of errors, virus, interception or interference.
________________________________
Impetus Ranked in the Top 50 India’s Best Companies to Work For 2012.
Impetus webcast ‘Designing a Test Automation Framework for Multi-vendor Interoperable Systems’ available at http://lf1.me/0E/.
NOTE: This message may contain information that is confidential, proprietary, privileged or otherwise protected by law. The message is intended solely for the named addressee. If received in error, please destroy and notify the sender. Any use of this email is prohibited when received in error. Impetus does not represent, warrant and/or guarantee, that the integrity of this communication has been maintained nor that the communication is free of errors, virus, interception or interference.