You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Nick Dimiduk <nd...@apache.org> on 2020/02/20 18:53:38 UTC
Running HBase on Hadoop 3.1, was Re: [ANNOUNCE] Apache HBase 2.1.9 is
now available for download
changed subject, dropped dev
Hello Mich,
The HBase binary distribution includes all Hadoop client jars necessary for
HBase to function on top of HDFS. The version of those Hadoop jars is that
of Hadoop 2.8.5. Duo is saying that the Hadoop 2.8.5 client works against a
HDFS 3.1.x cluster. Thus, this binary release of HBase is expected to work
with HDFS 3.1.
Alternatively, you can choose to recompile HBase using Hadoop 3.1.x as the
dependency. In that case, you'll need to retrieve the source, establish a
build environment, and perform the build yourself -- the HBase project does
not distribute a binary artifact compiled against Hadoop 3.
Thanks,
Nick
On Thu, Feb 20, 2020 at 3:20 AM Mich Talebzadeh <mi...@gmail.com>
wrote:
> Thank you.
>
> In your points below
>
> If you want to run HBase on top of a 3.1.x HDFS cluster, you can just use
> the binaries, hadoop 2.8.5 client can communicate with 3.1.x server.
>
> If you want HBase to use hadoop 3.1.x, then you need to build the binaries
> by yourself, use the hadoop 3 profile.
>
> Could you please clarify (since Hbase can work with Hadoop), which version
> of new release has been tested against which version of Hadoop.
>
> Also it is not very clear which binaries you are referring to? "Hadoop
> 2.8.5 client can communicate with "Hadoop" 3.1 server. So basically
> download Hadoop 2.8.5 client for Hbase use?
>
>
> Regards,
>
>
>
> Dr Mich Talebzadeh
>
>
>
> LinkedIn *
> https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
> <
> https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
> >*
>
>
>
> http://talebzadehmich.wordpress.com
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
>
> On Thu, 20 Feb 2020 at 09:45, 张铎(Duo Zhang) <pa...@gmail.com> wrote:
>
> > If you want to run HBase on top of a 3.1.x HDFS cluster, you can just use
> > the binaries, hadoop 2.8.5 client can communicate with 3.1.x server.
> >
> > If you want HBase to use hadoop 3.1.x, then you need to build the
> binaries
> > by yourself, use the hadoop 3 profile.
> >
> > Mich Talebzadeh <mi...@gmail.com> 于2020年2月20日周四 下午5:15写道：
> >
> > > Hi,
> > >
> > > Thanks.
> > >
> > > Does this version of Hbase work with Hadoop 3.1? I am still stuck with
> > > Hbase 1.2.7
> > >
> > > Hadoop 3.1.0
> > > Source code repository https://github.com/apache/hadoop -r
> > > 16b70619a24cdcf5d3b0fcf4b58ca77238ccbe6d
> > > Compiled by centos on 2018-03-30T00:00Z
> > > Compiled with protoc 2.5.0
> > >
> > > Regards,
> > >
> > > Dr Mich Talebzadeh
> > >
> > >
> > >
> > > LinkedIn *
> > >
> >
> https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
> > > <
> > >
> >
> https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
> > > >*
> > >
> > >
> > >
> > > http://talebzadehmich.wordpress.com
> > >
> > >
> > > *Disclaimer:* Use it at your own risk. Any and all responsibility for
> any
> > > loss, damage or destruction of data or any other property which may
> arise
> > > from relying on this email's technical content is explicitly
> disclaimed.
> > > The author will in no case be liable for any monetary damages arising
> > from
> > > such loss, damage or destruction.
> > >
> > >
> > >
> > >
> > > On Thu, 20 Feb 2020 at 08:48, Duo Zhang <zh...@apache.org> wrote:
> > >
> > > > The HBase team is happy to announce the immediate availability of
> > Apache
> > > > HBase 2.1.9.
> > > >
> > > > Download from https://hbase.apache.org/downloads.html
> > > >
> > > > Apache HBase is an open-source, distributed, versioned,
> non-relational
> > > > database. Apache HBase gives you low latency random access to
> billions
> > of
> > > > rows with millions of columns atop non-specialized hardware. To learn
> > > more
> > > > about HBase, see https://hbase.apache.org/.
> > > >
> > > > HBase 2.1.9 is the latest release of the HBase 2.1 line, continuing
> on
> > > the
> > > > theme of bringing a stable, reliable database to the Apache Big Data
> > > > ecosystem and beyond. 2.1.9 includes ~62 bug and improvement fixes
> done
> > > > since the 2.1.8.
> > > >
> > > > For instructions on verifying ASF release downloads, please see
> > > >
> > > > https://www.apache.org/dyn/closer.cgi#verify
> > > >
> > > > Project member signature keys can be found at
> > > >
> > > > https://www.apache.org/dist/hbase/KEYS
> > > >
> > > > Thanks to all the contributors who made this release possible!
> > > >
> > > > Best,
> > > > The HBase Dev Team
> > > >
> > >
> >
>
Re: Running HBase on Hadoop 3.1, was Re: [ANNOUNCE] Apache HBase
2.1.9 is now available for download
Posted by Mich Talebzadeh <mi...@gmail.com>.
Thanks Nick.
My bad. One of the nodes holding region server did not have Snappy package
installed.
yum install snappy snappy-devel did the trick
Regards,
Dr Mich Talebzadeh
LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
<https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
http://talebzadehmich.wordpress.com
*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.
On Fri, 21 Feb 2020 at 21:36, Nick Dimiduk <nd...@apache.org> wrote:
> Hi Mich,
>
> I'm glad you're able to make progress. It sounds like your native libraries
> are not installed or activated correctly. It's possible they're not drop-in
> compatible between Hadoop2 and Hadoop3, in which case recompilation might
> be necessary. I'm not up-to-current on the state of those affairs (though
> potentially someone else reading here is...)
>
> Have you looked through this section of our book?
> http://hbase.apache.org/book.html#hadoop.native.lib
>
> Bonne chance,
> Nick
>
> On Thu, Feb 20, 2020 at 11:45 AM Mich Talebzadeh <
> mich.talebzadeh@gmail.com>
> wrote:
>
> > Thanks Nick,
> >
> > Looking at this matrix <http://hbase.apache.org/book.html#hadoop>, I
> > upgraded Hadoop from 3.1 to 3.1.1. This should allow me to used Hbase
> 2.1.8
> >
> > So I did both. One needs to change few hbase-site.xml setting etc to make
> > it work. Also the addition of htrace-core-3.1.0-incubating.jar to
> > $HBASE_HOME/lib.
> >
> > So both the HMaster and HRegionServer are running on master node
> > and HRegionServer on another node.
> >
> > The drawback is that one cannot use snappy compression in create table
> > command, something like below
> >
> > create 'trading:MARKETDATAHBASESPEED', {NAME=> 'PRICE_INFO', COMPRESSION
> > => 'SNAPPY'}
> >
> > This is the error I get on HRegionServer log
> >
> > org.apache.hadoop.hbase.DoNotRetryIOException: Compression algorithm
> > 'snappy' previously failed test.
> >
> > Eventually table is sort of created but mot complete. Also getting rid of
> > this table becomes problematic
> >
> > hbase(main):002:0> drop 'trading:MARKETDATAHBASESPEED'
> > ERROR: The procedure 31 is still running
> > For usage try 'help "drop"'
> > Took 668.3693 seconds
> > hbase(main):003:0> drop 'trading:MARKETDATAHBASESPEED'
> > ERROR: The procedure 32 is still running
> > For usage try 'help "drop"'
> > Took 668.2910 seconds
> >
> >
> > So the table cannot be dropped.
> >
> > All fun and games
> >
> > Cheers,
> >
> > Dr Mich Talebzadeh
> >
> >
> >
> > LinkedIn *
> >
> https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
> > <
> >
> https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
> > >*
> >
> >
> >
> > http://talebzadehmich.wordpress.com
> >
> >
> > *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> > loss, damage or destruction of data or any other property which may arise
> > from relying on this email's technical content is explicitly disclaimed.
> > The author will in no case be liable for any monetary damages arising
> from
> > such loss, damage or destruction.
> >
> >
> >
> >
> > On Thu, 20 Feb 2020 at 18:54, Nick Dimiduk <nd...@apache.org> wrote:
> >
> > > changed subject, dropped dev
> > >
> > > Hello Mich,
> > >
> > > The HBase binary distribution includes all Hadoop client jars necessary
> > for
> > > HBase to function on top of HDFS. The version of those Hadoop jars is
> > that
> > > of Hadoop 2.8.5. Duo is saying that the Hadoop 2.8.5 client works
> > against a
> > > HDFS 3.1.x cluster. Thus, this binary release of HBase is expected to
> > work
> > > with HDFS 3.1.
> > >
> > > Alternatively, you can choose to recompile HBase using Hadoop 3.1.x as
> > the
> > > dependency. In that case, you'll need to retrieve the source,
> establish a
> > > build environment, and perform the build yourself -- the HBase project
> > does
> > > not distribute a binary artifact compiled against Hadoop 3.
> > >
> > > Thanks,
> > > Nick
> > >
> > > On Thu, Feb 20, 2020 at 3:20 AM Mich Talebzadeh <
> > mich.talebzadeh@gmail.com
> > > >
> > > wrote:
> > >
> > > > Thank you.
> > > >
> > > > In your points below
> > > >
> > > > If you want to run HBase on top of a 3.1.x HDFS cluster, you can just
> > use
> > > > the binaries, hadoop 2.8.5 client can communicate with 3.1.x server.
> > > >
> > > > If you want HBase to use hadoop 3.1.x, then you need to build the
> > > binaries
> > > > by yourself, use the hadoop 3 profile.
> > > >
> > > > Could you please clarify (since Hbase can work with Hadoop), which
> > > version
> > > > of new release has been tested against which version of Hadoop.
> > > >
> > > > Also it is not very clear which binaries you are referring to?
> "Hadoop
> > > > 2.8.5 client can communicate with "Hadoop" 3.1 server. So basically
> > > > download Hadoop 2.8.5 client for Hbase use?
> > > >
> > > >
> > > > Regards,
> > > >
> > > >
> > > >
> > > > Dr Mich Talebzadeh
> > > >
> > > >
> > > >
> > > > LinkedIn *
> > > >
> > >
> >
> https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
> > > > <
> > > >
> > >
> >
> https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
> > > > >*
> > > >
> > > >
> > > >
> > > > http://talebzadehmich.wordpress.com
> > > >
> > > >
> > > > *Disclaimer:* Use it at your own risk. Any and all responsibility for
> > any
> > > > loss, damage or destruction of data or any other property which may
> > arise
> > > > from relying on this email's technical content is explicitly
> > disclaimed.
> > > > The author will in no case be liable for any monetary damages arising
> > > from
> > > > such loss, damage or destruction.
> > > >
> > > >
> > > >
> > > >
> > > > On Thu, 20 Feb 2020 at 09:45, 张铎(Duo Zhang) <pa...@gmail.com>
> > > wrote:
> > > >
> > > > > If you want to run HBase on top of a 3.1.x HDFS cluster, you can
> just
> > > use
> > > > > the binaries, hadoop 2.8.5 client can communicate with 3.1.x
> server.
> > > > >
> > > > > If you want HBase to use hadoop 3.1.x, then you need to build the
> > > > binaries
> > > > > by yourself, use the hadoop 3 profile.
> > > > >
> > > > > Mich Talebzadeh <mi...@gmail.com> 于2020年2月20日周四
> 下午5:15写道：
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > Thanks.
> > > > > >
> > > > > > Does this version of Hbase work with Hadoop 3.1? I am still stuck
> > > with
> > > > > > Hbase 1.2.7
> > > > > >
> > > > > > Hadoop 3.1.0
> > > > > > Source code repository https://github.com/apache/hadoop -r
> > > > > > 16b70619a24cdcf5d3b0fcf4b58ca77238ccbe6d
> > > > > > Compiled by centos on 2018-03-30T00:00Z
> > > > > > Compiled with protoc 2.5.0
> > > > > >
> > > > > > Regards,
> > > > > >
> > > > > > Dr Mich Talebzadeh
> > > > > >
> > > > > >
> > > > > >
> > > > > > LinkedIn *
> > > > > >
> > > > >
> > > >
> > >
> >
> https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
> > > > > > <
> > > > > >
> > > > >
> > > >
> > >
> >
> https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
> > > > > > >*
> > > > > >
> > > > > >
> > > > > >
> > > > > > http://talebzadehmich.wordpress.com
> > > > > >
> > > > > >
> > > > > > *Disclaimer:* Use it at your own risk. Any and all responsibility
> > for
> > > > any
> > > > > > loss, damage or destruction of data or any other property which
> may
> > > > arise
> > > > > > from relying on this email's technical content is explicitly
> > > > disclaimed.
> > > > > > The author will in no case be liable for any monetary damages
> > arising
> > > > > from
> > > > > > such loss, damage or destruction.
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Thu, 20 Feb 2020 at 08:48, Duo Zhang <zh...@apache.org>
> > wrote:
> > > > > >
> > > > > > > The HBase team is happy to announce the immediate availability
> of
> > > > > Apache
> > > > > > > HBase 2.1.9.
> > > > > > >
> > > > > > > Download from https://hbase.apache.org/downloads.html
> > > > > > >
> > > > > > > Apache HBase is an open-source, distributed, versioned,
> > > > non-relational
> > > > > > > database. Apache HBase gives you low latency random access to
> > > > billions
> > > > > of
> > > > > > > rows with millions of columns atop non-specialized hardware. To
> > > learn
> > > > > > more
> > > > > > > about HBase, see https://hbase.apache.org/.
> > > > > > >
> > > > > > > HBase 2.1.9 is the latest release of the HBase 2.1 line,
> > continuing
> > > > on
> > > > > > the
> > > > > > > theme of bringing a stable, reliable database to the Apache Big
> > > Data
> > > > > > > ecosystem and beyond. 2.1.9 includes ~62 bug and improvement
> > fixes
> > > > done
> > > > > > > since the 2.1.8.
> > > > > > >
> > > > > > > For instructions on verifying ASF release downloads, please see
> > > > > > >
> > > > > > > https://www.apache.org/dyn/closer.cgi#verify
> > > > > > >
> > > > > > > Project member signature keys can be found at
> > > > > > >
> > > > > > > https://www.apache.org/dist/hbase/KEYS
> > > > > > >
> > > > > > > Thanks to all the contributors who made this release possible!
> > > > > > >
> > > > > > > Best,
> > > > > > > The HBase Dev Team
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: Running HBase on Hadoop 3.1, was Re: [ANNOUNCE] Apache HBase
2.1.9 is now available for download
Posted by Nick Dimiduk <nd...@apache.org>.
Hi Mich,
I'm glad you're able to make progress. It sounds like your native libraries
are not installed or activated correctly. It's possible they're not drop-in
compatible between Hadoop2 and Hadoop3, in which case recompilation might
be necessary. I'm not up-to-current on the state of those affairs (though
potentially someone else reading here is...)
Have you looked through this section of our book?
http://hbase.apache.org/book.html#hadoop.native.lib
Bonne chance,
Nick
On Thu, Feb 20, 2020 at 11:45 AM Mich Talebzadeh <mi...@gmail.com>
wrote:
> Thanks Nick,
>
> Looking at this matrix <http://hbase.apache.org/book.html#hadoop>, I
> upgraded Hadoop from 3.1 to 3.1.1. This should allow me to used Hbase 2.1.8
>
> So I did both. One needs to change few hbase-site.xml setting etc to make
> it work. Also the addition of htrace-core-3.1.0-incubating.jar to
> $HBASE_HOME/lib.
>
> So both the HMaster and HRegionServer are running on master node
> and HRegionServer on another node.
>
> The drawback is that one cannot use snappy compression in create table
> command, something like below
>
> create 'trading:MARKETDATAHBASESPEED', {NAME=> 'PRICE_INFO', COMPRESSION
> => 'SNAPPY'}
>
> This is the error I get on HRegionServer log
>
> org.apache.hadoop.hbase.DoNotRetryIOException: Compression algorithm
> 'snappy' previously failed test.
>
> Eventually table is sort of created but mot complete. Also getting rid of
> this table becomes problematic
>
> hbase(main):002:0> drop 'trading:MARKETDATAHBASESPEED'
> ERROR: The procedure 31 is still running
> For usage try 'help "drop"'
> Took 668.3693 seconds
> hbase(main):003:0> drop 'trading:MARKETDATAHBASESPEED'
> ERROR: The procedure 32 is still running
> For usage try 'help "drop"'
> Took 668.2910 seconds
>
>
> So the table cannot be dropped.
>
> All fun and games
>
> Cheers,
>
> Dr Mich Talebzadeh
>
>
>
> LinkedIn *
> https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
> <
> https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
> >*
>
>
>
> http://talebzadehmich.wordpress.com
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
>
> On Thu, 20 Feb 2020 at 18:54, Nick Dimiduk <nd...@apache.org> wrote:
>
> > changed subject, dropped dev
> >
> > Hello Mich,
> >
> > The HBase binary distribution includes all Hadoop client jars necessary
> for
> > HBase to function on top of HDFS. The version of those Hadoop jars is
> that
> > of Hadoop 2.8.5. Duo is saying that the Hadoop 2.8.5 client works
> against a
> > HDFS 3.1.x cluster. Thus, this binary release of HBase is expected to
> work
> > with HDFS 3.1.
> >
> > Alternatively, you can choose to recompile HBase using Hadoop 3.1.x as
> the
> > dependency. In that case, you'll need to retrieve the source, establish a
> > build environment, and perform the build yourself -- the HBase project
> does
> > not distribute a binary artifact compiled against Hadoop 3.
> >
> > Thanks,
> > Nick
> >
> > On Thu, Feb 20, 2020 at 3:20 AM Mich Talebzadeh <
> mich.talebzadeh@gmail.com
> > >
> > wrote:
> >
> > > Thank you.
> > >
> > > In your points below
> > >
> > > If you want to run HBase on top of a 3.1.x HDFS cluster, you can just
> use
> > > the binaries, hadoop 2.8.5 client can communicate with 3.1.x server.
> > >
> > > If you want HBase to use hadoop 3.1.x, then you need to build the
> > binaries
> > > by yourself, use the hadoop 3 profile.
> > >
> > > Could you please clarify (since Hbase can work with Hadoop), which
> > version
> > > of new release has been tested against which version of Hadoop.
> > >
> > > Also it is not very clear which binaries you are referring to? "Hadoop
> > > 2.8.5 client can communicate with "Hadoop" 3.1 server. So basically
> > > download Hadoop 2.8.5 client for Hbase use?
> > >
> > >
> > > Regards,
> > >
> > >
> > >
> > > Dr Mich Talebzadeh
> > >
> > >
> > >
> > > LinkedIn *
> > >
> >
> https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
> > > <
> > >
> >
> https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
> > > >*
> > >
> > >
> > >
> > > http://talebzadehmich.wordpress.com
> > >
> > >
> > > *Disclaimer:* Use it at your own risk. Any and all responsibility for
> any
> > > loss, damage or destruction of data or any other property which may
> arise
> > > from relying on this email's technical content is explicitly
> disclaimed.
> > > The author will in no case be liable for any monetary damages arising
> > from
> > > such loss, damage or destruction.
> > >
> > >
> > >
> > >
> > > On Thu, 20 Feb 2020 at 09:45, 张铎(Duo Zhang) <pa...@gmail.com>
> > wrote:
> > >
> > > > If you want to run HBase on top of a 3.1.x HDFS cluster, you can just
> > use
> > > > the binaries, hadoop 2.8.5 client can communicate with 3.1.x server.
> > > >
> > > > If you want HBase to use hadoop 3.1.x, then you need to build the
> > > binaries
> > > > by yourself, use the hadoop 3 profile.
> > > >
> > > > Mich Talebzadeh <mi...@gmail.com> 于2020年2月20日周四 下午5:15写道：
> > > >
> > > > > Hi,
> > > > >
> > > > > Thanks.
> > > > >
> > > > > Does this version of Hbase work with Hadoop 3.1? I am still stuck
> > with
> > > > > Hbase 1.2.7
> > > > >
> > > > > Hadoop 3.1.0
> > > > > Source code repository https://github.com/apache/hadoop -r
> > > > > 16b70619a24cdcf5d3b0fcf4b58ca77238ccbe6d
> > > > > Compiled by centos on 2018-03-30T00:00Z
> > > > > Compiled with protoc 2.5.0
> > > > >
> > > > > Regards,
> > > > >
> > > > > Dr Mich Talebzadeh
> > > > >
> > > > >
> > > > >
> > > > > LinkedIn *
> > > > >
> > > >
> > >
> >
> https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
> > > > > <
> > > > >
> > > >
> > >
> >
> https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
> > > > > >*
> > > > >
> > > > >
> > > > >
> > > > > http://talebzadehmich.wordpress.com
> > > > >
> > > > >
> > > > > *Disclaimer:* Use it at your own risk. Any and all responsibility
> for
> > > any
> > > > > loss, damage or destruction of data or any other property which may
> > > arise
> > > > > from relying on this email's technical content is explicitly
> > > disclaimed.
> > > > > The author will in no case be liable for any monetary damages
> arising
> > > > from
> > > > > such loss, damage or destruction.
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > On Thu, 20 Feb 2020 at 08:48, Duo Zhang <zh...@apache.org>
> wrote:
> > > > >
> > > > > > The HBase team is happy to announce the immediate availability of
> > > > Apache
> > > > > > HBase 2.1.9.
> > > > > >
> > > > > > Download from https://hbase.apache.org/downloads.html
> > > > > >
> > > > > > Apache HBase is an open-source, distributed, versioned,
> > > non-relational
> > > > > > database. Apache HBase gives you low latency random access to
> > > billions
> > > > of
> > > > > > rows with millions of columns atop non-specialized hardware. To
> > learn
> > > > > more
> > > > > > about HBase, see https://hbase.apache.org/.
> > > > > >
> > > > > > HBase 2.1.9 is the latest release of the HBase 2.1 line,
> continuing
> > > on
> > > > > the
> > > > > > theme of bringing a stable, reliable database to the Apache Big
> > Data
> > > > > > ecosystem and beyond. 2.1.9 includes ~62 bug and improvement
> fixes
> > > done
> > > > > > since the 2.1.8.
> > > > > >
> > > > > > For instructions on verifying ASF release downloads, please see
> > > > > >
> > > > > > https://www.apache.org/dyn/closer.cgi#verify
> > > > > >
> > > > > > Project member signature keys can be found at
> > > > > >
> > > > > > https://www.apache.org/dist/hbase/KEYS
> > > > > >
> > > > > > Thanks to all the contributors who made this release possible!
> > > > > >
> > > > > > Best,
> > > > > > The HBase Dev Team
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: Running HBase on Hadoop 3.1, was Re: [ANNOUNCE] Apache HBase
2.1.9 is now available for download
Posted by Mich Talebzadeh <mi...@gmail.com>.
Thanks Nick,
Looking at this matrix <http://hbase.apache.org/book.html#hadoop>, I
upgraded Hadoop from 3.1 to 3.1.1. This should allow me to used Hbase 2.1.8
So I did both. One needs to change few hbase-site.xml setting etc to make
it work. Also the addition of htrace-core-3.1.0-incubating.jar to
$HBASE_HOME/lib.
So both the HMaster and HRegionServer are running on master node
and HRegionServer on another node.
The drawback is that one cannot use snappy compression in create table
command, something like below
create 'trading:MARKETDATAHBASESPEED', {NAME=> 'PRICE_INFO', COMPRESSION
=> 'SNAPPY'}
This is the error I get on HRegionServer log
org.apache.hadoop.hbase.DoNotRetryIOException: Compression algorithm
'snappy' previously failed test.
Eventually table is sort of created but mot complete. Also getting rid of
this table becomes problematic
hbase(main):002:0> drop 'trading:MARKETDATAHBASESPEED'
ERROR: The procedure 31 is still running
For usage try 'help "drop"'
Took 668.3693 seconds
hbase(main):003:0> drop 'trading:MARKETDATAHBASESPEED'
ERROR: The procedure 32 is still running
For usage try 'help "drop"'
Took 668.2910 seconds
So the table cannot be dropped.
All fun and games
Cheers,
Dr Mich Talebzadeh
LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
<https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
http://talebzadehmich.wordpress.com
*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.
On Thu, 20 Feb 2020 at 18:54, Nick Dimiduk <nd...@apache.org> wrote:
> changed subject, dropped dev
>
> Hello Mich,
>
> The HBase binary distribution includes all Hadoop client jars necessary for
> HBase to function on top of HDFS. The version of those Hadoop jars is that
> of Hadoop 2.8.5. Duo is saying that the Hadoop 2.8.5 client works against a
> HDFS 3.1.x cluster. Thus, this binary release of HBase is expected to work
> with HDFS 3.1.
>
> Alternatively, you can choose to recompile HBase using Hadoop 3.1.x as the
> dependency. In that case, you'll need to retrieve the source, establish a
> build environment, and perform the build yourself -- the HBase project does
> not distribute a binary artifact compiled against Hadoop 3.
>
> Thanks,
> Nick
>
> On Thu, Feb 20, 2020 at 3:20 AM Mich Talebzadeh <mich.talebzadeh@gmail.com
> >
> wrote:
>
> > Thank you.
> >
> > In your points below
> >
> > If you want to run HBase on top of a 3.1.x HDFS cluster, you can just use
> > the binaries, hadoop 2.8.5 client can communicate with 3.1.x server.
> >
> > If you want HBase to use hadoop 3.1.x, then you need to build the
> binaries
> > by yourself, use the hadoop 3 profile.
> >
> > Could you please clarify (since Hbase can work with Hadoop), which
> version
> > of new release has been tested against which version of Hadoop.
> >
> > Also it is not very clear which binaries you are referring to? "Hadoop
> > 2.8.5 client can communicate with "Hadoop" 3.1 server. So basically
> > download Hadoop 2.8.5 client for Hbase use?
> >
> >
> > Regards,
> >
> >
> >
> > Dr Mich Talebzadeh
> >
> >
> >
> > LinkedIn *
> >
> https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
> > <
> >
> https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
> > >*
> >
> >
> >
> > http://talebzadehmich.wordpress.com
> >
> >
> > *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> > loss, damage or destruction of data or any other property which may arise
> > from relying on this email's technical content is explicitly disclaimed.
> > The author will in no case be liable for any monetary damages arising
> from
> > such loss, damage or destruction.
> >
> >
> >
> >
> > On Thu, 20 Feb 2020 at 09:45, 张铎(Duo Zhang) <pa...@gmail.com>
> wrote:
> >
> > > If you want to run HBase on top of a 3.1.x HDFS cluster, you can just
> use
> > > the binaries, hadoop 2.8.5 client can communicate with 3.1.x server.
> > >
> > > If you want HBase to use hadoop 3.1.x, then you need to build the
> > binaries
> > > by yourself, use the hadoop 3 profile.
> > >
> > > Mich Talebzadeh <mi...@gmail.com> 于2020年2月20日周四 下午5:15写道：
> > >
> > > > Hi,
> > > >
> > > > Thanks.
> > > >
> > > > Does this version of Hbase work with Hadoop 3.1? I am still stuck
> with
> > > > Hbase 1.2.7
> > > >
> > > > Hadoop 3.1.0
> > > > Source code repository https://github.com/apache/hadoop -r
> > > > 16b70619a24cdcf5d3b0fcf4b58ca77238ccbe6d
> > > > Compiled by centos on 2018-03-30T00:00Z
> > > > Compiled with protoc 2.5.0
> > > >
> > > > Regards,
> > > >
> > > > Dr Mich Talebzadeh
> > > >
> > > >
> > > >
> > > > LinkedIn *
> > > >
> > >
> >
> https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
> > > > <
> > > >
> > >
> >
> https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
> > > > >*
> > > >
> > > >
> > > >
> > > > http://talebzadehmich.wordpress.com
> > > >
> > > >
> > > > *Disclaimer:* Use it at your own risk. Any and all responsibility for
> > any
> > > > loss, damage or destruction of data or any other property which may
> > arise
> > > > from relying on this email's technical content is explicitly
> > disclaimed.
> > > > The author will in no case be liable for any monetary damages arising
> > > from
> > > > such loss, damage or destruction.
> > > >
> > > >
> > > >
> > > >
> > > > On Thu, 20 Feb 2020 at 08:48, Duo Zhang <zh...@apache.org> wrote:
> > > >
> > > > > The HBase team is happy to announce the immediate availability of
> > > Apache
> > > > > HBase 2.1.9.
> > > > >
> > > > > Download from https://hbase.apache.org/downloads.html
> > > > >
> > > > > Apache HBase is an open-source, distributed, versioned,
> > non-relational
> > > > > database. Apache HBase gives you low latency random access to
> > billions
> > > of
> > > > > rows with millions of columns atop non-specialized hardware. To
> learn
> > > > more
> > > > > about HBase, see https://hbase.apache.org/.
> > > > >
> > > > > HBase 2.1.9 is the latest release of the HBase 2.1 line, continuing
> > on
> > > > the
> > > > > theme of bringing a stable, reliable database to the Apache Big
> Data
> > > > > ecosystem and beyond. 2.1.9 includes ~62 bug and improvement fixes
> > done
> > > > > since the 2.1.8.
> > > > >
> > > > > For instructions on verifying ASF release downloads, please see
> > > > >
> > > > > https://www.apache.org/dyn/closer.cgi#verify
> > > > >
> > > > > Project member signature keys can be found at
> > > > >
> > > > > https://www.apache.org/dist/hbase/KEYS
> > > > >
> > > > > Thanks to all the contributors who made this release possible!
> > > > >
> > > > > Best,
> > > > > The HBase Dev Team
> > > > >
> > > >
> > >
> >
>