Arrow and HBase

[Matt Duffy <ma...@gmail.com>]

I've seen several presentations listing HBase as a technology that
integrates with Arrow dataframes.  Is there an example for reading/writing
to HBase from an Arrow dataframe?  I'm hoping to utilize Arrow to expedite
the translation between HBase and Spark or Pandas.

Re: Arrow and HBase

[Micah Kornfield <em...@gmail.com>]

Hi Matt,
I'm not sure which presentations you are referring to, but if this is in
reference to the diagram on the Overview page [1].  That is aspirational
there are some integrations in various levels of depth for some of the
technologies but it isn't the final state.

To my knowledge there isn't a library within the Arrow project itself that
currently connects to HBase and does any sort of HBase schema to Arrow
schema conversion, but i think there might be some interest in building one
at some point.

There was a thread a little while ago on using Arrow within Redis [2] which
might be of interest depending on how you are using HBase.

Hope this helps.

-Micah

[1] https://arrow.apache.org/overview/
[2]
https://mail-archives.apache.org/mod_mbox/arrow-user/202007.mbox/%3CCAKg4KDy3uji7dsZiLjRQhs%3DTfnBP%2BJanbdr-LFtNnBZftHKsdw%40mail.gmail.com%3E

On Tue, Oct 6, 2020 at 8:21 AM Matt Duffy <ma...@gmail.com> wrote:

> I've seen several presentations listing HBase as a technology that
> integrates with Arrow dataframes.  Is there an example for reading/writing
> to HBase from an Arrow dataframe?  I'm hoping to utilize Arrow to expedite
> the translation between HBase and Spark or Pandas.
>