You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Bryan Beaudreault <bb...@hubspot.com.INVALID> on 2021/08/20 22:14:05 UTC

hbase.pb package breaks protobuf compatibility between pre-1.3 and post

It's been decided in a thread on the dev list that protobuf should be
considered InterfaceAudience.PRIVATE. See here:
https://lists.apache.org/thread.html/r9e6eb11106727d245f6eb2a5023823901637971d6ed0f0aedaf8d149%40%3Cdev.hbase.apache.org%3E

So with that being said, this is mostly just to put out there what I've
encountered in case others come across the same.

I'm working on an upgrade from cdh5 (hbase 1.2-ish) to hbase 2.4.4. We have
a few coprocessors and one step was to update them all to work with hbase2
based on the documented upgrade paths. This went mostly fine, except we
have endpoint coprocessors and filters which make use of HBase protobufs.
For example, we have a filter which accepts a CompareFilter proto as part
of its message. In another example, we used NameBytesPair in the response
from an endpoint.

When I tried deploying the updated coprocessors onto an hbase2 cluster, I
got some spectacular failures because the coprocessors could not load due
to DescriptorValidationException. As an example:

Caused by: com.google.protobuf.Descriptors$DescriptorValidationException:
HyperLogLogFilter.compare_filter: ".CompareFilter" is not defined.
        at
com.google.protobuf.Descriptors$DescriptorPool.lookupSymbol(Descriptors.java:1784)
        at
com.google.protobuf.Descriptors$FieldDescriptor.crossLink(Descriptors.java:982)

I believe the reason for this is that
https://issues.apache.org/jira/browse/HBASE-14077 added a package to the
hbase protobufs, "hbase.pb". As a result, our attempt to reference the bare
".Comparefilter" fails.

I think the fix for users, per the above discussion is to avoid using
internal hbase protobufs. In my case, thankfully these fields actually
weren't used in our current usages of these endpoints/filters. So I simply
removed the references.

If you must depend on hbase protobufs, I think you probably will need to
build 2 versions of your server-side endpoint/filter -- one for your older
cluster, and one for your newer. The old one would remain unchanged, while
the newer would prefix includes with "hbase.pb". The client side can
probably use the same as your older cluster since I believe it should be
wire compatible just not binary (I could be wrong that though, YMMV).

Hope this helps anyone who comes across it, and feel free to add other
ideas (though I'm no longer blocked by this personally due to being able to
delete the dependencies).