Serialization/deserialization mechanism unification in HAWQ

["Alex (Oleksandr) Diachenko" <od...@pivotal.io>]

Hi,

I have noticed that we have multiple places where we
serialize/deserialize data to/from a string.

Couple examples:

   - Filters strings in pxffilters.c.
   - Information about data fragments in hd_work_mgr.c, pxfuriparser.c.

I was thinking to use some library(like protobuf-c or similar) instead of
having plenty of boilerplate code.
Any thoughts?

Regards, Alex.

Re: Serialization/deserialization mechanism unification in HAWQ

[Hong Wu <xu...@gmail.com>]

Hi Alex,

It is reasonable to unify the serialization/deserialization process inside
HAWQ. We need additional effort to do that refactor and further evaluation.

Using protobuf-c or thrift library is one choice, but I think they are
using IDL to solve serialization/deserialization problem between different
languages. In other words, we need to do compile IDL to C code before
compiling HAWQ. This is not good. For HAWQ,  we just need a
unified protocol to do serialization/deserialization between C and
JAVA(at pxf end) codes.

Another choice is using an existing protocol to do
serialization/deserialization.

Best,
Hong

2016-12-09 8:46 GMT+08:00 Alex (Oleksandr) Diachenko <od...@pivotal.io>
:

> Hi,
>
> I have noticed that we have multiple places where we
> serialize/deserialize data to/from a string.
>
> Couple examples:
>
>    - Filters strings in pxffilters.c.
>    - Information about data fragments in hd_work_mgr.c, pxfuriparser.c.
>
> I was thinking to use some library(like protobuf-c or similar) instead of
> having plenty of boilerplate code.
> Any thoughts?
>
> Regards, Alex.
>