Questions for HAWQ dev community: Pluggable storage formats and files systems vs. PXF profiles

[Ed Espino <es...@apache.org>]

As HAWQ has matured, the introduction of the Pluggable storage formats is
giving users two methods to access external data. Is the project at a point
where we can advise the community that we will recommend the use of
the Pluggable
storage format instead of PXF? I believe this was discussed in a thread
earlier in the year but I haven't seen any further discussion or action
plan of action for it.

I suggest we recommend HAWQ community users start the migration and upon
graduation the project should indicate we will stop supporting PXF in favor
of the Pluggable storage formats. Thoughts? Depending on feedback, I would
like to put this to a vote for the dev community's consideration.

email ref:
https://lists.apache.org/thread.html/3a1e5c358bc0b0ecf04d49e38979d6bf0ae21e3a56188d34986fc8a5@%3Cdev.hawq.apache.org%3E

For reference, PXF is also in use and under active development in another
open source project: Greenplum Database (
https://github.com/greenplum-db/gpdb/tree/5X_STABLE/gpAux/extensions/pxf )
and depends on the PXF server-side components that exist in HAWQ’s source
tree.

Thanks,
-=e

Re: Questions for HAWQ dev community: Pluggable storage formats and files systems vs. PXF profiles

[Lei Chang <ch...@gmail.com>]

Hi Ed,

Thanks for bringing this out. The framework part of the pluggable storage
format has been committed, but the plugins still need some development work
and take some time.

Thanks
Lei








On Wed, May 30, 2018 at 11:35 PM, Ed Espino <es...@apache.org> wrote:

> As HAWQ has matured, the introduction of the Pluggable storage formats is
> giving users two methods to access external data. Is the project at a point
> where we can advise the community that we will recommend the use of the Pluggable
> storage format instead of PXF? I believe this was discussed in a thread
> earlier in the year but I haven't seen any further discussion or action
> plan of action for it.
>
> I suggest we recommend HAWQ community users start the migration and upon
> graduation the project should indicate we will stop supporting PXF in favor
> of the Pluggable storage formats. Thoughts? Depending on feedback, I would
> like to put this to a vote for the dev community's consideration.
>
> email ref: https://lists.apache.org/thread.html/
> 3a1e5c358bc0b0ecf04d49e38979d6bf0ae21e3a56188d34986fc8a5@%
> 3Cdev.hawq.apache.org%3E
>
> For reference, PXF is also in use and under active development in another
> open source project: Greenplum Database ( https://github.com/greenplum-
> db/gpdb/tree/5X_STABLE/gpAux/extensions/pxf ) and depends on the PXF
> server-side components that exist in HAWQ’s source tree.
>
> Thanks,
> -=e
>
>