[jira] [Comment Edited] (HAWQ-583) Extend PXF to allow plugins to support returning partial content of SELECT(column projection)

["Shivram Mani (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HAWQ-583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15398007#comment-15398007 ] 

Shivram Mani edited comment on HAWQ-583 at 7/28/16 7:10 PM:
------------------------------------------------------------

Refer to HAWQ-927 for the http header params introduced to support column projection.
If the table has 3 columns col1, col2, col3 the following table describes the PXF response corresponding to various types of query patterns

||Query||X-GP-ATTRS-PROJ||X-GP-ATTRS-PROJ-IDX||PXF Action||
| select col1 |1|0|Project col1 data with NULL values padded in col2 and col3|
| select col1,col2 |2|0, 1|Project col1 and col2 data with NULL values in col3|
| select count(col2) |1|1|Project col2 with NULL values in col1 and col3|
| select count ( * ) |0|NULL|Project only one column. We will pick col1 for consistency purpose|
| select * |NULL|NULL|Project all columns|
| select col1,* |4|0,0,1,2|Project col1,col2 and col3|


was (Author: shivram):
Refer to HAWQ-927 for the http header params introduced to support column projection.
If the table has 3 columns col1, col2, col3 the following table describes the PXF response corresponding to various types of query patterns

||Query||X-GP-ATTRS-PROJ||X-GP-ATTRS-PROJ-IDX||PXF Action||
| select col1 |1|0|Project col1 data with NULL values padded in col2 and col3|
| select col1,col2 |2|0, 1|Project col1 and col2 data with NULL values in col3|
| select count(col2) |1|1|Project col2 with NULL values in col1 and col3|
| select count(*) |0|NULL|Project only one column. We will pick col1 for consistency purpose|
| select * |NULL|NULL|Project all columns|
| select col1,* |4|0,0,1,2|Project col1,col2 and col3|

> Extend PXF to allow plugins to support returning partial content of SELECT(column projection)
> ---------------------------------------------------------------------------------------------
>
>                 Key: HAWQ-583
>                 URL: https://issues.apache.org/jira/browse/HAWQ-583
>             Project: Apache HAWQ
>          Issue Type: New Feature
>          Components: PXF
>            Reporter: Michael Andre Pearce (IG)
>            Assignee: Kavinder Dhaliwal
>             Fix For: backlog
>
>
> Currently PXF supports being able to push down the predicate WHERE logic to the external system to reduce the amount data needed to be retrieved.
> SELECT a, b FROM external_pxf_source WHERE z < 3 AND x > 6
> As such we can filter the rows returned, but currently still would have to return all the fields / complete row.
> This proposal is so that we can return only the columns in SELECT part.
> For data sources where it is columnar storage or selectable such as remote database that PXF can read or connect to this has advantages in the data that needs to be accessed or even transferred.
> As like with the push down Filter it should be optional so that plugins that provide support can use it but others that do not, continue to work as they do.
> The proposal would be for
> 1) create an interface created for plugins to optionally implement, where the columns needed to be returned are given to the plugin.
> 2) update pxf api for hawq to send columns defined in SELECT, for pxf to invoke the plugin interface and pass this information onto if provided
> 3) update pxf integration within hawq itself so that hawq passes this additonal  information to pxf.
> This Ticket is off the back of discussion on HAWQ-492.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)