[GitHub] [pinot] mqliang commented on pull request #7167: Bump up data table version to V4 to optimize space usage.

[GitBox <gi...@apache.org>]

mqliang commented on pull request #7167:
URL: https://github.com/apache/pinot/pull/7167#issuecomment-994099004


   @mayankshriv @dharakk
   
   Should we keep back-comp of DataTable at interface level or at binary level (the bytes layout when transfer on wire)? 
   
   * If presto connector import pinot code as a package to build a DataTable or to parse a bytes as a DataTable, then we only only need interface level back-comp. Then we will be fine -- this change does not change the DataTable interface. The presto connector just need to bump up its pinot dependency version. 
   * If presto connector manipulate DataTable bytes by itself, we then need binary level back-comp. The the situation will becomes much more complex -- we need a protocol:
        - between pinot-server and pinot-broker
        - between pinot-server and presto-connector
        - between pinot-broker and presto-connector
   
   
   @xiangfu0 
   >  Is it possible to make the query itself contain the version of which DataTable protocol to use?
   So the Pinot server can work with v3 broker/v4 broker or v2 presto.
   
   Instead of specifying the version for each query, we can change the server config of `pinot.server.instance.currentDataTableVersion`. In this way, new server will continue to return old version DataTable, old broker/presto-connector can continue work. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org