You are viewing a plain text version of this content. The canonical link for it is here.

Posted to notifications@kyuubi.apache.org by GitBox <gi...@apache.org> on 2022/08/18 09:22:32 UTC

[GitHub] [incubator-kyuubi] pan3793 commented on issue #3259: [FEATURE] Support Hive V2 DataSource in Kyuubi

pan3793 commented on issue #3259:
URL: https://github.com/apache/incubator-kyuubi/issues/3259#issuecomment-1219243969

   Based on @yikf 's comments and given it's a future-proof component, I think we can make it support Spark 3.3+ only. 
   For example, the currently Spark Hive writer has a long-standing issue, it generates too many small files. `RequiresDistributionAndOrdering` has been introduced in SPARK-33779 since Spark 3.2, and got enhanced in SPARK-37523 which will be available in Spark 3.4, then we can leverage it to solve the small file issue if we don't support 3.1 and early versions.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org