[jira] [Work started] (HBASE-30178) Add support to spark 4 in hbase-connectors

["Wellington Chevreuil (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HBASE-30178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Work on HBASE-30178 started by Wellington Chevreuil.
----------------------------------------------------
> Add support to spark 4 in hbase-connectors
> ------------------------------------------
>
>                 Key: HBASE-30178
>                 URL: https://issues.apache.org/jira/browse/HBASE-30178
>             Project: HBase
>          Issue Type: Umbrella
>          Components: hbase-connectors
>            Reporter: Wellington Chevreuil
>            Assignee: Wellington Chevreuil
>            Priority: Major
>
> With spark 4.0.0 GA for around a year now, we should start working on making hbase-connectors integrate with spark 4.
> Since we would still need to support spark 3 for a while and due to spark 4 requiring minimum jdk17 and scala 2.13, it would be easier to define a specific spark 4 module under the hbase-connectors root, so that spark.version, scala.version, compiler flags, and optional spark-4 only APIs never collide with spark 3 in a single parent pom. 
> We should keep feature parity as much as possible. Datasources, context, rdd and connection cache related logic can be duplicated under this new hbase-spark4 module, in order to compile and package with jdk17 and scala 2.13. This module build should be isolated on a specific maven profile, so it wouldn't be build by default. 
> Pushdown filter related classes and filter protocol would remain on a common module (no code duplication needed).
>  
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)