You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "HyukjinKwon (via GitHub)" <gi...@apache.org> on 2023/09/26 03:21:00 UTC
[GitHub] [spark] HyukjinKwon opened a new pull request, #43116: [SPARK-45328][SQL] Remove Hive support prior to 2.0.0
HyukjinKwon opened a new pull request, #43116:
URL: https://github.com/apache/spark/pull/43116
### What changes were proposed in this pull request?
This PR proposes to remove Hive support prior to 2.0.0 (`spark.sql.hive.metastore.version`).
### Why are the changes needed?
We dropped JDK 8 and 11, and Hive prior to 2.0.0 cannot work together. They are actually already the dead code.
### Does this PR introduce _any_ user-facing change?
Technically no, because this wouldn't already work.
### How was this patch tested?
Nope because there is no way to test them.
### Was this patch authored or co-authored using generative AI tooling?
No.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] pan3793 commented on a diff in pull request #43116: [SPARK-45328][SQL] Remove Hive support prior to 2.0.0
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on code in PR #43116:
URL: https://github.com/apache/spark/pull/43116#discussion_r1336969275
##########
sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala:
##########
@@ -115,12 +115,6 @@ private[hive] class HiveClientImpl(
private val outputBuffer = new CircularBuffer()
private val shim = version match {
- case hive.v12 => new Shim_v0_12()
- case hive.v13 => new Shim_v0_13()
- case hive.v14 => new Shim_v0_14()
- case hive.v1_0 => new Shim_v1_0()
- case hive.v1_1 => new Shim_v1_1()
- case hive.v1_2 => new Shim_v1_2()
case hive.v2_0 => new Shim_v2_0()
Review Comment:
> I also worry about downstram users/vendors who still need to use them, and would like to alleviate the overhead.
I think they could easily find them in the old versions or from the git history.
> Maybe let's do this in a separate PR.
I'd like to make such a PR if you don't oppose
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] pan3793 commented on a diff in pull request #43116: [SPARK-45328][SQL] Remove Hive support prior to 2.0.0
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on code in PR #43116:
URL: https://github.com/apache/spark/pull/43116#discussion_r1336612849
##########
sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala:
##########
@@ -115,12 +115,6 @@ private[hive] class HiveClientImpl(
private val outputBuffer = new CircularBuffer()
private val shim = version match {
- case hive.v12 => new Shim_v0_12()
- case hive.v13 => new Shim_v0_13()
- case hive.v14 => new Shim_v0_14()
- case hive.v1_0 => new Shim_v1_0()
- case hive.v1_1 => new Shim_v1_1()
- case hive.v1_2 => new Shim_v1_2()
case hive.v2_0 => new Shim_v2_0()
Review Comment:
seems the above `Shim_v*` classes definition could be removed and merged the code into `Shim_v2_0`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a diff in pull request #43116: [SPARK-45328][SQL] Remove Hive support prior to 2.0.0
Posted by "HyukjinKwon (via GitHub)" <gi...@apache.org>.
HyukjinKwon commented on code in PR #43116:
URL: https://github.com/apache/spark/pull/43116#discussion_r1336961705
##########
sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala:
##########
@@ -115,12 +115,6 @@ private[hive] class HiveClientImpl(
private val outputBuffer = new CircularBuffer()
private val shim = version match {
- case hive.v12 => new Shim_v0_12()
- case hive.v13 => new Shim_v0_13()
- case hive.v14 => new Shim_v0_14()
- case hive.v1_0 => new Shim_v1_0()
- case hive.v1_1 => new Shim_v1_1()
- case hive.v1_2 => new Shim_v1_2()
case hive.v2_0 => new Shim_v2_0()
Review Comment:
Those dead code isn't really needed but keeping them as are doesn't risk, and they are well organized according to versions.
I also worry about downstram users/vendors who still need to use them, and would like to alleviate the overhead.
Maybe let's do this in a separate PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] pan3793 commented on a diff in pull request #43116: [SPARK-45328][SQL] Remove Hive support prior to 2.0.0
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on code in PR #43116:
URL: https://github.com/apache/spark/pull/43116#discussion_r1336808388
##########
sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala:
##########
@@ -115,12 +115,6 @@ private[hive] class HiveClientImpl(
private val outputBuffer = new CircularBuffer()
private val shim = version match {
- case hive.v12 => new Shim_v0_12()
- case hive.v13 => new Shim_v0_13()
- case hive.v14 => new Shim_v0_14()
- case hive.v1_0 => new Shim_v1_0()
- case hive.v1_1 => new Shim_v1_1()
- case hive.v1_2 => new Shim_v1_2()
case hive.v2_0 => new Shim_v2_0()
Review Comment:
Emm.. what's the benefit of retaining unreachable code?
For example, all of `Shim_v0_12`, `Shim_v0_14` and `Shim_v2_0` define `loadPartition`, after this patch, the `loadPartition` defined in `Shim_v0_12`, `Shim_v0_14` still is dead code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] pan3793 commented on pull request #43116: [SPARK-45328][SQL] Remove Hive support prior to 2.0.0
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on PR #43116:
URL: https://github.com/apache/spark/pull/43116#issuecomment-1734820185
Should it be mentioned at `docs/sql-migration-guide.md`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #43116: [SPARK-45328][SQL] Remove Hive support prior to 2.0.0
Posted by "HyukjinKwon (via GitHub)" <gi...@apache.org>.
HyukjinKwon commented on PR #43116:
URL: https://github.com/apache/spark/pull/43116#issuecomment-1736381727
Merged to master.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a diff in pull request #43116: [SPARK-45328][SQL] Remove Hive support prior to 2.0.0
Posted by "HyukjinKwon (via GitHub)" <gi...@apache.org>.
HyukjinKwon commented on code in PR #43116:
URL: https://github.com/apache/spark/pull/43116#discussion_r1336764205
##########
sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala:
##########
@@ -115,12 +115,6 @@ private[hive] class HiveClientImpl(
private val outputBuffer = new CircularBuffer()
private val shim = version match {
- case hive.v12 => new Shim_v0_12()
- case hive.v13 => new Shim_v0_13()
- case hive.v14 => new Shim_v0_14()
- case hive.v1_0 => new Shim_v1_0()
- case hive.v1_1 => new Shim_v1_1()
- case hive.v1_2 => new Shim_v1_2()
case hive.v2_0 => new Shim_v2_0()
Review Comment:
This inherits each other, and it's different from `v14`, etc.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a diff in pull request #43116: [SPARK-45328][SQL] Remove Hive support prior to 2.0.0
Posted by "HyukjinKwon (via GitHub)" <gi...@apache.org>.
HyukjinKwon commented on code in PR #43116:
URL: https://github.com/apache/spark/pull/43116#discussion_r1336763403
##########
sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala:
##########
@@ -115,12 +115,6 @@ private[hive] class HiveClientImpl(
private val outputBuffer = new CircularBuffer()
private val shim = version match {
- case hive.v12 => new Shim_v0_12()
- case hive.v13 => new Shim_v0_13()
- case hive.v14 => new Shim_v0_14()
- case hive.v1_0 => new Shim_v1_0()
- case hive.v1_1 => new Shim_v1_1()
- case hive.v1_2 => new Shim_v1_2()
case hive.v2_0 => new Shim_v2_0()
Review Comment:
I think this is really fine to keep them. I made them all private for now, and explicitly throw an exception if other versions are specified.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon closed pull request #43116: [SPARK-45328][SQL] Remove Hive support prior to 2.0.0
Posted by "HyukjinKwon (via GitHub)" <gi...@apache.org>.
HyukjinKwon closed pull request #43116: [SPARK-45328][SQL] Remove Hive support prior to 2.0.0
URL: https://github.com/apache/spark/pull/43116
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a diff in pull request #43116: [SPARK-45328][SQL] Remove Hive support prior to 2.0.0
Posted by "HyukjinKwon (via GitHub)" <gi...@apache.org>.
HyukjinKwon commented on code in PR #43116:
URL: https://github.com/apache/spark/pull/43116#discussion_r1337135025
##########
sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala:
##########
@@ -115,12 +115,6 @@ private[hive] class HiveClientImpl(
private val outputBuffer = new CircularBuffer()
private val shim = version match {
- case hive.v12 => new Shim_v0_12()
- case hive.v13 => new Shim_v0_13()
- case hive.v14 => new Shim_v0_14()
- case hive.v1_0 => new Shim_v1_0()
- case hive.v1_1 => new Shim_v1_1()
- case hive.v1_2 => new Shim_v1_2()
case hive.v2_0 => new Shim_v2_0()
Review Comment:
> I'd like to make such a PR if you don't oppose
Yes please :-).
##########
sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala:
##########
@@ -115,12 +115,6 @@ private[hive] class HiveClientImpl(
private val outputBuffer = new CircularBuffer()
private val shim = version match {
- case hive.v12 => new Shim_v0_12()
- case hive.v13 => new Shim_v0_13()
- case hive.v14 => new Shim_v0_14()
- case hive.v1_0 => new Shim_v1_0()
- case hive.v1_1 => new Shim_v1_1()
- case hive.v1_2 => new Shim_v1_2()
case hive.v2_0 => new Shim_v2_0()
Review Comment:
> I'd like to make such a PR if you don't oppose
Yes please go ahead :-).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a diff in pull request #43116: [SPARK-45328][SQL] Remove Hive support prior to 2.0.0
Posted by "HyukjinKwon (via GitHub)" <gi...@apache.org>.
HyukjinKwon commented on code in PR #43116:
URL: https://github.com/apache/spark/pull/43116#discussion_r1336961705
##########
sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala:
##########
@@ -115,12 +115,6 @@ private[hive] class HiveClientImpl(
private val outputBuffer = new CircularBuffer()
private val shim = version match {
- case hive.v12 => new Shim_v0_12()
- case hive.v13 => new Shim_v0_13()
- case hive.v14 => new Shim_v0_14()
- case hive.v1_0 => new Shim_v1_0()
- case hive.v1_1 => new Shim_v1_1()
- case hive.v1_2 => new Shim_v1_2()
case hive.v2_0 => new Shim_v2_0()
Review Comment:
Those dead code isn't really needed but keeping them as are doesn't risk, and they are well organized according to versions.
I also worry about downstram users/vendors who maintain their forks and still need to use them, and would like to alleviate the overhead.
Maybe let's do this in a separate PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] pan3793 commented on a diff in pull request #43116: [SPARK-45328][SQL] Remove Hive support prior to 2.0.0
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on code in PR #43116:
URL: https://github.com/apache/spark/pull/43116#discussion_r1336971541
##########
sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala:
##########
@@ -1366,7 +1366,7 @@ private[client] class Shim_v1_1 extends Shim_v1_0 {
}
-private[client] class Shim_v1_2 extends Shim_v1_1 {
+private class Shim_v1_2 extends Shim_v1_1 {
Review Comment:
It does not really help, would rather keep it as-is to reduce the git history
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] pan3793 commented on a diff in pull request #43116: [SPARK-45328][SQL] Remove Hive support prior to 2.0.0
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on code in PR #43116:
URL: https://github.com/apache/spark/pull/43116#discussion_r1336632646
##########
sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/HiveTempPath.scala:
##########
@@ -44,52 +44,21 @@ class HiveTempPath(session: SparkSession, val hadoopConf: Configuration, path: P
private def getExternalTmpPath(path: Path): Path = {
import org.apache.spark.sql.hive.client.hive._
- // Before Hive 1.1, when inserting into a table, Hive will create the staging directory under
- // a common scratch directory. After the writing is finished, Hive will simply empty the table
- // directory and move the staging directory to it.
- // After Hive 1.1, Hive will create the staging directory under the table directory, and when
+ // Hive will creates the staging directory under the table directory, and when
// moving staging directory to table directory, Hive will still empty the table directory, but
// will exclude the staging directory there.
- // We have to follow the Hive behavior here, to avoid troubles. For example, if we create
- // staging directory under the table director for Hive prior to 1.1, the staging directory will
- // be removed by Hive when Hive is trying to empty the table directory.
- val hiveVersionsUsingOldExternalTempPath: Set[HiveVersion] = Set(v12, v13, v14, v1_0)
- val hiveVersionsUsingNewExternalTempPath: Set[HiveVersion] =
- Set(v1_1, v1_2, v2_0, v2_1, v2_2, v2_3, v3_0, v3_1)
-
- // Ensure all the supported versions are considered here.
- assert(hiveVersionsUsingNewExternalTempPath ++ hiveVersionsUsingOldExternalTempPath ==
- allSupportedHiveVersions)
val externalCatalog = session.sharedState.externalCatalog
val hiveVersion = externalCatalog.unwrapped.asInstanceOf[HiveExternalCatalog].client.version
val stagingDir = hadoopConf.get("hive.exec.stagingdir", ".hive-staging")
- val scratchDir = hadoopConf.get("hive.exec.scratchdir", "/tmp/hive")
- if (hiveVersionsUsingOldExternalTempPath.contains(hiveVersion)) {
- oldVersionExternalTempPath(path, scratchDir)
- } else if (hiveVersionsUsingNewExternalTempPath.contains(hiveVersion)) {
+ if (allSupportedHiveVersions.contains(hiveVersion)) {
newVersionExternalTempPath(path, stagingDir)
} else {
throw new IllegalStateException("Unsupported hive version: " + hiveVersion.fullVersion)
}
}
- // Mostly copied from Context.java#getExternalTmpPath of Hive 0.13
- private def oldVersionExternalTempPath(path: Path, scratchDir: String): Path = {
- val extURI: URI = path.toUri
- val scratchPath = new Path(scratchDir, executionId)
- var dirPath = new Path(
- extURI.getScheme,
- extURI.getAuthority,
- scratchPath.toUri.getPath + "-" + TaskRunner.getTaskRunnerID())
-
- val fs = dirPath.getFileSystem(hadoopConf)
- dirPath = new Path(fs.makeQualified(dirPath).toString())
- stagingDirForCreating = Some(dirPath)
- dirPath
- }
-
// Mostly copied from Context.java#getExternalTmpPath of Hive 1.2
private def newVersionExternalTempPath(path: Path, stagingDir: String): Path = {
Review Comment:
it's a little bit weird to call it **newVersion** since there is no **oldVersion** now, maybe simply inline it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a diff in pull request #43116: [SPARK-45328][SQL] Remove Hive support prior to 2.0.0
Posted by "HyukjinKwon (via GitHub)" <gi...@apache.org>.
HyukjinKwon commented on code in PR #43116:
URL: https://github.com/apache/spark/pull/43116#discussion_r1337137433
##########
sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala:
##########
@@ -1366,7 +1366,7 @@ private[client] class Shim_v1_1 extends Shim_v1_0 {
}
-private[client] class Shim_v1_2 extends Shim_v1_1 {
+private class Shim_v1_2 extends Shim_v1_1 {
Review Comment:
Let's address them in your PR. Let me get this in first.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] pan3793 commented on a diff in pull request #43116: [SPARK-45328][SQL] Remove Hive support prior to 2.0.0
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on code in PR #43116:
URL: https://github.com/apache/spark/pull/43116#discussion_r1336612849
##########
sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala:
##########
@@ -115,12 +115,6 @@ private[hive] class HiveClientImpl(
private val outputBuffer = new CircularBuffer()
private val shim = version match {
- case hive.v12 => new Shim_v0_12()
- case hive.v13 => new Shim_v0_13()
- case hive.v14 => new Shim_v0_14()
- case hive.v1_0 => new Shim_v1_0()
- case hive.v1_1 => new Shim_v1_1()
- case hive.v1_2 => new Shim_v1_2()
case hive.v2_0 => new Shim_v2_0()
Review Comment:
seems the above `Shim_v*` classes could be removed and merged the code into `Shim_v2_0`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] pan3793 commented on a diff in pull request #43116: [SPARK-45328][SQL] Remove Hive support prior to 2.0.0
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on code in PR #43116:
URL: https://github.com/apache/spark/pull/43116#discussion_r1336808388
##########
sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala:
##########
@@ -115,12 +115,6 @@ private[hive] class HiveClientImpl(
private val outputBuffer = new CircularBuffer()
private val shim = version match {
- case hive.v12 => new Shim_v0_12()
- case hive.v13 => new Shim_v0_13()
- case hive.v14 => new Shim_v0_14()
- case hive.v1_0 => new Shim_v1_0()
- case hive.v1_1 => new Shim_v1_1()
- case hive.v1_2 => new Shim_v1_2()
case hive.v2_0 => new Shim_v2_0()
Review Comment:
Emm.. what's the benefit of retaining unreachable code?
For example, all of `Shim_v0_12`, `Shim_v0_14` and `Shim_v2_0` define `loadPartition`, after this patch, the `loadPartition` defined in `Shim_v0_12`, `Shim_v0_14` becomes dead code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org