You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by gatorsmile <gi...@git.apache.org> on 2016/08/28 22:51:00 UTC
[GitHub] spark pull request #14855: [SPARK-17284] [SQL] Remove Statistics-related Tab...
GitHub user gatorsmile opened a pull request:
https://github.com/apache/spark/pull/14855
[SPARK-17284] [SQL] Remove Statistics-related Table Properties from SHOW CREATE TABLE
### What changes were proposed in this pull request?
The statistics-related table properties should be skipped by ```SHOW CREATE TABLE```, since it could be incorrect in the newly created table. See the Hive JIRA: https://issues.apache.org/jira/browse/HIVE-13792
```SQL
CREATE TABLE t1 (
c1 INT COMMENT 'bla',
c2 STRING
)
LOCATION '$dir'
TBLPROPERTIES (
'prop1' = 'value1',
'prop2' = 'value2'
)
```
The output of ```SHOW CREATE TABLE t1``` is
```SQL
CREATE EXTERNAL TABLE `t1`(`c1` int COMMENT 'bla', `c2` string)
ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe'
WITH SERDEPROPERTIES (
'serialization.format' = '1'
)
STORED AS
INPUTFORMAT 'org.apache.hadoop.mapred.TextInputFormat'
OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'
LOCATION 'file:/private/var/folders/4b/sgmfldk15js406vk7lw5llzw0000gn/T/spark-ee317538-0f8c-42d0-b08c-cf077d94fe75'
TBLPROPERTIES (
'rawDataSize' = '-1',
'numFiles' = '0',
'transient_lastDdlTime' = '1472424052',
'totalSize' = '0',
'prop1' = 'value1',
'prop2' = 'value2',
'COLUMN_STATS_ACCURATE' = 'false',
'numRows' = '-1'
)
```
After the fix, the output becomes
```SQL
CREATE EXTERNAL TABLE `t1`(`c1` int COMMENT 'bla', `c2` string)
ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe'
WITH SERDEPROPERTIES (
'serialization.format' = '1'
)
STORED AS
INPUTFORMAT 'org.apache.hadoop.mapred.TextInputFormat'
OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'
LOCATION 'file:/private/var/folders/4b/sgmfldk15js406vk7lw5llzw0000gn/T/spark-74058a6d-db8b-41c1-9bda-bd449f1a78ed'
TBLPROPERTIES (
'transient_lastDdlTime' = '1472423603',
'prop1' = 'value1',
'prop2' = 'value2'
)
```
### How was this patch tested?
Updated the existing test cases.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/gatorsmile/spark showCreateTable
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/14855.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #14855
----
commit 92474c5a142fb9db2c86549c8347f910fc01fcbd
Author: gatorsmile <ga...@gmail.com>
Date: 2016-08-28T22:28:15Z
remove stats-related props
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14855: [SPARK-17284] [SQL] Remove Statistics-related Table Prop...
Posted by gatorsmile <gi...@git.apache.org>.
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/14855
This PR is part of another PR https://github.com/apache/spark/pull/14971. Close it now.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #14855: [SPARK-17284] [SQL] Remove Statistics-related Tab...
Posted by wzhfy <gi...@git.apache.org>.
Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/14855#discussion_r76724760
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala ---
@@ -791,11 +791,22 @@ case class ShowCreateTableCommand(table: TableIdentifier) extends RunnableComman
}
}
+ // These table properties should not be included in the output statement of SHOW CREATE TABLE
+ val excludedTableProperties = Set(
+ // The following are hive-generated statistics fields
+ "COLUMN_STATS_ACCURATE",
+ "numFiles",
+ "numPartitions",
+ "numRows",
+ "rawDataSize",
+ "totalSize"
+ )
--- End diff --
Shouldn't we set each of these property names as a constant so that we can use them in the translation layer?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14855: [SPARK-17284] [SQL] Remove Statistics-related Table Prop...
Posted by gatorsmile <gi...@git.apache.org>.
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/14855
cc @hvanhovell @cloud-fan @liancheng @yhuai Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14855: [SPARK-17284] [SQL] Remove Statistics-related Table Prop...
Posted by gatorsmile <gi...@git.apache.org>.
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/14855
Will try it.
My current solution is to first retrieve the table properties and fill these Hive-specific properties (if not set by the callers) before issuing the command to Hive. Will do it tonight. Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #14855: [SPARK-17284] [SQL] Remove Statistics-related Tab...
Posted by viirya <gi...@git.apache.org>.
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/14855#discussion_r76552894
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala ---
@@ -794,8 +794,10 @@ case class ShowCreateTableCommand(table: TableIdentifier) extends RunnableComman
private def showHiveTableProperties(metadata: CatalogTable, builder: StringBuilder): Unit = {
if (metadata.properties.nonEmpty) {
val filteredProps = metadata.properties.filterNot {
- // Skips "EXTERNAL" property for external tables
- case (key, _) => key == "EXTERNAL" && metadata.tableType == EXTERNAL
+ // Skips all the stats info (See the JIRA: HIVE-13792)
+ case (key, _) =>
+ key == "numFiles" || key == "numRows" || key == "totalSize" || key == "numPartitions" ||
--- End diff --
Is it possible that the list of hidden properties will grow in future? If so, can we not add them with `||` here? A separated list like the `excludedTableProperties` below seems good. And we can check if the key is in the list.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #14855: [SPARK-17284] [SQL] Remove Statistics-related Tab...
Posted by gatorsmile <gi...@git.apache.org>.
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/14855#discussion_r76694730
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala ---
@@ -791,11 +791,22 @@ case class ShowCreateTableCommand(table: TableIdentifier) extends RunnableComman
}
}
+ // These table properties should not be included in the output statement of SHOW CREATE TABLE
+ val excludedTableProperties = Set(
+ // The following are hive-generated statistics fields
+ "COLUMN_STATS_ACCURATE",
+ "numFiles",
+ "numPartitions",
+ "numRows",
+ "rawDataSize",
+ "totalSize"
+ )
+
private def showHiveTableProperties(metadata: CatalogTable, builder: StringBuilder): Unit = {
if (metadata.properties.nonEmpty) {
val filteredProps = metadata.properties.filterNot {
- // Skips "EXTERNAL" property for external tables
- case (key, _) => key == "EXTERNAL" && metadata.tableType == EXTERNAL
+ // Skips all the stats info (See the JIRA: HIVE-13792)
--- End diff --
Yeah, agree on this general rule.
When we support translations, we need to be very careful about including these statistics info in the SHOW CREATE TABLE DDL. Hive does not include them in SHOW CREATE TABLE, as shown in their JIRA: https://issues.apache.org/jira/browse/HIVE-13792. If we allow users to provide the statistics info when creating the tables, we might need to mark them as inaccurate, like what Hive does now?
BTW, should we merge this in 2.1 before we support the translation? So far, Spark 2.0 has the bug. Let me know what I should do next.
Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14855: [SPARK-17284] [SQL] Remove Statistics-related Table Prop...
Posted by gatorsmile <gi...@git.apache.org>.
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/14855
Another example is `ALTER TABLE`. Basically, we retrieve the original table properties and then modify the existing `CatalogTable` and then put it back.
https://github.com/apache/spark/blob/f7c9ff57c17a950cccdc26aadf8768c899a4d572/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala#L238-L246
How about hiding them in `HiveClientImp`? Like what we did for `EXTERNAL` and `comment` in the table properties? The advantage is to leave the others unaware of the existence of these Hive-specific attributes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14855: [SPARK-17284] [SQL] Remove Statistics-related Table Prop...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14855
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64560/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #14855: [SPARK-17284] [SQL] Remove Statistics-related Tab...
Posted by gatorsmile <gi...@git.apache.org>.
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/14855#discussion_r76725105
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala ---
@@ -791,11 +791,22 @@ case class ShowCreateTableCommand(table: TableIdentifier) extends RunnableComman
}
}
+ // These table properties should not be included in the output statement of SHOW CREATE TABLE
+ val excludedTableProperties = Set(
+ // The following are hive-generated statistics fields
+ "COLUMN_STATS_ACCURATE",
+ "numFiles",
+ "numPartitions",
+ "numRows",
+ "rawDataSize",
+ "totalSize"
+ )
--- End diff --
This PR is for fixing a bug. We might need to backport to 2.0. When we implementing the translation layer, we can do that, just like what we did for the property names of the Data Source Table schema
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14855: [SPARK-17284] [SQL] Remove Statistics-related Table Prop...
Posted by cloud-fan <gi...@git.apache.org>.
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/14855
Where do we need the hive table statistics properties? If nowhere, I think it's better to filter them out at the very beginning, i.e. `HiveExternalCatalog`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #14855: [SPARK-17284] [SQL] Remove Statistics-related Tab...
Posted by hvanhovell <gi...@git.apache.org>.
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/14855#discussion_r76692456
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala ---
@@ -791,11 +791,22 @@ case class ShowCreateTableCommand(table: TableIdentifier) extends RunnableComman
}
}
+ // These table properties should not be included in the output statement of SHOW CREATE TABLE
+ val excludedTableProperties = Set(
+ // The following are hive-generated statistics fields
+ "COLUMN_STATS_ACCURATE",
+ "numFiles",
+ "numPartitions",
+ "numRows",
+ "rawDataSize",
+ "totalSize"
+ )
+
private def showHiveTableProperties(metadata: CatalogTable, builder: StringBuilder): Unit = {
if (metadata.properties.nonEmpty) {
val filteredProps = metadata.properties.filterNot {
- // Skips "EXTERNAL" property for external tables
- case (key, _) => key == "EXTERNAL" && metadata.tableType == EXTERNAL
+ // Skips all the stats info (See the JIRA: HIVE-13792)
--- End diff --
I mean in general: we should not leak Hive specific implementation (i.e. hacks) into sql/core.
At first I thought that we might be able to hide/filter them but you have a point. The good news is that this becomes a non-issue when we have statistics as a part of the CatalogTable. Then these properties can become the problem of a yet-to-be written translation.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #14855: [SPARK-17284] [SQL] Remove Statistics-related Tab...
Posted by gatorsmile <gi...@git.apache.org>.
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/14855#discussion_r76554157
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala ---
@@ -794,8 +794,10 @@ case class ShowCreateTableCommand(table: TableIdentifier) extends RunnableComman
private def showHiveTableProperties(metadata: CatalogTable, builder: StringBuilder): Unit = {
if (metadata.properties.nonEmpty) {
val filteredProps = metadata.properties.filterNot {
- // Skips "EXTERNAL" property for external tables
- case (key, _) => key == "EXTERNAL" && metadata.tableType == EXTERNAL
+ // Skips all the stats info (See the JIRA: HIVE-13792)
+ case (key, _) =>
+ key == "numFiles" || key == "numRows" || key == "totalSize" || key == "numPartitions" ||
--- End diff --
Sure, let me change it. Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #14855: [SPARK-17284] [SQL] Remove Statistics-related Tab...
Posted by gatorsmile <gi...@git.apache.org>.
Github user gatorsmile closed the pull request at:
https://github.com/apache/spark/pull/14855
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #14855: [SPARK-17284] [SQL] Remove Statistics-related Tab...
Posted by hvanhovell <gi...@git.apache.org>.
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/14855#discussion_r76687807
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala ---
@@ -791,11 +791,22 @@ case class ShowCreateTableCommand(table: TableIdentifier) extends RunnableComman
}
}
+ // These table properties should not be included in the output statement of SHOW CREATE TABLE
+ val excludedTableProperties = Set(
+ // The following are hive-generated statistics fields
+ "COLUMN_STATS_ACCURATE",
+ "numFiles",
+ "numPartitions",
+ "numRows",
+ "rawDataSize",
+ "totalSize"
+ )
+
private def showHiveTableProperties(metadata: CatalogTable, builder: StringBuilder): Unit = {
if (metadata.properties.nonEmpty) {
val filteredProps = metadata.properties.filterNot {
- // Skips "EXTERNAL" property for external tables
- case (key, _) => key == "EXTERNAL" && metadata.tableType == EXTERNAL
+ // Skips all the stats info (See the JIRA: HIVE-13792)
--- End diff --
Shouldn't we fix this in the `HiveExternalCatalog`?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14855: [SPARK-17284] [SQL] Remove Statistics-related Table Prop...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14855
**[Test build #64560 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64560/consoleFull)** for PR 14855 at commit [`ce8e8b8`](https://github.com/apache/spark/commit/ce8e8b89a5b61648daaa59578e2b6a99ec2f6d74).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #14855: [SPARK-17284] [SQL] Remove Statistics-related Tab...
Posted by gatorsmile <gi...@git.apache.org>.
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/14855#discussion_r76690858
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala ---
@@ -791,11 +791,22 @@ case class ShowCreateTableCommand(table: TableIdentifier) extends RunnableComman
}
}
+ // These table properties should not be included in the output statement of SHOW CREATE TABLE
+ val excludedTableProperties = Set(
+ // The following are hive-generated statistics fields
+ "COLUMN_STATS_ACCURATE",
+ "numFiles",
+ "numPartitions",
+ "numRows",
+ "rawDataSize",
+ "totalSize"
+ )
+
private def showHiveTableProperties(metadata: CatalogTable, builder: StringBuilder): Unit = {
if (metadata.properties.nonEmpty) {
val filteredProps = metadata.properties.filterNot {
- // Skips "EXTERNAL" property for external tables
- case (key, _) => key == "EXTERNAL" && metadata.tableType == EXTERNAL
+ // Skips all the stats info (See the JIRA: HIVE-13792)
--- End diff --
Not sure whether I got your point. Are you saying `getTableOption` in `HiveExternalCatalog`?
Initially, I had the same design like you. Later, I realized we still use/display them in the other DDL statements, for example, `DESCRIBE EXTENDED TABLE`.
Let me know if my understanding is wrong. Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14855: [SPARK-17284] [SQL] Remove Statistics-related Table Prop...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14855
**[Test build #64560 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64560/consoleFull)** for PR 14855 at commit [`ce8e8b8`](https://github.com/apache/spark/commit/ce8e8b89a5b61648daaa59578e2b6a99ec2f6d74).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14855: [SPARK-17284] [SQL] Remove Statistics-related Table Prop...
Posted by cloud-fan <gi...@git.apache.org>.
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/14855
sgtm, do we need to retrieve the statistics properties in `HiveClientImp` before alter a table? Can hive metastore automatically fill them?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14855: [SPARK-17284] [SQL] Remove Statistics-related Table Prop...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14855
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org