You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "zhuqi-lucas (via GitHub)" <gi...@apache.org> on 2023/08/07 07:25:26 UTC
[GitHub] [spark] zhuqi-lucas opened a new pull request, #42374: [SPARK-44698][SQL] Create table like other table should also copy tab…
zhuqi-lucas opened a new pull request, #42374:
URL: https://github.com/apache/spark/pull/42374
…le stats.
### What changes were proposed in this pull request?
Create table like other table should also copy table stats, now the stats is missing from the source table to the target table, for example:
For example:
describe table extended tbl;
col0 int
col1 int
col2 int
col3 int
Detailed Table Information
Catalog spark_catalog
Database default
Table tbl
Owner zhuqi
Created Time Mon Aug 07 14:02:30 CST 2023
Last Access UNKNOWN
Created By Spark 4.0.0-SNAPSHOT
Type MANAGED
Provider hive
Table Properties [transient_lastDdlTime=1691388473]
Statistics 30 bytes
Location [file:/Users/zhuqi/spark/spark/spark-warehouse/tbl](file:///Users/zhuqi/spark/spark/spark-warehouse/tbl)
Serde Library org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
InputFormat org.apache.hadoop.mapred.TextInputFormat
OutputFormat org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
Storage Properties [serialization.format=1]
Partition Provider Catalog
Time taken: 0.032 seconds, Fetched 23 row(s)
create table tbl2 like tbl;
23/08/07 14:14:07 WARN HiveMetaStore: Location: [file:/Users/zhuqi/spark/spark/spark-warehouse/tbl2](file:///Users/zhuqi/spark/spark/spark-warehouse/tbl2) specified for non-external table:tbl2
Time taken: 0.098 seconds
spark-sql (default)> describe table extended tbl2;
col0 int
col1 int
col2 int
col3 int
Detailed Table Information
Catalog spark_catalog
Database default
Table tbl2
Owner zhuqi
Created Time Mon Aug 07 14:14:07 CST 2023
Last Access UNKNOWN
Created By Spark 4.0.0-SNAPSHOT
Type MANAGED
Provider hive
Table Properties [transient_lastDdlTime=1691388847]
Location [file:/Users/zhuqi/spark/spark/spark-warehouse/tbl2](file:///Users/zhuqi/spark/spark/spark-warehouse/tbl2)
Serde Library org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
InputFormat org.apache.hadoop.mapred.TextInputFormat
OutputFormat org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
Storage Properties [serialization.format=1]
Partition Provider Catalog
Time taken: 0.03 seconds, Fetched 22 row(s)
The table stats are missing.
### Why are the changes needed?
1. Add the stats to copy.
2. Add the corresponding unit test to test.
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
I create a unit test in code to confirm it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] zhuqi-lucas commented on pull request #42374: [SPARK-44698][SQL] Create table like other table should also copy tab…
Posted by "zhuqi-lucas (via GitHub)" <gi...@apache.org>.
zhuqi-lucas commented on PR #42374:
URL: https://github.com/apache/spark/pull/42374#issuecomment-1667358670
Sorry i misunderstand, the create table like don't need to copy data actually!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] zhuqi-lucas commented on pull request #42374: [SPARK-44698][SQL] Create table like other table should also copy tab…
Posted by "zhuqi-lucas (via GitHub)" <gi...@apache.org>.
zhuqi-lucas commented on PR #42374:
URL: https://github.com/apache/spark/pull/42374#issuecomment-1667337316
cc @MaxGekk @LuciferYang @HyukjinKwon @sunchao Could you help review this PR? Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] zhuqi-lucas closed pull request #42374: [SPARK-44698][SQL] Create table like other table should also copy tab…
Posted by "zhuqi-lucas (via GitHub)" <gi...@apache.org>.
zhuqi-lucas closed pull request #42374: [SPARK-44698][SQL] Create table like other table should also copy tab…
URL: https://github.com/apache/spark/pull/42374
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org