You are viewing a plain text version of this content. The canonical link for it is here.
Posted to gitbox@hive.apache.org by "ayushtkn (via GitHub)" <gi...@apache.org> on 2023/03/28 06:21:29 UTC
[GitHub] [hive] ayushtkn opened a new pull request, #4165: HIVE-27185: Iceberg: Cache iceberg table while loading for stats.
ayushtkn opened a new pull request, #4165:
URL: https://github.com/apache/hive/pull/4165
### What changes were proposed in this pull request?
Cache iceberg table while reading stats.
### Why are the changes needed?
Perf benifits.
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Existing Tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #4165: HIVE-27185: Iceberg: Cache iceberg table while loading for stats.
Posted by "deniskuzZ (via GitHub)" <gi...@apache.org>.
deniskuzZ commented on code in PR #4165:
URL: https://github.com/apache/hive/pull/4165#discussion_r1153208694
##########
iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/hive/IcebergTableUtil.java:
##########
@@ -52,12 +52,13 @@ private IcebergTableUtil() {
* Constructs the table properties needed for the Iceberg table loading by retrieving the information from the
* hmsTable. It then calls {@link IcebergTableUtil#getTable(Configuration, Properties)} with these properties.
* @param configuration a Hadoop configuration
- * @param hmsTable the HMS table
- * @param skipCache if set to true there won't be an attempt to retrieve the table from SessionState
+ * @param hmsTable the HMS table
+ * @param skipCache if set to true there won't be an attempt to retrieve the table from SessionState
+ * @param suffix the suffix to use for cache.
* @return the Iceberg table
*/
static Table getTable(Configuration configuration, org.apache.hadoop.hive.metastore.api.Table hmsTable,
- boolean skipCache) {
+ boolean skipCache, String suffix) {
Review Comment:
what do you mean only for stats, we don't have an unlimited heap. if you know that cache key is stale in certain cases (CTAS, MV) - invalidate the cache.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] ayushtkn commented on a diff in pull request #4165: HIVE-27185: Iceberg: Cache iceberg table while loading for stats.
Posted by "ayushtkn (via GitHub)" <gi...@apache.org>.
ayushtkn commented on code in PR #4165:
URL: https://github.com/apache/hive/pull/4165#discussion_r1153050938
##########
iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/hive/IcebergTableUtil.java:
##########
@@ -52,12 +52,13 @@ private IcebergTableUtil() {
* Constructs the table properties needed for the Iceberg table loading by retrieving the information from the
* hmsTable. It then calls {@link IcebergTableUtil#getTable(Configuration, Properties)} with these properties.
* @param configuration a Hadoop configuration
- * @param hmsTable the HMS table
- * @param skipCache if set to true there won't be an attempt to retrieve the table from SessionState
+ * @param hmsTable the HMS table
+ * @param skipCache if set to true there won't be an attempt to retrieve the table from SessionState
+ * @param suffix the suffix to use for cache.
* @return the Iceberg table
*/
static Table getTable(Configuration configuration, org.apache.hadoop.hive.metastore.api.Table hmsTable,
- boolean skipCache) {
+ boolean skipCache, String suffix) {
Review Comment:
We want to cache the table for Stats only, if you fetch once for Stats, then use that till end, if you fetch for something else, we don't use that.
In most cases that won't be a problem, but the way we do CTAS and MV's it did creates problem. It first creates an iceberg table and then does the load into that, so in that cases caching would have stale table. Got a test failure as well for those MV cases in the last run
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] ayushtkn commented on a diff in pull request #4165: HIVE-27185: Iceberg: Cache iceberg table while loading for stats.
Posted by "ayushtkn (via GitHub)" <gi...@apache.org>.
ayushtkn commented on code in PR #4165:
URL: https://github.com/apache/hive/pull/4165#discussion_r1154319629
##########
iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/hive/IcebergTableUtil.java:
##########
@@ -52,12 +52,13 @@ private IcebergTableUtil() {
* Constructs the table properties needed for the Iceberg table loading by retrieving the information from the
* hmsTable. It then calls {@link IcebergTableUtil#getTable(Configuration, Properties)} with these properties.
* @param configuration a Hadoop configuration
- * @param hmsTable the HMS table
- * @param skipCache if set to true there won't be an attempt to retrieve the table from SessionState
+ * @param hmsTable the HMS table
+ * @param skipCache if set to true there won't be an attempt to retrieve the table from SessionState
+ * @param suffix the suffix to use for cache.
* @return the Iceberg table
*/
static Table getTable(Configuration configuration, org.apache.hadoop.hive.metastore.api.Table hmsTable,
- boolean skipCache) {
+ boolean skipCache, String suffix) {
Review Comment:
I changed the approach, if it is write operation where the rows affected gets changed, I read from the Cache, if not do the same what we do today.
Give a check, if that sounds cool, or if you feel if it that can also cause some issues,
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #4165: HIVE-27185: Iceberg: Cache iceberg table while loading for stats.
Posted by "deniskuzZ (via GitHub)" <gi...@apache.org>.
deniskuzZ commented on code in PR #4165:
URL: https://github.com/apache/hive/pull/4165#discussion_r1157306903
##########
ql/src/java/org/apache/hadoop/hive/ql/session/SessionStateUtil.java:
##########
@@ -115,7 +115,7 @@ public static boolean addCommitInfo(Configuration conf, String tableName, String
return addResource(conf, COMMIT_INFO_PREFIX + tableName, newCommitInfoMap);
}
- private static Optional<QueryState> getQueryState(Configuration conf) {
+ public static Optional<QueryState> getQueryState(Configuration conf) {
return Optional.ofNullable(SessionState.get())
Review Comment:
could we refactor to:
````
return Optional.ofNullable(SessionState.get())
.map(ss -> ss.getQueryState(HiveConf.getQueryString(conf)));
````
HiveConf
````
public static String getQuertyId(Configuration conf) {
return getVar(conf, HiveConf.ConfVars.HIVEQUERYID, "");
}
````
see getQueryString
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #4165: HIVE-27185: Iceberg: Cache iceberg table while loading for stats.
Posted by "deniskuzZ (via GitHub)" <gi...@apache.org>.
deniskuzZ commented on code in PR #4165:
URL: https://github.com/apache/hive/pull/4165#discussion_r1153208694
##########
iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/hive/IcebergTableUtil.java:
##########
@@ -52,12 +52,13 @@ private IcebergTableUtil() {
* Constructs the table properties needed for the Iceberg table loading by retrieving the information from the
* hmsTable. It then calls {@link IcebergTableUtil#getTable(Configuration, Properties)} with these properties.
* @param configuration a Hadoop configuration
- * @param hmsTable the HMS table
- * @param skipCache if set to true there won't be an attempt to retrieve the table from SessionState
+ * @param hmsTable the HMS table
+ * @param skipCache if set to true there won't be an attempt to retrieve the table from SessionState
+ * @param suffix the suffix to use for cache.
* @return the Iceberg table
*/
static Table getTable(Configuration configuration, org.apache.hadoop.hive.metastore.api.Table hmsTable,
- boolean skipCache) {
+ boolean skipCache, String suffix) {
Review Comment:
what do you mean only for stats, we don't have an unlimited heap. if you know that cache is stale in certain cases (CTAS, MV) - invalidate the cache.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] ayushtkn merged pull request #4165: HIVE-27185: Iceberg: Cache iceberg table while loading for stats.
Posted by "ayushtkn (via GitHub)" <gi...@apache.org>.
ayushtkn merged PR #4165:
URL: https://github.com/apache/hive/pull/4165
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sonarcloud[bot] commented on pull request #4165: HIVE-27185: Iceberg: Cache iceberg table while loading for stats.
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #4165:
URL: https://github.com/apache/hive/pull/4165#issuecomment-1490951443
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4165)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4165&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4165&resolved=false&types=BUG) [1 Bug](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4165&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4165&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4165&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4165&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4165&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4165&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4165&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4165&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4165&resolved=false&types=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4165&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4165&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4165&metric=duplicated_lines_density&view=list) No Duplication information
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #4165: HIVE-27185: Iceberg: Cache iceberg table while loading for stats.
Posted by "deniskuzZ (via GitHub)" <gi...@apache.org>.
deniskuzZ commented on code in PR #4165:
URL: https://github.com/apache/hive/pull/4165#discussion_r1157306903
##########
ql/src/java/org/apache/hadoop/hive/ql/session/SessionStateUtil.java:
##########
@@ -115,7 +115,7 @@ public static boolean addCommitInfo(Configuration conf, String tableName, String
return addResource(conf, COMMIT_INFO_PREFIX + tableName, newCommitInfoMap);
}
- private static Optional<QueryState> getQueryState(Configuration conf) {
+ public static Optional<QueryState> getQueryState(Configuration conf) {
return Optional.ofNullable(SessionState.get())
Review Comment:
could we refactor to:
````
return Optional.ofNullable(SessionState.get())
.map(ss -> ss.getQueryState(HiveConf.getQueryId(conf)));
````
HiveConf
````
public static String getQuertyId(Configuration conf) {
return getVar(conf, HiveConf.ConfVars.HIVEQUERYID, "");
}
````
see getQueryString
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #4165: HIVE-27185: Iceberg: Cache iceberg table while loading for stats.
Posted by "deniskuzZ (via GitHub)" <gi...@apache.org>.
deniskuzZ commented on code in PR #4165:
URL: https://github.com/apache/hive/pull/4165#discussion_r1157274432
##########
iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergStorageHandler.java:
##########
@@ -315,8 +316,9 @@ public boolean canProvideBasicStatistics() {
@Override
public Map<String, String> getBasicStatistics(Partish partish) {
org.apache.hadoop.hive.ql.metadata.Table hmsTable = partish.getTable();
- TableDesc tableDesc = Utilities.getTableDesc(hmsTable);
- Table table = Catalogs.loadTable(conf, tableDesc.getProperties());
+ Table table;
+ // For write queries where rows got modified, don't fetch from cache as values could have changed.
+ table = getTable(hmsTable);
Review Comment:
can we inline the declaration and assignment?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #4165: HIVE-27185: Iceberg: Cache iceberg table while loading for stats.
Posted by "deniskuzZ (via GitHub)" <gi...@apache.org>.
deniskuzZ commented on code in PR #4165:
URL: https://github.com/apache/hive/pull/4165#discussion_r1157306903
##########
ql/src/java/org/apache/hadoop/hive/ql/session/SessionStateUtil.java:
##########
@@ -115,7 +115,7 @@ public static boolean addCommitInfo(Configuration conf, String tableName, String
return addResource(conf, COMMIT_INFO_PREFIX + tableName, newCommitInfoMap);
}
- private static Optional<QueryState> getQueryState(Configuration conf) {
+ public static Optional<QueryState> getQueryState(Configuration conf) {
return Optional.ofNullable(SessionState.get())
Review Comment:
could we refactor to:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sonarcloud[bot] commented on pull request #4165: HIVE-27185: Iceberg: Cache iceberg table while loading for stats.
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #4165:
URL: https://github.com/apache/hive/pull/4165#issuecomment-1486428000
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4165)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4165&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4165&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4165&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4165&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4165&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4165&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4165&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4165&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4165&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4165&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4165&resolved=false&types=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4165&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4165&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4165&metric=duplicated_lines_density&view=list) No Duplication information
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #4165: HIVE-27185: Iceberg: Cache iceberg table while loading for stats.
Posted by "deniskuzZ (via GitHub)" <gi...@apache.org>.
deniskuzZ commented on code in PR #4165:
URL: https://github.com/apache/hive/pull/4165#discussion_r1153022663
##########
iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/hive/IcebergTableUtil.java:
##########
@@ -52,12 +52,13 @@ private IcebergTableUtil() {
* Constructs the table properties needed for the Iceberg table loading by retrieving the information from the
* hmsTable. It then calls {@link IcebergTableUtil#getTable(Configuration, Properties)} with these properties.
* @param configuration a Hadoop configuration
- * @param hmsTable the HMS table
- * @param skipCache if set to true there won't be an attempt to retrieve the table from SessionState
+ * @param hmsTable the HMS table
+ * @param skipCache if set to true there won't be an attempt to retrieve the table from SessionState
+ * @param suffix the suffix to use for cache.
* @return the Iceberg table
*/
static Table getTable(Configuration configuration, org.apache.hadoop.hive.metastore.api.Table hmsTable,
- boolean skipCache) {
+ boolean skipCache, String suffix) {
Review Comment:
Why do we need the suffix?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #4165: HIVE-27185: Iceberg: Cache iceberg table while loading for stats.
Posted by "deniskuzZ (via GitHub)" <gi...@apache.org>.
deniskuzZ commented on code in PR #4165:
URL: https://github.com/apache/hive/pull/4165#discussion_r1153208694
##########
iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/hive/IcebergTableUtil.java:
##########
@@ -52,12 +52,13 @@ private IcebergTableUtil() {
* Constructs the table properties needed for the Iceberg table loading by retrieving the information from the
* hmsTable. It then calls {@link IcebergTableUtil#getTable(Configuration, Properties)} with these properties.
* @param configuration a Hadoop configuration
- * @param hmsTable the HMS table
- * @param skipCache if set to true there won't be an attempt to retrieve the table from SessionState
+ * @param hmsTable the HMS table
+ * @param skipCache if set to true there won't be an attempt to retrieve the table from SessionState
+ * @param suffix the suffix to use for cache.
* @return the Iceberg table
*/
static Table getTable(Configuration configuration, org.apache.hadoop.hive.metastore.api.Table hmsTable,
- boolean skipCache) {
+ boolean skipCache, String suffix) {
Review Comment:
what do you mean only for stats, we don't have an unlimited heap. if you know that stats is stale in certain cases (CTAS, MV) - invalidate the cache.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sonarcloud[bot] commented on pull request #4165: HIVE-27185: Iceberg: Cache iceberg table while loading for stats.
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #4165:
URL: https://github.com/apache/hive/pull/4165#issuecomment-1491338121
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4165)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4165&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4165&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4165&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4165&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4165&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4165&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4165&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4165&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4165&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4165&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4165&resolved=false&types=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4165&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4165&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4165&metric=duplicated_lines_density&view=list) No Duplication information
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #4165: HIVE-27185: Iceberg: Cache iceberg table while loading for stats.
Posted by "deniskuzZ (via GitHub)" <gi...@apache.org>.
deniskuzZ commented on code in PR #4165:
URL: https://github.com/apache/hive/pull/4165#discussion_r1157282843
##########
iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergStorageHandler.java:
##########
@@ -349,6 +351,17 @@ public Map<String, String> getBasicStatistics(Partish partish) {
return stats;
}
+ private Table getTable(org.apache.hadoop.hive.ql.metadata.Table hmsTable) {
+ Table table;
+ final Optional<QueryState> queryState = SessionStateUtil.getQueryState(conf);
+ if (!queryState.isPresent() || queryState.get().getNumModifiedRows() != 0) {
Review Comment:
`queryState.get().getNumModifiedRows() > 0`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sonarcloud[bot] commented on pull request #4165: HIVE-27185: Iceberg: Cache iceberg table while loading for stats.
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #4165:
URL: https://github.com/apache/hive/pull/4165#issuecomment-1496272763
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4165)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4165&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4165&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4165&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4165&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4165&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4165&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4165&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4165&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4165&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4165&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4165&resolved=false&types=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4165&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4165&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4165&metric=duplicated_lines_density&view=list) No Duplication information
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sonarcloud[bot] commented on pull request #4165: HIVE-27185: Iceberg: Cache iceberg table while loading for stats.
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #4165:
URL: https://github.com/apache/hive/pull/4165#issuecomment-1486981460
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4165)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4165&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4165&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4165&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4165&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4165&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4165&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4165&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4165&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4165&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4165&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4165&resolved=false&types=CODE_SMELL) [1 Code Smell](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4165&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4165&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4165&metric=duplicated_lines_density&view=list) No Duplication information
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org