You are viewing a plain text version of this content. The canonical link for it is here.
Posted to gitbox@hive.apache.org by "henrib (via GitHub)" <gi...@apache.org> on 2023/05/09 09:56:53 UTC

[GitHub] [hive] henrib opened a new pull request, #4303: HIVE-27268: synthetize empty id lists if no session available

henrib opened a new pull request, #4303:
URL: https://github.com/apache/hive/pull/4303

   
   ### What changes were proposed in this pull request?
   When a tool runs without a session, create an empty id list.
   
   ### Why are the changes needed?
   To avoid a potential NPE and be consistent.
   
   ### Does this PR introduce _any_ user-facing change?
   No
   
   ### How was this patch tested?
   Unit test explicitly closing session before call


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org

[GitHub] [hive] henrib commented on a diff in pull request #4303: HIVE-27268: synthetize empty id lists if no session available

Posted by "henrib (via GitHub)" <gi...@apache.org>.
henrib commented on code in PR #4303:
URL: https://github.com/apache/hive/pull/4303#discussion_r1188989921


##########
ql/src/test/org/apache/hadoop/hive/ql/TestTxnNoBuckets.java:
##########
@@ -909,5 +910,31 @@ public void testEmptyCompactionResult() throws Exception {
     Assert.assertEquals(stringifyValues(data), rs);
 
   }
+
+  /**
+   * HIVE-27268
+   */
+  @Test
+  public void testGetPartitionsNoSession() throws Exception {
+    hiveConf.setIntVar(HiveConf.ConfVars.HIVEOPTSORTDYNAMICPARTITIONTHRESHOLD, -1);
+    runStatementOnDriver("drop table if exists T");
+    runStatementOnDriver("create table T(a int, b int) partitioned by (p int, q int) " +
+        "stored as orc TBLPROPERTIES ('transactional'='true')");
+
+    int[][] targetVals = {{4, 1, 1}, {4, 2, 2}, {4, 3, 1}, {4, 4, 2}};
+    //we only recompute stats after major compact if they existed before
+    runStatementOnDriver("insert into T partition(p=1,q) " + makeValuesClause(targetVals));
+    runStatementOnDriver("analyze table T  partition(p=1) compute statistics for columns");
+
+    Hive hive = Hive.get();
+    org.apache.hadoop.hive.ql.metadata.Table hiveTable = hive.getTable("T");
+    // this will ensure the getValidWriteIdList has no session to work with (thru getPartitions)
+    SessionState.detachSession();
+    List<org.apache.hadoop.hive.ql.metadata.Partition> partitions = hive.getPartitions(hiveTable);
+    Assert.assertNotNull(partitions);
+    // prevent tear down failure
+    d.close();
+    d = null;

Review Comment:
   We do because tearDown() (attempts to) drop the table whilst our session has already been severed.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org

[GitHub] [hive] saihemanth-cloudera merged pull request #4303: HIVE-27268: synthetize empty id lists if no session available

Posted by "saihemanth-cloudera (via GitHub)" <gi...@apache.org>.
saihemanth-cloudera merged PR #4303:
URL: https://github.com/apache/hive/pull/4303


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org

[GitHub] [hive] sonarcloud[bot] commented on pull request #4303: HIVE-27268: synthetize empty id lists if no session available

Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #4303:
URL: https://github.com/apache/hive/pull/4303#issuecomment-1540325301

   Kudos, SonarCloud Quality Gate passed!&nbsp; &nbsp; [![Quality Gate passed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/passed-16px.png 'Quality Gate passed')](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4303)
   
   [![Bug](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/bug-16px.png 'Bug')](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4303&resolved=false&types=BUG) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4303&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4303&resolved=false&types=BUG)  
   [![Vulnerability](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/vulnerability-16px.png 'Vulnerability')](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4303&resolved=false&types=VULNERABILITY) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4303&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4303&resolved=false&types=VULNERABILITY)  
   [![Security Hotspot](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/security_hotspot-16px.png 'Security Hotspot')](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4303&resolved=false&types=SECURITY_HOTSPOT) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4303&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4303&resolved=false&types=SECURITY_HOTSPOT)  
   [![Code Smell](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/code_smell-16px.png 'Code Smell')](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4303&resolved=false&types=CODE_SMELL) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4303&resolved=false&types=CODE_SMELL) [7 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4303&resolved=false&types=CODE_SMELL)
   
   [![No Coverage information](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/CoverageChart/NoCoverageInfo-16px.png 'No Coverage information')](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4303&metric=coverage&view=list) No Coverage information  
   [![No Duplication information](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/Duplications/NoDuplicationInfo-16px.png 'No Duplication information')](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4303&metric=duplicated_lines_density&view=list) No Duplication information
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org

[GitHub] [hive] saihemanth-cloudera commented on a diff in pull request #4303: HIVE-27268: synthetize empty id lists if no session available

Posted by "saihemanth-cloudera (via GitHub)" <gi...@apache.org>.
saihemanth-cloudera commented on code in PR #4303:
URL: https://github.com/apache/hive/pull/4303#discussion_r1188922553


##########
ql/src/test/org/apache/hadoop/hive/ql/TestTxnNoBuckets.java:
##########
@@ -909,5 +910,31 @@ public void testEmptyCompactionResult() throws Exception {
     Assert.assertEquals(stringifyValues(data), rs);
 
   }
+
+  /**
+   * HIVE-27268
+   */
+  @Test
+  public void testGetPartitionsNoSession() throws Exception {
+    hiveConf.setIntVar(HiveConf.ConfVars.HIVEOPTSORTDYNAMICPARTITIONTHRESHOLD, -1);
+    runStatementOnDriver("drop table if exists T");
+    runStatementOnDriver("create table T(a int, b int) partitioned by (p int, q int) " +
+        "stored as orc TBLPROPERTIES ('transactional'='true')");
+
+    int[][] targetVals = {{4, 1, 1}, {4, 2, 2}, {4, 3, 1}, {4, 4, 2}};
+    //we only recompute stats after major compact if they existed before
+    runStatementOnDriver("insert into T partition(p=1,q) " + makeValuesClause(targetVals));
+    runStatementOnDriver("analyze table T  partition(p=1) compute statistics for columns");
+
+    Hive hive = Hive.get();
+    org.apache.hadoop.hive.ql.metadata.Table hiveTable = hive.getTable("T");
+    // this will ensure the getValidWriteIdList has no session to work with (thru getPartitions)
+    SessionState.detachSession();
+    List<org.apache.hadoop.hive.ql.metadata.Partition> partitions = hive.getPartitions(hiveTable);
+    Assert.assertNotNull(partitions);
+    // prevent tear down failure
+    d.close();
+    d = null;

Review Comment:
   Nit: Do we even need this after closing the driver state in the line above?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org