You are viewing a plain text version of this content. The canonical link for it is here.
Posted to gitbox@hive.apache.org by GitBox <gi...@apache.org> on 2022/10/27 05:58:49 UTC
[GitHub] [hive] rkirtir opened a new pull request, #3708: HIVE-26580: SHOW COMPACTIONS should support ordering and limiting fun…
rkirtir opened a new pull request, #3708:
URL: https://github.com/apache/hive/pull/3708
…ctionality in filtering options
<!--
Thanks for sending a pull request! Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://cwiki.apache.org/confluence/display/Hive/HowToContribute
2. Ensure that you have created an issue on the Hive project JIRA: https://issues.apache.org/jira/projects/HIVE/summary
3. Ensure you have added or run the appropriate tests for your PR:
4. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP]HIVE-XXXXX: Your PR title ...'.
5. Be sure to keep the PR description updated to reflect all changes.
6. Please write your PR title to summarize what this PR proposes.
7. If possible, provide a concise example to reproduce the issue for a faster review.
-->
### What changes were proposed in this pull request?
Adding sorting and limit functionality to show compaction
### Why are the changes needed?
https://issues.apache.org/jira/browse/HIVE-26580
### Does this PR introduce _any_ user-facing change?
yes
### How was this patch tested?
junit
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] rkirtir closed pull request #3708: HIVE-26580: SHOW COMPACTIONS should support ordering and limiting fun…
Posted by GitBox <gi...@apache.org>.
rkirtir closed pull request #3708: HIVE-26580: SHOW COMPACTIONS should support ordering and limiting fun…
URL: https://github.com/apache/hive/pull/3708
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] rkirtir commented on a diff in pull request #3708: HIVE-26580: SHOW COMPACTIONS should support ordering and limiting fun…
Posted by GitBox <gi...@apache.org>.
rkirtir commented on code in PR #3708:
URL: https://github.com/apache/hive/pull/3708#discussion_r1007694800
##########
ql/src/test/org/apache/hadoop/hive/ql/TestTxnCommands.java:
##########
@@ -2388,6 +2396,65 @@ public void testShowCompactionInputValidation() throws Exception {
"STATUS 'ready for clean'");//validates compaction status
}
+ @Test
+ public void testShowCompactionFilterSortingAndLimit()throws Exception {
+ runStatementOnDriver("drop database if exists mydb1 cascade");
+ runStatementOnDriver("create database mydb1");
+ runStatementOnDriver("create table mydb1.tbl0 " + "(a int, b int) partitioned by (p string) clustered by (a) into " +
+ BUCKET_COUNT + " buckets stored as orc TBLPROPERTIES ('transactional'='true')");
+ runStatementOnDriver("insert into mydb1.tbl0" + " PARTITION(p) " +
+ " values(1,2,'p1'),(3,4,'p1'),(1,2,'p2'),(3,4,'p2'),(1,2,'p3'),(3,4,'p3')");
+ runStatementOnDriver("alter table mydb1.tbl0" + " PARTITION(p='p1') compact 'MAJOR'");
+ TestTxnCommands2.runWorker(hiveConf);
+ runStatementOnDriver("alter table mydb1.tbl0" + " PARTITION(p='p2') compact 'MAJOR'");
+ TestTxnCommands2.runWorker(hiveConf);
+
+
+ runStatementOnDriver("drop database if exists mydb cascade");
+ runStatementOnDriver("create database mydb");
+ runStatementOnDriver("create table mydb.tbl " + "(a int, b int) partitioned by (ds string) clustered by (a) into " +
+ BUCKET_COUNT + " buckets stored as orc TBLPROPERTIES ('transactional'='true')");
+ runStatementOnDriver("insert into mydb.tbl" + " PARTITION(ds) " +
+ " values(1,2,'mon'),(3,4,'tue'),(1,2,'mon'),(3,4,'tue'),(1,2,'wed'),(3,4,'wed')");
+ runStatementOnDriver("alter table mydb.tbl" + " PARTITION(ds='mon') compact 'MAJOR'");
+ TestTxnCommands2.runWorker(hiveConf);
+ runStatementOnDriver("alter table mydb.tbl" + " PARTITION(ds='tue') compact 'MAJOR'");
+ TestTxnCommands2.runWorker(hiveConf);
+
+ runStatementOnDriver("create table mydb.tbl2 " + "(a int, b int) partitioned by (dm string) clustered by (a) into " +
+ BUCKET_COUNT + " buckets stored as orc TBLPROPERTIES ('transactional'='true')");
+ runStatementOnDriver("insert into mydb.tbl2" + " PARTITION(dm) " +
+ " values(1,2,'xxx'),(3,4,'xxx'),(1,2,'yyy'),(3,4,'yyy'),(1,2,'zzz'),(3,4,'zzz')");
+ runStatementOnDriver("alter table mydb.tbl2" + " PARTITION(dm='yyy') compact 'MAJOR'");
+ TestTxnCommands2.runWorker(hiveConf);
+ runStatementOnDriver("alter table mydb.tbl2" + " PARTITION(dm='zzz') compact 'MAJOR'");
+ TestTxnCommands2.runWorker(hiveConf);
+
+ ShowCompactResponse rsp = txnHandler.showCompact(new ShowCompactRequest());
+
+ //includes Header row
+ List<String> r = runStatementOnDriver("SHOW COMPACTIONS");
+ Assert.assertEquals(rsp.getCompacts().size() + 1, r.size());
+ r = runStatementOnDriver("SHOW COMPACTIONS LIMIT 3");
+ Assert.assertEquals(4, r.size());
+ r = runStatementOnDriver("SHOW COMPACTIONS SCHEMA mydb TYPE 'MAJOR' LIMIT 2");
+ Assert.assertEquals(3, r.size());
+
+ r = runStatementOnDriver("SHOW COMPACTIONS SCHEMA mydb TYPE 'MAJOR' ORDER BY CC_TABLE DESC, CC_PARTITION ASC");
Review Comment:
Agree.
I have hanged show compaction output like
"CompactionId\tDbname\tTbname\tPartitions\tType\tState\tWorker host\tWorker
And show commanction command with custom sort order would be like(casing does not matter)
SHOW COMPACTIONS SCHEMA mydb TYPE 'MAJOR' ORDER BY tbname DESC,PARTITIONS ASC
what do you think
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sonarcloud[bot] commented on pull request #3708: HIVE-26580: SHOW COMPACTIONS should support ordering and limiting fun…
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on PR #3708:
URL: https://github.com/apache/hive/pull/3708#issuecomment-1293456691
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=3708)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3708&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3708&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3708&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3708&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3708&metric=duplicated_lines_density&view=list) No Duplication information
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sonarcloud[bot] commented on pull request #3708: HIVE-26580: SHOW COMPACTIONS should support ordering and limiting fun…
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on PR #3708:
URL: https://github.com/apache/hive/pull/3708#issuecomment-1313686588
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=3708)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3708&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3708&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3708&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=CODE_SMELL) [2 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3708&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3708&metric=duplicated_lines_density&view=list) No Duplication information
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] rkirtir commented on a diff in pull request #3708: HIVE-26580: SHOW COMPACTIONS should support ordering and limiting fun…
Posted by GitBox <gi...@apache.org>.
rkirtir commented on code in PR #3708:
URL: https://github.com/apache/hive/pull/3708#discussion_r1007691246
##########
ql/src/java/org/apache/hadoop/hive/ql/ddl/process/show/compactions/ShowCompactionsDesc.java:
##########
@@ -45,10 +45,12 @@ public class ShowCompactionsDesc implements DDLDesc, Serializable {
private final String compactionType;
private final String compactionStatus;
private final Map<String, String> partSpec;
+ private final short limit;
+ private final String order;
public ShowCompactionsDesc(Path resFile, long compactionId, String dbName, String tbName, String poolName, String compactionType,
- String compactionStatus, Map<String, String> partSpec) {
+ String compactionStatus, Map<String, String> partSpec, short limit, String order) {
Review Comment:
fixed
##########
ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java:
##########
@@ -1984,4 +1984,28 @@ public ParseContext getParseContext() {
return pCtx;
}
+ public PTFInvocationSpec.OrderSpec processOrderSpec(ASTNode sortNode) {
+ PTFInvocationSpec.OrderSpec oSpec = new PTFInvocationSpec.OrderSpec();
Review Comment:
fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #3708: HIVE-26580: SHOW COMPACTIONS should support ordering and limiting fun…
Posted by GitBox <gi...@apache.org>.
deniskuzZ commented on code in PR #3708:
URL: https://github.com/apache/hive/pull/3708#discussion_r1022849389
##########
ql/src/java/org/apache/hadoop/hive/ql/ddl/process/show/compactions/ShowCompactionsOperation.java:
##########
@@ -102,6 +102,12 @@ private ShowCompactRequest getShowCompactioRequest(ShowCompactionsDesc desc) thr
if(desc.getCompactionId()>0){
request.setId(desc.getCompactionId());
}
+ if (desc.getLimit() > -1) {
Review Comment:
> 0?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #3708: HIVE-26580: SHOW COMPACTIONS should support ordering and limiting fun…
Posted by GitBox <gi...@apache.org>.
deniskuzZ commented on code in PR #3708:
URL: https://github.com/apache/hive/pull/3708#discussion_r1022844493
##########
ql/src/java/org/apache/hadoop/hive/ql/ddl/process/show/compactions/ShowCompactionsAnalyzer.java:
##########
@@ -80,15 +83,64 @@ public void analyzeInternal(ASTNode root) throws SemanticException {
case HiveParser.TOK_COMPACT_ID:
compactionId = Long.parseLong(child.getChild(0).getText());
break;
+ case HiveParser.TOK_LIMIT:
+ limit = Short.valueOf((child.getChild(0)).getText());
+ break;
+ case HiveParser.TOK_ORDERBY:
+ orderBy = processSortOrderSpec(child);
+ break;
default:
dbName = stripQuotes(child.getText());
}
}
ShowCompactionsDesc desc = new ShowCompactionsDesc(ctx.getResFile(), compactionId, dbName, tblName, poolName, compactionType,
- compactionStatus, partitionSpec);
+ compactionStatus, partitionSpec, limit, orderBy);
Task<DDLWork> task = TaskFactory.get(new DDLWork(getInputs(), getOutputs(), desc));
rootTasks.add(task);
task.setFetchSource(true);
setFetchTask(createFetchTask(ShowCompactionsDesc.SCHEMA));
}
+
+
+ private String processSortOrderSpec(ASTNode sortNode) {
+ List<PTFInvocationSpec.OrderExpression> orderExp = processOrderSpec(sortNode).getExpressions();
+ Map<String, String> orderByAttributes = orderExp.stream().collect(
+ Collectors.toMap(x -> {
+ return x.getExpression().getChild(0)==null ?
+ CompactionColumn.valueOf(x.getExpression().getText().replaceAll("\'","").toUpperCase()).toString():
+ CompactionColumn.valueOf(x.getExpression().getChild(0).getText()).toString();
+ }, x -> x.getOrder().toString()));
+ return orderByAttributes.entrySet().stream().map(e -> e.getKey() + "\t" + e.getValue()).collect(Collectors.joining(","));
+ }
+ public enum CompactionColumn {
+ COMPACTIONID("CC_ID"),
+ PARTITION("CC_PARTITION"),
+ DATABASE("CC_DATABASE"),
Review Comment:
add SCHEMA("CC_DATABASE")
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #3708: HIVE-26580: SHOW COMPACTIONS should support ordering and limiting fun…
Posted by GitBox <gi...@apache.org>.
deniskuzZ commented on code in PR #3708:
URL: https://github.com/apache/hive/pull/3708#discussion_r1018194153
##########
ql/src/java/org/apache/hadoop/hive/ql/io/AcidUtils.java:
##########
@@ -230,6 +230,20 @@ public boolean accept(Path path) {
}
};
+ public enum CompactionColumn {
Review Comment:
If you want to create a column name mapping function, please keep it localized (i.e. ShowCompactionsAnalyzer) + add mapping for "SCHEMA" -> "DATABASE". I don't think you need to define an enum, simply append "CC_".
Note: users should be able to order by any of the columns from `show compactions` output
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sonarcloud[bot] commented on pull request #3708: HIVE-26580: SHOW COMPACTIONS should support ordering and limiting fun…
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on PR #3708:
URL: https://github.com/apache/hive/pull/3708#issuecomment-1311588047
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=3708)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3708&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3708&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3708&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=CODE_SMELL) [2 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3708&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3708&metric=duplicated_lines_density&view=list) No Duplication information
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #3708: HIVE-26580: SHOW COMPACTIONS should support ordering and limiting fun…
Posted by GitBox <gi...@apache.org>.
deniskuzZ commented on code in PR #3708:
URL: https://github.com/apache/hive/pull/3708#discussion_r1018202359
##########
standalone-metastore/metastore-common/src/main/thrift/hive_metastore.thrift:
##########
@@ -1351,7 +1351,9 @@ struct ShowCompactRequest {
4: required string tablename,
5: optional string partitionname,
6: required CompactionType type,
- 7: required string state
+ 7: required string state,
+ 8: optional i64 limit,
+ 9: optional string order
Review Comment:
it's not a map, but map serialized into string
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #3708: HIVE-26580: SHOW COMPACTIONS should support ordering and limiting fun…
Posted by GitBox <gi...@apache.org>.
deniskuzZ commented on code in PR #3708:
URL: https://github.com/apache/hive/pull/3708#discussion_r1018189155
##########
ql/src/java/org/apache/hadoop/hive/ql/io/AcidUtils.java:
##########
@@ -3438,4 +3452,14 @@ public static CompactionState compactionStateStr2Enum(String inputValue) throws
throw new SemanticException("Unexpected compaction state " + inputValue);
}
}
+
+ public static CompactionColumn compactionColStr2Enum(String inputValue) throws SemanticException {
+ try {
+ return CompactionColumn.valueOf(inputValue.toUpperCase());
+ }catch (IllegalArgumentException e) {
+ throw new SemanticException("Unexpected sorting column name" + inputValue);
Review Comment:
space at the end
##########
ql/src/java/org/apache/hadoop/hive/ql/io/AcidUtils.java:
##########
@@ -3438,4 +3452,14 @@ public static CompactionState compactionStateStr2Enum(String inputValue) throws
throw new SemanticException("Unexpected compaction state " + inputValue);
}
}
+
+ public static CompactionColumn compactionColStr2Enum(String inputValue) throws SemanticException {
Review Comment:
not needed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] rkirtir commented on a diff in pull request #3708: HIVE-26580: SHOW COMPACTIONS should support ordering and limiting fun…
Posted by GitBox <gi...@apache.org>.
rkirtir commented on code in PR #3708:
URL: https://github.com/apache/hive/pull/3708#discussion_r1022368037
##########
ql/src/test/org/apache/hadoop/hive/ql/TestTxnCommands.java:
##########
@@ -2388,6 +2396,65 @@ public void testShowCompactionInputValidation() throws Exception {
"STATUS 'ready for clean'");//validates compaction status
}
+ @Test
+ public void testShowCompactionFilterSortingAndLimit()throws Exception {
+ runStatementOnDriver("drop database if exists mydb1 cascade");
+ runStatementOnDriver("create database mydb1");
+ runStatementOnDriver("create table mydb1.tbl0 " + "(a int, b int) partitioned by (p string) clustered by (a) into " +
+ BUCKET_COUNT + " buckets stored as orc TBLPROPERTIES ('transactional'='true')");
+ runStatementOnDriver("insert into mydb1.tbl0" + " PARTITION(p) " +
+ " values(1,2,'p1'),(3,4,'p1'),(1,2,'p2'),(3,4,'p2'),(1,2,'p3'),(3,4,'p3')");
+ runStatementOnDriver("alter table mydb1.tbl0" + " PARTITION(p='p1') compact 'MAJOR'");
+ TestTxnCommands2.runWorker(hiveConf);
+ runStatementOnDriver("alter table mydb1.tbl0" + " PARTITION(p='p2') compact 'MAJOR'");
+ TestTxnCommands2.runWorker(hiveConf);
+
+
+ runStatementOnDriver("drop database if exists mydb cascade");
+ runStatementOnDriver("create database mydb");
+ runStatementOnDriver("create table mydb.tbl " + "(a int, b int) partitioned by (ds string) clustered by (a) into " +
+ BUCKET_COUNT + " buckets stored as orc TBLPROPERTIES ('transactional'='true')");
+ runStatementOnDriver("insert into mydb.tbl" + " PARTITION(ds) " +
+ " values(1,2,'mon'),(3,4,'tue'),(1,2,'mon'),(3,4,'tue'),(1,2,'wed'),(3,4,'wed')");
+ runStatementOnDriver("alter table mydb.tbl" + " PARTITION(ds='mon') compact 'MAJOR'");
+ TestTxnCommands2.runWorker(hiveConf);
+ runStatementOnDriver("alter table mydb.tbl" + " PARTITION(ds='tue') compact 'MAJOR'");
+ TestTxnCommands2.runWorker(hiveConf);
+
+ runStatementOnDriver("create table mydb.tbl2 " + "(a int, b int) partitioned by (dm string) clustered by (a) into " +
+ BUCKET_COUNT + " buckets stored as orc TBLPROPERTIES ('transactional'='true')");
+ runStatementOnDriver("insert into mydb.tbl2" + " PARTITION(dm) " +
+ " values(1,2,'xxx'),(3,4,'xxx'),(1,2,'yyy'),(3,4,'yyy'),(1,2,'zzz'),(3,4,'zzz')");
+ runStatementOnDriver("alter table mydb.tbl2" + " PARTITION(dm='yyy') compact 'MAJOR'");
+ TestTxnCommands2.runWorker(hiveConf);
+ runStatementOnDriver("alter table mydb.tbl2" + " PARTITION(dm='zzz') compact 'MAJOR'");
+ TestTxnCommands2.runWorker(hiveConf);
+
+ ShowCompactResponse rsp = txnHandler.showCompact(new ShowCompactRequest());
+
+ //includes Header row
+ List<String> r = runStatementOnDriver("SHOW COMPACTIONS");
+ Assert.assertEquals(rsp.getCompacts().size() + 1, r.size());
+ r = runStatementOnDriver("SHOW COMPACTIONS LIMIT 3");
+ Assert.assertEquals(4, r.size());
+ r = runStatementOnDriver("SHOW COMPACTIONS SCHEMA mydb TYPE 'MAJOR' LIMIT 2");
+ Assert.assertEquals(3, r.size());
+
+ r = runStatementOnDriver("SHOW COMPACTIONS SCHEMA mydb TYPE 'MAJOR' ORDER BY CC_TABLE DESC, CC_PARTITION ASC");
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #3708: HIVE-26580: SHOW COMPACTIONS should support ordering and limiting fun…
Posted by GitBox <gi...@apache.org>.
deniskuzZ commented on code in PR #3708:
URL: https://github.com/apache/hive/pull/3708#discussion_r1022843105
##########
ql/src/java/org/apache/hadoop/hive/ql/ddl/process/show/compactions/ShowCompactionsAnalyzer.java:
##########
@@ -80,15 +83,64 @@ public void analyzeInternal(ASTNode root) throws SemanticException {
case HiveParser.TOK_COMPACT_ID:
compactionId = Long.parseLong(child.getChild(0).getText());
break;
+ case HiveParser.TOK_LIMIT:
+ limit = Short.valueOf((child.getChild(0)).getText());
+ break;
+ case HiveParser.TOK_ORDERBY:
+ orderBy = processSortOrderSpec(child);
+ break;
default:
dbName = stripQuotes(child.getText());
}
}
ShowCompactionsDesc desc = new ShowCompactionsDesc(ctx.getResFile(), compactionId, dbName, tblName, poolName, compactionType,
- compactionStatus, partitionSpec);
+ compactionStatus, partitionSpec, limit, orderBy);
Task<DDLWork> task = TaskFactory.get(new DDLWork(getInputs(), getOutputs(), desc));
rootTasks.add(task);
task.setFetchSource(true);
setFetchTask(createFetchTask(ShowCompactionsDesc.SCHEMA));
}
+
+
+ private String processSortOrderSpec(ASTNode sortNode) {
+ List<PTFInvocationSpec.OrderExpression> orderExp = processOrderSpec(sortNode).getExpressions();
+ Map<String, String> orderByAttributes = orderExp.stream().collect(
+ Collectors.toMap(x -> {
+ return x.getExpression().getChild(0)==null ?
+ CompactionColumn.valueOf(x.getExpression().getText().replaceAll("\'","").toUpperCase()).toString():
+ CompactionColumn.valueOf(x.getExpression().getChild(0).getText()).toString();
+ }, x -> x.getOrder().toString()));
+ return orderByAttributes.entrySet().stream().map(e -> e.getKey() + "\t" + e.getValue()).collect(Collectors.joining(","));
+ }
+ public enum CompactionColumn {
Review Comment:
should it be public? new line is missing. Do we even need an enum (CC_+value)?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #3708: HIVE-26580: SHOW COMPACTIONS should support ordering and limiting fun…
Posted by GitBox <gi...@apache.org>.
deniskuzZ commented on code in PR #3708:
URL: https://github.com/apache/hive/pull/3708#discussion_r1023682497
##########
itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/txn/compactor/CompactionPoolOnTezTest.java:
##########
@@ -138,8 +138,8 @@ public void testShowCompactionsContainsPoolName() throws Exception {
List results = new ArrayList();
driver.getResults(results);
Assert.assertEquals(3, results.size());
- Assert.assertEquals("CompactionId\tDatabase\tTable\tPartition\tType\tState\tWorker host\tWorker\tEnqueue Time\tStart Time\tDuration(ms)" +
- "\tHadoopJobId\tError message\tInitiator host\tInitiator\tPool name\tTxnId\tNext TxnId\tCommit Time\tHighest WriteID", results.get(0));
+ Assert.assertEquals("Id\tDatabase\tTable\tPartition\tType\tState\tWorker host\tWorker\tEnqueue Time\tStart Time\tDuration(ms)" +
Review Comment:
sorry, Kirti, probably i didn't elaborate well and was lacking some context. We shouldn't change the existing headers except the Id. Also, we should update 'Highest WriteID' to camel-case 'Highest WriteId'.
For those columns where we can't apply default CC+value approach we should have a mapping logic (i.e. HadoopJobId -> CC_HADOOP_JOB_ID).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sonarcloud[bot] commented on pull request #3708: HIVE-26580: SHOW COMPACTIONS should support ordering and limiting fun…
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on PR #3708:
URL: https://github.com/apache/hive/pull/3708#issuecomment-1319753377
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=3708)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3708&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3708&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3708&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3708&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3708&metric=duplicated_lines_density&view=list) No Duplication information
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] rkirtir commented on a diff in pull request #3708: HIVE-26580: SHOW COMPACTIONS should support ordering and limiting fun…
Posted by GitBox <gi...@apache.org>.
rkirtir commented on code in PR #3708:
URL: https://github.com/apache/hive/pull/3708#discussion_r1029203537
##########
itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/txn/compactor/CompactionPoolOnTezTest.java:
##########
@@ -138,8 +138,8 @@ public void testShowCompactionsContainsPoolName() throws Exception {
List results = new ArrayList();
driver.getResults(results);
Assert.assertEquals(3, results.size());
- Assert.assertEquals("CompactionId\tDatabase\tTable\tPartition\tType\tState\tWorker host\tWorker\tEnqueue Time\tStart Time\tDuration(ms)" +
- "\tHadoopJobId\tError message\tInitiator host\tInitiator\tPool name\tTxnId\tNext TxnId\tCommit Time\tHighest WriteID", results.get(0));
+ Assert.assertEquals("Id\tDatabase\tTable\tPartition\tType\tState\tWorker host\tWorker\tEnqueue Time\tStart Time\tDuration(ms)" +
Review Comment:
fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sonarcloud[bot] commented on pull request #3708: HIVE-26580: SHOW COMPACTIONS should support ordering and limiting fun…
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on PR #3708:
URL: https://github.com/apache/hive/pull/3708#issuecomment-1323328660
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=3708)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3708&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3708&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3708&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3708&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3708&metric=duplicated_lines_density&view=list) No Duplication information
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #3708: HIVE-26580: SHOW COMPACTIONS should support ordering and limiting fun…
Posted by GitBox <gi...@apache.org>.
deniskuzZ commented on code in PR #3708:
URL: https://github.com/apache/hive/pull/3708#discussion_r1018189155
##########
ql/src/java/org/apache/hadoop/hive/ql/io/AcidUtils.java:
##########
@@ -3438,4 +3452,14 @@ public static CompactionState compactionStateStr2Enum(String inputValue) throws
throw new SemanticException("Unexpected compaction state " + inputValue);
}
}
+
+ public static CompactionColumn compactionColStr2Enum(String inputValue) throws SemanticException {
+ try {
+ return CompactionColumn.valueOf(inputValue.toUpperCase());
+ }catch (IllegalArgumentException e) {
+ throw new SemanticException("Unexpected sorting column name" + inputValue);
Review Comment:
space at the end
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #3708: HIVE-26580: SHOW COMPACTIONS should support ordering and limiting fun…
Posted by GitBox <gi...@apache.org>.
deniskuzZ commented on code in PR #3708:
URL: https://github.com/apache/hive/pull/3708#discussion_r1018194153
##########
ql/src/java/org/apache/hadoop/hive/ql/io/AcidUtils.java:
##########
@@ -230,6 +230,20 @@ public boolean accept(Path path) {
}
};
+ public enum CompactionColumn {
Review Comment:
redundant, just add cc_ to the column name.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #3708: HIVE-26580: SHOW COMPACTIONS should support ordering and limiting fun…
Posted by GitBox <gi...@apache.org>.
deniskuzZ commented on code in PR #3708:
URL: https://github.com/apache/hive/pull/3708#discussion_r1018178679
##########
ql/src/java/org/apache/hadoop/hive/ql/ddl/process/show/compactions/ShowCompactionsOperation.java:
##########
@@ -99,20 +99,26 @@ private ShowCompactRequest getShowCompactioRequest(ShowCompactionsDesc desc) thr
if (isNotEmpty(desc.getPartSpec())) {
request.setPartitionname(AcidUtils.getPartitionName(desc.getPartSpec()));
}
- if(desc.getCompactionId()>0){
+ if(desc.getCompactionId() > 0){
request.setId(desc.getCompactionId());
}
+ if (desc.getLimit() > -1) {
+ request.setLimit(desc.getLimit());
+ }
+ if (isNotBlank(desc.getOrderBy())) {
+ request.setOrder(desc.getOrderBy());
+ }
return request;
}
private void writeHeader(DataOutputStream os) throws IOException {
os.writeBytes("CompactionId");
os.write(Utilities.tabCode);
- os.writeBytes("Database");
+ os.writeBytes("Dbname");
Review Comment:
do not change existing headers, some of the clients might have scripts relying on them
##########
ql/src/java/org/apache/hadoop/hive/ql/ddl/process/show/compactions/ShowCompactionsOperation.java:
##########
@@ -99,20 +99,26 @@ private ShowCompactRequest getShowCompactioRequest(ShowCompactionsDesc desc) thr
if (isNotEmpty(desc.getPartSpec())) {
request.setPartitionname(AcidUtils.getPartitionName(desc.getPartSpec()));
}
- if(desc.getCompactionId()>0){
+ if(desc.getCompactionId() > 0){
request.setId(desc.getCompactionId());
}
+ if (desc.getLimit() > -1) {
+ request.setLimit(desc.getLimit());
+ }
+ if (isNotBlank(desc.getOrderBy())) {
+ request.setOrder(desc.getOrderBy());
+ }
return request;
}
private void writeHeader(DataOutputStream os) throws IOException {
os.writeBytes("CompactionId");
os.write(Utilities.tabCode);
- os.writeBytes("Database");
+ os.writeBytes("Dbname");
Review Comment:
do not change the existing headers, some of the clients might have scripts relying on them
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #3708: HIVE-26580: SHOW COMPACTIONS should support ordering and limiting fun…
Posted by GitBox <gi...@apache.org>.
deniskuzZ commented on code in PR #3708:
URL: https://github.com/apache/hive/pull/3708#discussion_r1018194153
##########
ql/src/java/org/apache/hadoop/hive/ql/io/AcidUtils.java:
##########
@@ -230,6 +230,20 @@ public boolean accept(Path path) {
}
};
+ public enum CompactionColumn {
Review Comment:
If you want to create a column name mapping function, please keep it localized (i.e. ShowCompactionsAnalyzer) + add mapping for "SCHEMA" -> "DATABASE"
"TABLE" -> "CC_TABLE"
"PARTITION" -> "CC_PARTITION"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #3708: HIVE-26580: SHOW COMPACTIONS should support ordering and limiting fun…
Posted by GitBox <gi...@apache.org>.
deniskuzZ commented on code in PR #3708:
URL: https://github.com/apache/hive/pull/3708#discussion_r1018203932
##########
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java:
##########
@@ -4013,6 +4018,11 @@ private String getShowCompactFilterClause(ShowCompactRequest request) {
" WHERE " + StringUtils.join(" AND ", params) : EMPTY;
}
+ private String getShowCompactSortingOrderClause(ShowCompactRequest request) {
+ String sortingOrder = request.getOrder();
+ return isNotBlank(sortingOrder) ? String.join("\t"," ORDER BY ",sortingOrder) : TxnQueries.SHOW_COMPACTION_ORDERBY_CLAUSE;
Review Comment:
reformat, spaces are missing
also, what is String. join is doing here? can we simply "ORDER BY "+sortingOrder
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] rkirtir commented on a diff in pull request #3708: HIVE-26580: SHOW COMPACTIONS should support ordering and limiting fun…
Posted by GitBox <gi...@apache.org>.
rkirtir commented on code in PR #3708:
URL: https://github.com/apache/hive/pull/3708#discussion_r1021173460
##########
ql/src/java/org/apache/hadoop/hive/ql/io/AcidUtils.java:
##########
@@ -230,6 +230,20 @@ public boolean accept(Path path) {
}
};
+ public enum CompactionColumn {
Review Comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #3708: HIVE-26580: SHOW COMPACTIONS should support ordering and limiting fun…
Posted by GitBox <gi...@apache.org>.
deniskuzZ commented on code in PR #3708:
URL: https://github.com/apache/hive/pull/3708#discussion_r1022843680
##########
ql/src/java/org/apache/hadoop/hive/ql/ddl/process/show/compactions/ShowCompactionsAnalyzer.java:
##########
@@ -80,15 +83,64 @@ public void analyzeInternal(ASTNode root) throws SemanticException {
case HiveParser.TOK_COMPACT_ID:
compactionId = Long.parseLong(child.getChild(0).getText());
break;
+ case HiveParser.TOK_LIMIT:
+ limit = Short.valueOf((child.getChild(0)).getText());
+ break;
+ case HiveParser.TOK_ORDERBY:
+ orderBy = processSortOrderSpec(child);
+ break;
default:
dbName = stripQuotes(child.getText());
}
}
ShowCompactionsDesc desc = new ShowCompactionsDesc(ctx.getResFile(), compactionId, dbName, tblName, poolName, compactionType,
- compactionStatus, partitionSpec);
+ compactionStatus, partitionSpec, limit, orderBy);
Task<DDLWork> task = TaskFactory.get(new DDLWork(getInputs(), getOutputs(), desc));
rootTasks.add(task);
task.setFetchSource(true);
setFetchTask(createFetchTask(ShowCompactionsDesc.SCHEMA));
}
+
+
+ private String processSortOrderSpec(ASTNode sortNode) {
+ List<PTFInvocationSpec.OrderExpression> orderExp = processOrderSpec(sortNode).getExpressions();
+ Map<String, String> orderByAttributes = orderExp.stream().collect(
+ Collectors.toMap(x -> {
+ return x.getExpression().getChild(0)==null ?
+ CompactionColumn.valueOf(x.getExpression().getText().replaceAll("\'","").toUpperCase()).toString():
+ CompactionColumn.valueOf(x.getExpression().getChild(0).getText()).toString();
+ }, x -> x.getOrder().toString()));
+ return orderByAttributes.entrySet().stream().map(e -> e.getKey() + "\t" + e.getValue()).collect(Collectors.joining(","));
+ }
+ public enum CompactionColumn {
+ COMPACTIONID("CC_ID"),
Review Comment:
why COMPACTIONID not just "ID" ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #3708: HIVE-26580: SHOW COMPACTIONS should support ordering and limiting fun…
Posted by GitBox <gi...@apache.org>.
deniskuzZ commented on code in PR #3708:
URL: https://github.com/apache/hive/pull/3708#discussion_r1022858303
##########
ql/src/java/org/apache/hadoop/hive/ql/ddl/process/show/compactions/ShowCompactionsAnalyzer.java:
##########
@@ -80,15 +83,64 @@ public void analyzeInternal(ASTNode root) throws SemanticException {
case HiveParser.TOK_COMPACT_ID:
compactionId = Long.parseLong(child.getChild(0).getText());
break;
+ case HiveParser.TOK_LIMIT:
+ limit = Short.valueOf((child.getChild(0)).getText());
+ break;
+ case HiveParser.TOK_ORDERBY:
+ orderBy = processSortOrderSpec(child);
+ break;
default:
dbName = stripQuotes(child.getText());
}
}
ShowCompactionsDesc desc = new ShowCompactionsDesc(ctx.getResFile(), compactionId, dbName, tblName, poolName, compactionType,
- compactionStatus, partitionSpec);
+ compactionStatus, partitionSpec, limit, orderBy);
Task<DDLWork> task = TaskFactory.get(new DDLWork(getInputs(), getOutputs(), desc));
rootTasks.add(task);
task.setFetchSource(true);
setFetchTask(createFetchTask(ShowCompactionsDesc.SCHEMA));
}
+
+
+ private String processSortOrderSpec(ASTNode sortNode) {
+ List<PTFInvocationSpec.OrderExpression> orderExp = processOrderSpec(sortNode).getExpressions();
+ Map<String, String> orderByAttributes = orderExp.stream().collect(
+ Collectors.toMap(x -> {
+ return x.getExpression().getChild(0)==null ?
+ CompactionColumn.valueOf(x.getExpression().getText().replaceAll("\'","").toUpperCase()).toString():
+ CompactionColumn.valueOf(x.getExpression().getChild(0).getText()).toString();
+ }, x -> x.getOrder().toString()));
+ return orderByAttributes.entrySet().stream().map(e -> e.getKey() + "\t" + e.getValue()).collect(Collectors.joining(","));
+ }
+ public enum CompactionColumn {
+ COMPACTIONID("CC_ID"),
Review Comment:
why not just ID("CC_ID")
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #3708: HIVE-26580: SHOW COMPACTIONS should support ordering and limiting fun…
Posted by GitBox <gi...@apache.org>.
deniskuzZ commented on code in PR #3708:
URL: https://github.com/apache/hive/pull/3708#discussion_r1023637301
##########
ql/src/java/org/apache/hadoop/hive/ql/ddl/process/show/compactions/ShowCompactionsAnalyzer.java:
##########
@@ -80,15 +83,34 @@ public void analyzeInternal(ASTNode root) throws SemanticException {
case HiveParser.TOK_COMPACT_ID:
compactionId = Long.parseLong(child.getChild(0).getText());
break;
+ case HiveParser.TOK_LIMIT:
+ limit = Short.valueOf((child.getChild(0)).getText());
+ break;
+ case HiveParser.TOK_ORDERBY:
+ orderBy = processSortOrderSpec(child);
+ break;
default:
dbName = stripQuotes(child.getText());
}
}
ShowCompactionsDesc desc = new ShowCompactionsDesc(ctx.getResFile(), compactionId, dbName, tblName, poolName, compactionType,
- compactionStatus, partitionSpec);
+ compactionStatus, partitionSpec, limit, orderBy);
Task<DDLWork> task = TaskFactory.get(new DDLWork(getInputs(), getOutputs(), desc));
rootTasks.add(task);
task.setFetchSource(true);
setFetchTask(createFetchTask(ShowCompactionsDesc.SCHEMA));
}
+ private String processSortOrderSpec(ASTNode sortNode) {
+ List<PTFInvocationSpec.OrderExpression> orderExp = processOrderSpec(sortNode).getExpressions();
+ Map<String, String> orderByAttributes = orderExp.stream().
+ collect(Collectors.toMap(x -> getDbColumnName(x.getExpression()), x -> x.getOrder().toString()));
+ return orderByAttributes.entrySet().stream().map(e -> e.getKey() + "\t" + e.getValue()).collect(Collectors.joining(","));
+ }
+ private String getDbColumnName(ASTNode expression) {
+ String dbColumnPrefix = "CC_";
+ String dbColumnName = expression.getChild(0) == null ? expression.getText().replaceAll("\'", "").replaceAll(" ", "_").toUpperCase() :
Review Comment:
code smells report: use replace() instead of replaceAll() if you don't provide regex pattern
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] rkirtir commented on a diff in pull request #3708: HIVE-26580: SHOW COMPACTIONS should support ordering and limiting fun…
Posted by GitBox <gi...@apache.org>.
rkirtir commented on code in PR #3708:
URL: https://github.com/apache/hive/pull/3708#discussion_r1022367918
##########
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java:
##########
@@ -4013,6 +4018,11 @@ private String getShowCompactFilterClause(ShowCompactRequest request) {
" WHERE " + StringUtils.join(" AND ", params) : EMPTY;
}
+ private String getShowCompactSortingOrderClause(ShowCompactRequest request) {
+ String sortingOrder = request.getOrder();
+ return isNotBlank(sortingOrder) ? String.join("\t"," ORDER BY ",sortingOrder) : TxnQueries.SHOW_COMPACTION_ORDERBY_CLAUSE;
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] rkirtir commented on a diff in pull request #3708: HIVE-26580: SHOW COMPACTIONS should support ordering and limiting fun…
Posted by GitBox <gi...@apache.org>.
rkirtir commented on code in PR #3708:
URL: https://github.com/apache/hive/pull/3708#discussion_r1007691520
##########
ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java:
##########
@@ -1984,4 +1984,28 @@ public ParseContext getParseContext() {
return pCtx;
}
+ public PTFInvocationSpec.OrderSpec processOrderSpec(ASTNode sortNode) {
+ PTFInvocationSpec.OrderSpec oSpec = new PTFInvocationSpec.OrderSpec();
+ int exprCnt = sortNode.getChildCount();
+ for(int i=0; i < exprCnt; i++) {
+ PTFInvocationSpec.OrderExpression exprSpec = new PTFInvocationSpec.OrderExpression();
+ ASTNode orderSpec = (ASTNode) sortNode.getChild(i);
+ ASTNode nullOrderSpec = (ASTNode) orderSpec.getChild(0);
+ exprSpec.setExpression((ASTNode) nullOrderSpec.getChild(0));
+ if ( orderSpec.getType() == HiveParser.TOK_TABSORTCOLNAMEASC ) {
+ exprSpec.setOrder(org.apache.hadoop.hive.ql.parse.PTFInvocationSpec.Order.ASC);
Review Comment:
fixed
##########
ql/src/test/org/apache/hadoop/hive/ql/TestTxnCommands.java:
##########
@@ -2155,6 +2155,11 @@ public void testShowCompactions() throws Exception {
ShowCompactResponse rsp = txnHandler.showCompact(new ShowCompactRequest());
List<String> r = runStatementOnDriver("SHOW COMPACTIONS");
Assert.assertEquals(rsp.getCompacts().size()+1, r.size());//includes Header row
+ r = runStatementOnDriver("SHOW COMPACTIONS LIMIT 3");
Review Comment:
fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] rkirtir commented on a diff in pull request #3708: HIVE-26580: SHOW COMPACTIONS should support ordering and limiting fun…
Posted by GitBox <gi...@apache.org>.
rkirtir commented on code in PR #3708:
URL: https://github.com/apache/hive/pull/3708#discussion_r1007690398
##########
standalone-metastore/metastore-common/src/main/thrift/hive_metastore.thrift:
##########
@@ -1351,7 +1351,9 @@ struct ShowCompactRequest {
4: required string tablename,
5: optional string partitionname,
6: required CompactionType type,
- 7: required string state
+ 7: required string state,
+ 8: optional i64 limit,
+ 9: optional string order
Review Comment:
Agree added map for exp,order in ShowCompactionAnalyzer
##########
ql/src/test/org/apache/hadoop/hive/ql/TestTxnCommands.java:
##########
@@ -2155,6 +2155,11 @@ public void testShowCompactions() throws Exception {
ShowCompactResponse rsp = txnHandler.showCompact(new ShowCompactRequest());
List<String> r = runStatementOnDriver("SHOW COMPACTIONS");
Assert.assertEquals(rsp.getCompacts().size()+1, r.size());//includes Header row
+ r = runStatementOnDriver("SHOW COMPACTIONS LIMIT 3");
+ Assert.assertEquals(4, r.size());
+
+ r = runStatementOnDriver("SHOW COMPACTIONS SCHEMA mydb1 TYPE 'MAJOR' ORDER BY CC_PARTITION ASC , CC_TABLE DESC");
Review Comment:
fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sonarcloud[bot] commented on pull request #3708: HIVE-26580: SHOW COMPACTIONS should support ordering and limiting fun…
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on PR #3708:
URL: https://github.com/apache/hive/pull/3708#issuecomment-1305447435
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=3708)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3708&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3708&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3708&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=CODE_SMELL) [1 Code Smell](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3708&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3708&metric=duplicated_lines_density&view=list) No Duplication information
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] rkirtir commented on a diff in pull request #3708: HIVE-26580: SHOW COMPACTIONS should support ordering and limiting fun…
Posted by GitBox <gi...@apache.org>.
rkirtir commented on code in PR #3708:
URL: https://github.com/apache/hive/pull/3708#discussion_r1021173460
##########
ql/src/java/org/apache/hadoop/hive/ql/io/AcidUtils.java:
##########
@@ -230,6 +230,20 @@ public boolean accept(Path path) {
}
};
+ public enum CompactionColumn {
Review Comment:
Added UI input->Schema mapping enum to ShowCompactionsAnalyzer. We cannot take name as it is which is in output. It will be like this
COMPACTIONID("CC_ID"),
PART_NAME("CC_PARTITION"),
DB_NAME("CC_DATABASE"),
TB_NAME("CC_TABLE"),
TYPE("CC_TYPE"),
STATE("CC_STATE"),
WORKERHOST("CC_WORKER_ID"),
ENQUEUETIME("CC_ENQUEUE_TIME"),
POOLNAME("CC_POOL_NAME"),
Worker("CC_WORKER_VERSION"),
START_TIME("CC_START"),
TXNID ("CC_TXN_ID"),
HADOOPJOBID("CC_HADOOP_JOB_ID");
This has to go in hive show compaction wiki
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #3708: HIVE-26580: SHOW COMPACTIONS should support ordering and limiting fun…
Posted by GitBox <gi...@apache.org>.
deniskuzZ commented on code in PR #3708:
URL: https://github.com/apache/hive/pull/3708#discussion_r1018194153
##########
ql/src/java/org/apache/hadoop/hive/ql/io/AcidUtils.java:
##########
@@ -230,6 +230,20 @@ public boolean accept(Path path) {
}
};
+ public enum CompactionColumn {
Review Comment:
redundant, just add `CC_ to the column.toUpperCase()
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #3708: HIVE-26580: SHOW COMPACTIONS should support ordering and limiting fun…
Posted by GitBox <gi...@apache.org>.
deniskuzZ commented on code in PR #3708:
URL: https://github.com/apache/hive/pull/3708#discussion_r1018194153
##########
ql/src/java/org/apache/hadoop/hive/ql/io/AcidUtils.java:
##########
@@ -230,6 +230,20 @@ public boolean accept(Path path) {
}
};
+ public enum CompactionColumn {
Review Comment:
If you want to create a column name mapping function, please keep it localized (i.e. ShowCompactionsAnalyzer) + add mapping for "SCHEMA" -> "DATABASE".
I don't think you need to define an enum, simply append "CC_"+ provided value.
Note: users should be able to order by any of the columns from `show compactions` output
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #3708: HIVE-26580: SHOW COMPACTIONS should support ordering and limiting fun…
Posted by GitBox <gi...@apache.org>.
deniskuzZ commented on code in PR #3708:
URL: https://github.com/apache/hive/pull/3708#discussion_r1018203932
##########
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java:
##########
@@ -4013,6 +4018,11 @@ private String getShowCompactFilterClause(ShowCompactRequest request) {
" WHERE " + StringUtils.join(" AND ", params) : EMPTY;
}
+ private String getShowCompactSortingOrderClause(ShowCompactRequest request) {
+ String sortingOrder = request.getOrder();
+ return isNotBlank(sortingOrder) ? String.join("\t"," ORDER BY ",sortingOrder) : TxnQueries.SHOW_COMPACTION_ORDERBY_CLAUSE;
Review Comment:
reformat, spaces are missing
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #3708: HIVE-26580: SHOW COMPACTIONS should support ordering and limiting fun…
Posted by GitBox <gi...@apache.org>.
deniskuzZ commented on code in PR #3708:
URL: https://github.com/apache/hive/pull/3708#discussion_r1018184988
##########
ql/src/java/org/apache/hadoop/hive/ql/ddl/process/show/compactions/ShowCompactionsOperation.java:
##########
@@ -99,20 +99,26 @@ private ShowCompactRequest getShowCompactioRequest(ShowCompactionsDesc desc) thr
if (isNotEmpty(desc.getPartSpec())) {
request.setPartitionname(AcidUtils.getPartitionName(desc.getPartSpec()));
}
- if(desc.getCompactionId()>0){
+ if(desc.getCompactionId() > 0){
Review Comment:
nit: space
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sonarcloud[bot] commented on pull request #3708: HIVE-26580: SHOW COMPACTIONS should support ordering and limiting fun…
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on PR #3708:
URL: https://github.com/apache/hive/pull/3708#issuecomment-1294553975
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=3708)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3708&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3708&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3708&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=CODE_SMELL) [1 Code Smell](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3708&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3708&metric=duplicated_lines_density&view=list) No Duplication information
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] rkirtir commented on a diff in pull request #3708: HIVE-26580: SHOW COMPACTIONS should support ordering and limiting fun…
Posted by GitBox <gi...@apache.org>.
rkirtir commented on code in PR #3708:
URL: https://github.com/apache/hive/pull/3708#discussion_r1007689934
##########
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java:
##########
@@ -3881,15 +3881,18 @@ public boolean submitForCleanup(CompactionRequest rqst, long highestWriteId, lon
public ShowCompactResponse showCompact(ShowCompactRequest rqst) throws MetaException {
try {
ShowCompactResponse response = new ShowCompactResponse(new ArrayList<>());
- String query = TxnQueries.SHOW_COMPACTION_QUERY + getShowCompactFilterClause(rqst) +
- TxnQueries.SHOW_COMPACTION_ORDERBY_CLAUSE;
+ String query = TxnQueries.SHOW_COMPACTION_QUERY +
+ getShowCompactFilterClause(rqst) +
+ getShowCompactSortingOrderClause(rqst) ;
List<String> params = getShowCompactParamList(rqst);
-
Review Comment:
fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] rkirtir commented on a diff in pull request #3708: HIVE-26580: SHOW COMPACTIONS should support ordering and limiting fun…
Posted by GitBox <gi...@apache.org>.
rkirtir commented on code in PR #3708:
URL: https://github.com/apache/hive/pull/3708#discussion_r1007689626
##########
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java:
##########
@@ -4013,6 +4016,12 @@ private String getShowCompactFilterClause(ShowCompactRequest request) {
" WHERE " + StringUtils.join(" AND ", params) : EMPTY;
}
+ private String getShowCompactSortingOrderClause(ShowCompactRequest request){
+ String sortingOrder = request.getOrder();
+ return isNotBlank(sortingOrder) ? TxnQueries.SHOW_COMPACTION_ORDERBY_CLAUSE + " , " + sortingOrder :
Review Comment:
fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] rkirtir commented on a diff in pull request #3708: HIVE-26580: SHOW COMPACTIONS should support ordering and limiting fun…
Posted by GitBox <gi...@apache.org>.
rkirtir commented on code in PR #3708:
URL: https://github.com/apache/hive/pull/3708#discussion_r1021171760
##########
ql/src/java/org/apache/hadoop/hive/ql/io/AcidUtils.java:
##########
@@ -3438,4 +3452,14 @@ public static CompactionState compactionStateStr2Enum(String inputValue) throws
throw new SemanticException("Unexpected compaction state " + inputValue);
}
}
+
+ public static CompactionColumn compactionColStr2Enum(String inputValue) throws SemanticException {
Review Comment:
removed. Fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sonarcloud[bot] commented on pull request #3708: HIVE-26580: SHOW COMPACTIONS should support ordering and limiting fun…
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on PR #3708:
URL: https://github.com/apache/hive/pull/3708#issuecomment-1314998531
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=3708)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3708&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3708&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3708&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=CODE_SMELL) [2 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3708&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3708&metric=duplicated_lines_density&view=list) No Duplication information
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #3708: HIVE-26580: SHOW COMPACTIONS should support ordering and limiting fun…
Posted by GitBox <gi...@apache.org>.
deniskuzZ commented on code in PR #3708:
URL: https://github.com/apache/hive/pull/3708#discussion_r1022843680
##########
ql/src/java/org/apache/hadoop/hive/ql/ddl/process/show/compactions/ShowCompactionsAnalyzer.java:
##########
@@ -80,15 +83,64 @@ public void analyzeInternal(ASTNode root) throws SemanticException {
case HiveParser.TOK_COMPACT_ID:
compactionId = Long.parseLong(child.getChild(0).getText());
break;
+ case HiveParser.TOK_LIMIT:
+ limit = Short.valueOf((child.getChild(0)).getText());
+ break;
+ case HiveParser.TOK_ORDERBY:
+ orderBy = processSortOrderSpec(child);
+ break;
default:
dbName = stripQuotes(child.getText());
}
}
ShowCompactionsDesc desc = new ShowCompactionsDesc(ctx.getResFile(), compactionId, dbName, tblName, poolName, compactionType,
- compactionStatus, partitionSpec);
+ compactionStatus, partitionSpec, limit, orderBy);
Task<DDLWork> task = TaskFactory.get(new DDLWork(getInputs(), getOutputs(), desc));
rootTasks.add(task);
task.setFetchSource(true);
setFetchTask(createFetchTask(ShowCompactionsDesc.SCHEMA));
}
+
+
+ private String processSortOrderSpec(ASTNode sortNode) {
+ List<PTFInvocationSpec.OrderExpression> orderExp = processOrderSpec(sortNode).getExpressions();
+ Map<String, String> orderByAttributes = orderExp.stream().collect(
+ Collectors.toMap(x -> {
+ return x.getExpression().getChild(0)==null ?
+ CompactionColumn.valueOf(x.getExpression().getText().replaceAll("\'","").toUpperCase()).toString():
+ CompactionColumn.valueOf(x.getExpression().getChild(0).getText()).toString();
+ }, x -> x.getOrder().toString()));
+ return orderByAttributes.entrySet().stream().map(e -> e.getKey() + "\t" + e.getValue()).collect(Collectors.joining(","));
+ }
+ public enum CompactionColumn {
+ COMPACTIONID("CC_ID"),
Review Comment:
why COMPACTIONID not just "ID" ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] rkirtir commented on a diff in pull request #3708: HIVE-26580: SHOW COMPACTIONS should support ordering and limiting fun…
Posted by GitBox <gi...@apache.org>.
rkirtir commented on code in PR #3708:
URL: https://github.com/apache/hive/pull/3708#discussion_r1007690676
##########
ql/src/java/org/apache/hadoop/hive/ql/ddl/process/show/compactions/ShowCompactionsAnalyzer.java:
##########
@@ -80,15 +82,38 @@ public void analyzeInternal(ASTNode root) throws SemanticException {
case HiveParser.TOK_COMPACT_ID:
compactionId = Long.parseLong(child.getChild(0).getText());
break;
+ case HiveParser.TOK_LIMIT:
+ limit = Short.valueOf((child.getChild(0)).getText());
+ break;
+ case HiveParser.TOK_ORDERBY:
+ order = processSortOrderSpec(child);
+ break;
default:
dbName = stripQuotes(child.getText());
}
}
ShowCompactionsDesc desc = new ShowCompactionsDesc(ctx.getResFile(), compactionId, dbName, tblName, poolName, compactionType,
- compactionStatus, partitionSpec);
+ compactionStatus, partitionSpec, limit, order);
Task<DDLWork> task = TaskFactory.get(new DDLWork(getInputs(), getOutputs(), desc));
rootTasks.add(task);
task.setFetchSource(true);
setFetchTask(createFetchTask(ShowCompactionsDesc.SCHEMA));
}
+ private String processSortOrderSpec(ASTNode sortNode)
+ {
+ StringBuilder orderByExp = new StringBuilder();
+ ArrayList<PTFInvocationSpec.OrderExpression> orderExp = processOrderSpec(sortNode).getExpressions();
+ PTFInvocationSpec.OrderExpression exp ;
+ for (int i = 0; i < orderExp.size() - 1; i++) {
Review Comment:
fixed
##########
ql/src/java/org/apache/hadoop/hive/ql/ddl/process/show/compactions/ShowCompactionsAnalyzer.java:
##########
@@ -80,15 +82,38 @@ public void analyzeInternal(ASTNode root) throws SemanticException {
case HiveParser.TOK_COMPACT_ID:
compactionId = Long.parseLong(child.getChild(0).getText());
break;
+ case HiveParser.TOK_LIMIT:
+ limit = Short.valueOf((child.getChild(0)).getText());
+ break;
+ case HiveParser.TOK_ORDERBY:
+ order = processSortOrderSpec(child);
+ break;
default:
dbName = stripQuotes(child.getText());
}
}
ShowCompactionsDesc desc = new ShowCompactionsDesc(ctx.getResFile(), compactionId, dbName, tblName, poolName, compactionType,
- compactionStatus, partitionSpec);
+ compactionStatus, partitionSpec, limit, order);
Task<DDLWork> task = TaskFactory.get(new DDLWork(getInputs(), getOutputs(), desc));
rootTasks.add(task);
task.setFetchSource(true);
setFetchTask(createFetchTask(ShowCompactionsDesc.SCHEMA));
}
+ private String processSortOrderSpec(ASTNode sortNode)
+ {
+ StringBuilder orderByExp = new StringBuilder();
+ ArrayList<PTFInvocationSpec.OrderExpression> orderExp = processOrderSpec(sortNode).getExpressions();
Review Comment:
fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] rkirtir commented on a diff in pull request #3708: HIVE-26580: SHOW COMPACTIONS should support ordering and limiting fun…
Posted by GitBox <gi...@apache.org>.
rkirtir commented on code in PR #3708:
URL: https://github.com/apache/hive/pull/3708#discussion_r1007690900
##########
ql/src/java/org/apache/hadoop/hive/ql/ddl/process/show/compactions/ShowCompactionsAnalyzer.java:
##########
@@ -80,15 +82,38 @@ public void analyzeInternal(ASTNode root) throws SemanticException {
case HiveParser.TOK_COMPACT_ID:
compactionId = Long.parseLong(child.getChild(0).getText());
break;
+ case HiveParser.TOK_LIMIT:
+ limit = Short.valueOf((child.getChild(0)).getText());
+ break;
+ case HiveParser.TOK_ORDERBY:
+ order = processSortOrderSpec(child);
+ break;
default:
dbName = stripQuotes(child.getText());
}
}
ShowCompactionsDesc desc = new ShowCompactionsDesc(ctx.getResFile(), compactionId, dbName, tblName, poolName, compactionType,
- compactionStatus, partitionSpec);
+ compactionStatus, partitionSpec, limit, order);
Task<DDLWork> task = TaskFactory.get(new DDLWork(getInputs(), getOutputs(), desc));
rootTasks.add(task);
task.setFetchSource(true);
setFetchTask(createFetchTask(ShowCompactionsDesc.SCHEMA));
}
+ private String processSortOrderSpec(ASTNode sortNode)
+ {
Review Comment:
fixed
##########
ql/src/java/org/apache/hadoop/hive/ql/ddl/process/show/compactions/ShowCompactionsAnalyzer.java:
##########
@@ -50,6 +50,8 @@ public void analyzeInternal(ASTNode root) throws SemanticException {
String compactionType = null;
String compactionStatus = null;
long compactionId = 0;
+ String order = null;
+ short limit = 0;
Review Comment:
fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] rkirtir commented on a diff in pull request #3708: HIVE-26580: SHOW COMPACTIONS should support ordering and limiting fun…
Posted by GitBox <gi...@apache.org>.
rkirtir commented on code in PR #3708:
URL: https://github.com/apache/hive/pull/3708#discussion_r1007691949
##########
ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java:
##########
@@ -1984,4 +1984,28 @@ public ParseContext getParseContext() {
return pCtx;
}
+ public PTFInvocationSpec.OrderSpec processOrderSpec(ASTNode sortNode) {
+ PTFInvocationSpec.OrderSpec oSpec = new PTFInvocationSpec.OrderSpec();
+ int exprCnt = sortNode.getChildCount();
+ for(int i=0; i < exprCnt; i++) {
+ PTFInvocationSpec.OrderExpression exprSpec = new PTFInvocationSpec.OrderExpression();
+ ASTNode orderSpec = (ASTNode) sortNode.getChild(i);
+ ASTNode nullOrderSpec = (ASTNode) orderSpec.getChild(0);
+ exprSpec.setExpression((ASTNode) nullOrderSpec.getChild(0));
+ if ( orderSpec.getType() == HiveParser.TOK_TABSORTCOLNAMEASC ) {
Review Comment:
fixed
##########
ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java:
##########
@@ -1984,4 +1984,28 @@ public ParseContext getParseContext() {
return pCtx;
}
+ public PTFInvocationSpec.OrderSpec processOrderSpec(ASTNode sortNode) {
+ PTFInvocationSpec.OrderSpec oSpec = new PTFInvocationSpec.OrderSpec();
+ int exprCnt = sortNode.getChildCount();
+ for(int i=0; i < exprCnt; i++) {
Review Comment:
fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #3708: HIVE-26580: SHOW COMPACTIONS should support ordering and limiting fun…
Posted by GitBox <gi...@apache.org>.
deniskuzZ commented on code in PR #3708:
URL: https://github.com/apache/hive/pull/3708#discussion_r1018185710
##########
ql/src/java/org/apache/hadoop/hive/ql/io/AcidUtils.java:
##########
@@ -230,6 +230,20 @@ public boolean accept(Path path) {
}
};
+ public enum CompactionColumn {
+ PARTITIONS("CC_PARTITION"),
Review Comment:
it's single
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #3708: HIVE-26580: SHOW COMPACTIONS should support ordering and limiting fun…
Posted by GitBox <gi...@apache.org>.
deniskuzZ commented on code in PR #3708:
URL: https://github.com/apache/hive/pull/3708#discussion_r1018186960
##########
ql/src/java/org/apache/hadoop/hive/ql/io/AcidUtils.java:
##########
@@ -230,6 +230,20 @@ public boolean accept(Path path) {
}
};
+ public enum CompactionColumn {
+ PARTITIONS("CC_PARTITION"),
+ DBNAME("CC_DATABASE"),
Review Comment:
DB_NAME
##########
ql/src/java/org/apache/hadoop/hive/ql/io/AcidUtils.java:
##########
@@ -230,6 +230,20 @@ public boolean accept(Path path) {
}
};
+ public enum CompactionColumn {
+ PARTITIONS("CC_PARTITION"),
+ DBNAME("CC_DATABASE"),
+ TBNAME("CC_TABLE"),
Review Comment:
TABLE_NAME
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sonarcloud[bot] commented on pull request #3708: HIVE-26580: SHOW COMPACTIONS should support ordering and limiting fun…
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on PR #3708:
URL: https://github.com/apache/hive/pull/3708#issuecomment-1317541761
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=3708)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3708&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3708&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3708&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3708&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3708&metric=duplicated_lines_density&view=list) No Duplication information
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sonarcloud[bot] commented on pull request #3708: HIVE-26580: SHOW COMPACTIONS should support ordering and limiting fun…
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on PR #3708:
URL: https://github.com/apache/hive/pull/3708#issuecomment-1319553438
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=3708)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3708&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3708&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3708&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3708&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3708&metric=duplicated_lines_density&view=list) No Duplication information
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #3708: HIVE-26580: SHOW COMPACTIONS should support ordering and limiting fun…
Posted by GitBox <gi...@apache.org>.
deniskuzZ commented on code in PR #3708:
URL: https://github.com/apache/hive/pull/3708#discussion_r1022843105
##########
ql/src/java/org/apache/hadoop/hive/ql/ddl/process/show/compactions/ShowCompactionsAnalyzer.java:
##########
@@ -80,15 +83,64 @@ public void analyzeInternal(ASTNode root) throws SemanticException {
case HiveParser.TOK_COMPACT_ID:
compactionId = Long.parseLong(child.getChild(0).getText());
break;
+ case HiveParser.TOK_LIMIT:
+ limit = Short.valueOf((child.getChild(0)).getText());
+ break;
+ case HiveParser.TOK_ORDERBY:
+ orderBy = processSortOrderSpec(child);
+ break;
default:
dbName = stripQuotes(child.getText());
}
}
ShowCompactionsDesc desc = new ShowCompactionsDesc(ctx.getResFile(), compactionId, dbName, tblName, poolName, compactionType,
- compactionStatus, partitionSpec);
+ compactionStatus, partitionSpec, limit, orderBy);
Task<DDLWork> task = TaskFactory.get(new DDLWork(getInputs(), getOutputs(), desc));
rootTasks.add(task);
task.setFetchSource(true);
setFetchTask(createFetchTask(ShowCompactionsDesc.SCHEMA));
}
+
+
+ private String processSortOrderSpec(ASTNode sortNode) {
+ List<PTFInvocationSpec.OrderExpression> orderExp = processOrderSpec(sortNode).getExpressions();
+ Map<String, String> orderByAttributes = orderExp.stream().collect(
+ Collectors.toMap(x -> {
+ return x.getExpression().getChild(0)==null ?
+ CompactionColumn.valueOf(x.getExpression().getText().replaceAll("\'","").toUpperCase()).toString():
+ CompactionColumn.valueOf(x.getExpression().getChild(0).getText()).toString();
+ }, x -> x.getOrder().toString()));
+ return orderByAttributes.entrySet().stream().map(e -> e.getKey() + "\t" + e.getValue()).collect(Collectors.joining(","));
+ }
+ public enum CompactionColumn {
Review Comment:
should it be public? new line is missing
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] rkirtir commented on a diff in pull request #3708: HIVE-26580: SHOW COMPACTIONS should support ordering and limiting fun…
Posted by GitBox <gi...@apache.org>.
rkirtir commented on code in PR #3708:
URL: https://github.com/apache/hive/pull/3708#discussion_r1021171366
##########
ql/src/java/org/apache/hadoop/hive/ql/ddl/process/show/compactions/ShowCompactionsOperation.java:
##########
@@ -99,20 +99,26 @@ private ShowCompactRequest getShowCompactioRequest(ShowCompactionsDesc desc) thr
if (isNotEmpty(desc.getPartSpec())) {
request.setPartitionname(AcidUtils.getPartitionName(desc.getPartSpec()));
}
- if(desc.getCompactionId()>0){
+ if(desc.getCompactionId() > 0){
request.setId(desc.getCompactionId());
}
+ if (desc.getLimit() > -1) {
+ request.setLimit(desc.getLimit());
+ }
+ if (isNotBlank(desc.getOrderBy())) {
+ request.setOrder(desc.getOrderBy());
+ }
return request;
}
private void writeHeader(DataOutputStream os) throws IOException {
os.writeBytes("CompactionId");
os.write(Utilities.tabCode);
- os.writeBytes("Database");
+ os.writeBytes("Dbname");
Review Comment:
fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sonarcloud[bot] commented on pull request #3708: HIVE-26580: SHOW COMPACTIONS should support ordering and limiting fun…
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on PR #3708:
URL: https://github.com/apache/hive/pull/3708#issuecomment-1314178489
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=3708)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3708&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3708&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3708&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=CODE_SMELL) [1 Code Smell](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3708&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3708&metric=duplicated_lines_density&view=list) No Duplication information
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #3708: HIVE-26580: SHOW COMPACTIONS should support ordering and limiting fun…
Posted by GitBox <gi...@apache.org>.
deniskuzZ commented on code in PR #3708:
URL: https://github.com/apache/hive/pull/3708#discussion_r1022847138
##########
ql/src/java/org/apache/hadoop/hive/ql/ddl/process/show/compactions/ShowCompactionsAnalyzer.java:
##########
@@ -80,15 +83,64 @@ public void analyzeInternal(ASTNode root) throws SemanticException {
case HiveParser.TOK_COMPACT_ID:
compactionId = Long.parseLong(child.getChild(0).getText());
break;
+ case HiveParser.TOK_LIMIT:
+ limit = Short.valueOf((child.getChild(0)).getText());
+ break;
+ case HiveParser.TOK_ORDERBY:
+ orderBy = processSortOrderSpec(child);
+ break;
default:
dbName = stripQuotes(child.getText());
}
}
ShowCompactionsDesc desc = new ShowCompactionsDesc(ctx.getResFile(), compactionId, dbName, tblName, poolName, compactionType,
- compactionStatus, partitionSpec);
+ compactionStatus, partitionSpec, limit, orderBy);
Task<DDLWork> task = TaskFactory.get(new DDLWork(getInputs(), getOutputs(), desc));
rootTasks.add(task);
task.setFetchSource(true);
setFetchTask(createFetchTask(ShowCompactionsDesc.SCHEMA));
}
+
+
+ private String processSortOrderSpec(ASTNode sortNode) {
+ List<PTFInvocationSpec.OrderExpression> orderExp = processOrderSpec(sortNode).getExpressions();
+ Map<String, String> orderByAttributes = orderExp.stream().collect(
+ Collectors.toMap(x -> {
+ return x.getExpression().getChild(0)==null ?
+ CompactionColumn.valueOf(x.getExpression().getText().replaceAll("\'","").toUpperCase()).toString():
+ CompactionColumn.valueOf(x.getExpression().getChild(0).getText()).toString();
+ }, x -> x.getOrder().toString()));
+ return orderByAttributes.entrySet().stream().map(e -> e.getKey() + "\t" + e.getValue()).collect(Collectors.joining(","));
+ }
+ public enum CompactionColumn {
+ COMPACTIONID("CC_ID"),
+ PARTITION("CC_PARTITION"),
+ DATABASE("CC_DATABASE"),
+ TABLE("CC_TABLE"),
+ TYPE("CC_TYPE"),
+ STATE("CC_STATE"),
+ WORKER_HOST("CC_WORKER_ID"),
+ ENQUEUE_TIME("CC_ENQUEUE_TIME"),
+ POOLNAME("CC_POOL_NAME"),
+ WORKER("CC_WORKER_VERSION"),
+ START_TIME("CC_START"),
+ TXNID ("CC_TXN_ID"),
+ HADOOP_JOB_ID("CC_HADOOP_JOB_ID"),
+ NEXT_TXN_ID("CC_NEXT_TXN_ID"),
+ HIGHEST_WRITE_ID("CC_HIGHEST_WRITE_ID");
+
+ private final String colVal;
+ CompactionColumn(String colVal) {
+ this.colVal = colVal;
+ }
+ public String getColValue() {
Review Comment:
redundant
##########
ql/src/java/org/apache/hadoop/hive/ql/ddl/process/show/compactions/ShowCompactionsAnalyzer.java:
##########
@@ -80,15 +83,64 @@ public void analyzeInternal(ASTNode root) throws SemanticException {
case HiveParser.TOK_COMPACT_ID:
compactionId = Long.parseLong(child.getChild(0).getText());
break;
+ case HiveParser.TOK_LIMIT:
+ limit = Short.valueOf((child.getChild(0)).getText());
+ break;
+ case HiveParser.TOK_ORDERBY:
+ orderBy = processSortOrderSpec(child);
+ break;
default:
dbName = stripQuotes(child.getText());
}
}
ShowCompactionsDesc desc = new ShowCompactionsDesc(ctx.getResFile(), compactionId, dbName, tblName, poolName, compactionType,
- compactionStatus, partitionSpec);
+ compactionStatus, partitionSpec, limit, orderBy);
Task<DDLWork> task = TaskFactory.get(new DDLWork(getInputs(), getOutputs(), desc));
rootTasks.add(task);
task.setFetchSource(true);
setFetchTask(createFetchTask(ShowCompactionsDesc.SCHEMA));
}
+
+
+ private String processSortOrderSpec(ASTNode sortNode) {
+ List<PTFInvocationSpec.OrderExpression> orderExp = processOrderSpec(sortNode).getExpressions();
+ Map<String, String> orderByAttributes = orderExp.stream().collect(
+ Collectors.toMap(x -> {
+ return x.getExpression().getChild(0)==null ?
+ CompactionColumn.valueOf(x.getExpression().getText().replaceAll("\'","").toUpperCase()).toString():
+ CompactionColumn.valueOf(x.getExpression().getChild(0).getText()).toString();
+ }, x -> x.getOrder().toString()));
+ return orderByAttributes.entrySet().stream().map(e -> e.getKey() + "\t" + e.getValue()).collect(Collectors.joining(","));
+ }
+ public enum CompactionColumn {
+ COMPACTIONID("CC_ID"),
+ PARTITION("CC_PARTITION"),
+ DATABASE("CC_DATABASE"),
+ TABLE("CC_TABLE"),
+ TYPE("CC_TYPE"),
+ STATE("CC_STATE"),
+ WORKER_HOST("CC_WORKER_ID"),
+ ENQUEUE_TIME("CC_ENQUEUE_TIME"),
+ POOLNAME("CC_POOL_NAME"),
+ WORKER("CC_WORKER_VERSION"),
+ START_TIME("CC_START"),
+ TXNID ("CC_TXN_ID"),
+ HADOOP_JOB_ID("CC_HADOOP_JOB_ID"),
+ NEXT_TXN_ID("CC_NEXT_TXN_ID"),
+ HIGHEST_WRITE_ID("CC_HIGHEST_WRITE_ID");
+
+ private final String colVal;
+ CompactionColumn(String colVal) {
Review Comment:
new line
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #3708: HIVE-26580: SHOW COMPACTIONS should support ordering and limiting fun…
Posted by GitBox <gi...@apache.org>.
deniskuzZ commented on code in PR #3708:
URL: https://github.com/apache/hive/pull/3708#discussion_r1022845969
##########
ql/src/java/org/apache/hadoop/hive/ql/ddl/process/show/compactions/ShowCompactionsAnalyzer.java:
##########
@@ -80,15 +83,64 @@ public void analyzeInternal(ASTNode root) throws SemanticException {
case HiveParser.TOK_COMPACT_ID:
compactionId = Long.parseLong(child.getChild(0).getText());
break;
+ case HiveParser.TOK_LIMIT:
+ limit = Short.valueOf((child.getChild(0)).getText());
+ break;
+ case HiveParser.TOK_ORDERBY:
+ orderBy = processSortOrderSpec(child);
+ break;
default:
dbName = stripQuotes(child.getText());
}
}
ShowCompactionsDesc desc = new ShowCompactionsDesc(ctx.getResFile(), compactionId, dbName, tblName, poolName, compactionType,
- compactionStatus, partitionSpec);
+ compactionStatus, partitionSpec, limit, orderBy);
Task<DDLWork> task = TaskFactory.get(new DDLWork(getInputs(), getOutputs(), desc));
rootTasks.add(task);
task.setFetchSource(true);
setFetchTask(createFetchTask(ShowCompactionsDesc.SCHEMA));
}
+
+
+ private String processSortOrderSpec(ASTNode sortNode) {
Review Comment:
please format this method
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sonarcloud[bot] commented on pull request #3708: HIVE-26580: SHOW COMPACTIONS should support ordering and limiting fun…
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on PR #3708:
URL: https://github.com/apache/hive/pull/3708#issuecomment-1321174453
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=3708)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3708&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3708&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3708&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3708&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3708&metric=duplicated_lines_density&view=list) No Duplication information
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sonarcloud[bot] commented on pull request #3708: HIVE-26580: SHOW COMPACTIONS should support ordering and limiting fun…
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on PR #3708:
URL: https://github.com/apache/hive/pull/3708#issuecomment-1321508436
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=3708)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3708&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3708&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3708&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3708&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3708&metric=duplicated_lines_density&view=list) No Duplication information
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] rkirtir commented on a diff in pull request #3708: HIVE-26580: SHOW COMPACTIONS should support ordering and limiting fun…
Posted by GitBox <gi...@apache.org>.
rkirtir commented on code in PR #3708:
URL: https://github.com/apache/hive/pull/3708#discussion_r1023522073
##########
ql/src/java/org/apache/hadoop/hive/ql/ddl/process/show/compactions/ShowCompactionsOperation.java:
##########
@@ -102,6 +102,12 @@ private ShowCompactRequest getShowCompactioRequest(ShowCompactionsDesc desc) thr
if(desc.getCompactionId()>0){
request.setId(desc.getCompactionId());
}
+ if (desc.getLimit() > -1) {
Review Comment:
fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #3708: HIVE-26580: SHOW COMPACTIONS should support ordering and limiting fun…
Posted by GitBox <gi...@apache.org>.
deniskuzZ commented on code in PR #3708:
URL: https://github.com/apache/hive/pull/3708#discussion_r1006701744
##########
ql/src/test/org/apache/hadoop/hive/ql/TestTxnCommands.java:
##########
@@ -2388,6 +2396,65 @@ public void testShowCompactionInputValidation() throws Exception {
"STATUS 'ready for clean'");//validates compaction status
}
+ @Test
+ public void testShowCompactionFilterSortingAndLimit()throws Exception {
+ runStatementOnDriver("drop database if exists mydb1 cascade");
+ runStatementOnDriver("create database mydb1");
+ runStatementOnDriver("create table mydb1.tbl0 " + "(a int, b int) partitioned by (p string) clustered by (a) into " +
+ BUCKET_COUNT + " buckets stored as orc TBLPROPERTIES ('transactional'='true')");
+ runStatementOnDriver("insert into mydb1.tbl0" + " PARTITION(p) " +
+ " values(1,2,'p1'),(3,4,'p1'),(1,2,'p2'),(3,4,'p2'),(1,2,'p3'),(3,4,'p3')");
+ runStatementOnDriver("alter table mydb1.tbl0" + " PARTITION(p='p1') compact 'MAJOR'");
+ TestTxnCommands2.runWorker(hiveConf);
+ runStatementOnDriver("alter table mydb1.tbl0" + " PARTITION(p='p2') compact 'MAJOR'");
+ TestTxnCommands2.runWorker(hiveConf);
+
+
+ runStatementOnDriver("drop database if exists mydb cascade");
+ runStatementOnDriver("create database mydb");
+ runStatementOnDriver("create table mydb.tbl " + "(a int, b int) partitioned by (ds string) clustered by (a) into " +
+ BUCKET_COUNT + " buckets stored as orc TBLPROPERTIES ('transactional'='true')");
+ runStatementOnDriver("insert into mydb.tbl" + " PARTITION(ds) " +
+ " values(1,2,'mon'),(3,4,'tue'),(1,2,'mon'),(3,4,'tue'),(1,2,'wed'),(3,4,'wed')");
+ runStatementOnDriver("alter table mydb.tbl" + " PARTITION(ds='mon') compact 'MAJOR'");
+ TestTxnCommands2.runWorker(hiveConf);
+ runStatementOnDriver("alter table mydb.tbl" + " PARTITION(ds='tue') compact 'MAJOR'");
+ TestTxnCommands2.runWorker(hiveConf);
+
+ runStatementOnDriver("create table mydb.tbl2 " + "(a int, b int) partitioned by (dm string) clustered by (a) into " +
+ BUCKET_COUNT + " buckets stored as orc TBLPROPERTIES ('transactional'='true')");
+ runStatementOnDriver("insert into mydb.tbl2" + " PARTITION(dm) " +
+ " values(1,2,'xxx'),(3,4,'xxx'),(1,2,'yyy'),(3,4,'yyy'),(1,2,'zzz'),(3,4,'zzz')");
+ runStatementOnDriver("alter table mydb.tbl2" + " PARTITION(dm='yyy') compact 'MAJOR'");
+ TestTxnCommands2.runWorker(hiveConf);
+ runStatementOnDriver("alter table mydb.tbl2" + " PARTITION(dm='zzz') compact 'MAJOR'");
+ TestTxnCommands2.runWorker(hiveConf);
+
+ ShowCompactResponse rsp = txnHandler.showCompact(new ShowCompactRequest());
+
+ //includes Header row
+ List<String> r = runStatementOnDriver("SHOW COMPACTIONS");
+ Assert.assertEquals(rsp.getCompacts().size() + 1, r.size());
+ r = runStatementOnDriver("SHOW COMPACTIONS LIMIT 3");
+ Assert.assertEquals(4, r.size());
+ r = runStatementOnDriver("SHOW COMPACTIONS SCHEMA mydb TYPE 'MAJOR' LIMIT 2");
+ Assert.assertEquals(3, r.size());
+
+ r = runStatementOnDriver("SHOW COMPACTIONS SCHEMA mydb TYPE 'MAJOR' ORDER BY CC_TABLE DESC, CC_PARTITION ASC");
Review Comment:
there is no `CC_` prefix in the show compactions output. That would be confusing
##########
ql/src/java/org/apache/hadoop/hive/ql/ddl/process/show/compactions/ShowCompactionsAnalyzer.java:
##########
@@ -50,6 +50,8 @@ public void analyzeInternal(ASTNode root) throws SemanticException {
String compactionType = null;
String compactionStatus = null;
long compactionId = 0;
+ String order = null;
+ short limit = 0;
Review Comment:
maybe -1, meaning no limit?
##########
ql/src/java/org/apache/hadoop/hive/ql/ddl/process/show/compactions/ShowCompactionsAnalyzer.java:
##########
@@ -80,15 +82,38 @@ public void analyzeInternal(ASTNode root) throws SemanticException {
case HiveParser.TOK_COMPACT_ID:
compactionId = Long.parseLong(child.getChild(0).getText());
break;
+ case HiveParser.TOK_LIMIT:
+ limit = Short.valueOf((child.getChild(0)).getText());
+ break;
+ case HiveParser.TOK_ORDERBY:
+ order = processSortOrderSpec(child);
+ break;
default:
dbName = stripQuotes(child.getText());
}
}
ShowCompactionsDesc desc = new ShowCompactionsDesc(ctx.getResFile(), compactionId, dbName, tblName, poolName, compactionType,
- compactionStatus, partitionSpec);
+ compactionStatus, partitionSpec, limit, order);
Task<DDLWork> task = TaskFactory.get(new DDLWork(getInputs(), getOutputs(), desc));
rootTasks.add(task);
task.setFetchSource(true);
setFetchTask(createFetchTask(ShowCompactionsDesc.SCHEMA));
}
+ private String processSortOrderSpec(ASTNode sortNode)
+ {
+ StringBuilder orderByExp = new StringBuilder();
+ ArrayList<PTFInvocationSpec.OrderExpression> orderExp = processOrderSpec(sortNode).getExpressions();
+ PTFInvocationSpec.OrderExpression exp ;
+ for (int i = 0; i < orderExp.size() - 1; i++) {
Review Comment:
to avoid duplication, you could first collect all of the expressions into a list and then use StringUtils.join(" ,", list)
##########
ql/src/java/org/apache/hadoop/hive/ql/ddl/process/show/compactions/ShowCompactionsDesc.java:
##########
@@ -93,6 +97,16 @@ public String getCompactionStatus() {
public Map<String, String> getPartSpec() {
return partSpec;
}
+ @Explain(displayName = "limit", explainLevels = {Level.USER, Level.DEFAULT, Level.EXTENDED})
+ public short getLimit() {
+ return limit;
+ }
+
+ @Explain(displayName = "order", explainLevels = {Level.USER, Level.DEFAULT, Level.EXTENDED})
+ public String getOrder() {
Review Comment:
getOrderBy()
##########
ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java:
##########
@@ -1984,4 +1984,28 @@ public ParseContext getParseContext() {
return pCtx;
}
+ public PTFInvocationSpec.OrderSpec processOrderSpec(ASTNode sortNode) {
+ PTFInvocationSpec.OrderSpec oSpec = new PTFInvocationSpec.OrderSpec();
+ int exprCnt = sortNode.getChildCount();
+ for(int i=0; i < exprCnt; i++) {
+ PTFInvocationSpec.OrderExpression exprSpec = new PTFInvocationSpec.OrderExpression();
+ ASTNode orderSpec = (ASTNode) sortNode.getChild(i);
+ ASTNode nullOrderSpec = (ASTNode) orderSpec.getChild(0);
+ exprSpec.setExpression((ASTNode) nullOrderSpec.getChild(0));
+ if ( orderSpec.getType() == HiveParser.TOK_TABSORTCOLNAMEASC ) {
+ exprSpec.setOrder(org.apache.hadoop.hive.ql.parse.PTFInvocationSpec.Order.ASC);
+ }
+ else {
+ exprSpec.setOrder(org.apache.hadoop.hive.ql.parse.PTFInvocationSpec.Order.DESC);
+ }
+ if ( nullOrderSpec.getType() == HiveParser.TOK_NULLS_FIRST ) {
Review Comment:
nit: remove extra space
##########
ql/src/test/org/apache/hadoop/hive/ql/TestTxnCommands.java:
##########
@@ -2155,6 +2155,11 @@ public void testShowCompactions() throws Exception {
ShowCompactResponse rsp = txnHandler.showCompact(new ShowCompactRequest());
List<String> r = runStatementOnDriver("SHOW COMPACTIONS");
Assert.assertEquals(rsp.getCompacts().size()+1, r.size());//includes Header row
+ r = runStatementOnDriver("SHOW COMPACTIONS LIMIT 3");
Review Comment:
why do we need this if we have `testShowCompactionFilterSortingAndLimit`?
##########
standalone-metastore/metastore-common/src/main/thrift/hive_metastore.thrift:
##########
@@ -1351,7 +1351,9 @@ struct ShowCompactRequest {
4: required string tablename,
5: optional string partitionname,
6: required CompactionType type,
- 7: required string state
+ 7: required string state,
+ 8: optional i64 limit,
+ 9: optional string order
Review Comment:
wouldn't it be better to have map here, Map<String (column), String (order)> ordeerBy?
##########
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java:
##########
@@ -4013,6 +4016,12 @@ private String getShowCompactFilterClause(ShowCompactRequest request) {
" WHERE " + StringUtils.join(" AND ", params) : EMPTY;
}
+ private String getShowCompactSortingOrderClause(ShowCompactRequest request){
+ String sortingOrder = request.getOrder();
+ return isNotBlank(sortingOrder) ? TxnQueries.SHOW_COMPACTION_ORDERBY_CLAUSE + " , " + sortingOrder :
Review Comment:
if custom order is provided we should use that and don't append the default one
##########
ql/src/java/org/apache/hadoop/hive/ql/ddl/process/show/compactions/ShowCompactionsAnalyzer.java:
##########
@@ -80,15 +82,38 @@ public void analyzeInternal(ASTNode root) throws SemanticException {
case HiveParser.TOK_COMPACT_ID:
compactionId = Long.parseLong(child.getChild(0).getText());
break;
+ case HiveParser.TOK_LIMIT:
+ limit = Short.valueOf((child.getChild(0)).getText());
+ break;
+ case HiveParser.TOK_ORDERBY:
+ order = processSortOrderSpec(child);
+ break;
default:
dbName = stripQuotes(child.getText());
}
}
ShowCompactionsDesc desc = new ShowCompactionsDesc(ctx.getResFile(), compactionId, dbName, tblName, poolName, compactionType,
- compactionStatus, partitionSpec);
+ compactionStatus, partitionSpec, limit, order);
Task<DDLWork> task = TaskFactory.get(new DDLWork(getInputs(), getOutputs(), desc));
rootTasks.add(task);
task.setFetchSource(true);
setFetchTask(createFetchTask(ShowCompactionsDesc.SCHEMA));
}
+ private String processSortOrderSpec(ASTNode sortNode)
+ {
Review Comment:
formatting
##########
ql/src/java/org/apache/hadoop/hive/ql/ddl/process/show/compactions/ShowCompactionsDesc.java:
##########
@@ -45,10 +45,12 @@ public class ShowCompactionsDesc implements DDLDesc, Serializable {
private final String compactionType;
private final String compactionStatus;
private final Map<String, String> partSpec;
+ private final short limit;
+ private final String order;
public ShowCompactionsDesc(Path resFile, long compactionId, String dbName, String tbName, String poolName, String compactionType,
- String compactionStatus, Map<String, String> partSpec) {
+ String compactionStatus, Map<String, String> partSpec, short limit, String order) {
Review Comment:
orderBy
##########
ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java:
##########
@@ -1984,4 +1984,28 @@ public ParseContext getParseContext() {
return pCtx;
}
+ public PTFInvocationSpec.OrderSpec processOrderSpec(ASTNode sortNode) {
+ PTFInvocationSpec.OrderSpec oSpec = new PTFInvocationSpec.OrderSpec();
+ int exprCnt = sortNode.getChildCount();
+ for(int i=0; i < exprCnt; i++) {
+ PTFInvocationSpec.OrderExpression exprSpec = new PTFInvocationSpec.OrderExpression();
+ ASTNode orderSpec = (ASTNode) sortNode.getChild(i);
+ ASTNode nullOrderSpec = (ASTNode) orderSpec.getChild(0);
+ exprSpec.setExpression((ASTNode) nullOrderSpec.getChild(0));
+ if ( orderSpec.getType() == HiveParser.TOK_TABSORTCOLNAMEASC ) {
Review Comment:
nit: remove extra space
##########
ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java:
##########
@@ -1984,4 +1984,28 @@ public ParseContext getParseContext() {
return pCtx;
}
+ public PTFInvocationSpec.OrderSpec processOrderSpec(ASTNode sortNode) {
+ PTFInvocationSpec.OrderSpec oSpec = new PTFInvocationSpec.OrderSpec();
+ int exprCnt = sortNode.getChildCount();
+ for(int i=0; i < exprCnt; i++) {
Review Comment:
nit: space
##########
ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java:
##########
@@ -1984,4 +1984,28 @@ public ParseContext getParseContext() {
return pCtx;
}
+ public PTFInvocationSpec.OrderSpec processOrderSpec(ASTNode sortNode) {
+ PTFInvocationSpec.OrderSpec oSpec = new PTFInvocationSpec.OrderSpec();
Review Comment:
use static import
##########
ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java:
##########
@@ -1984,4 +1984,28 @@ public ParseContext getParseContext() {
return pCtx;
}
+ public PTFInvocationSpec.OrderSpec processOrderSpec(ASTNode sortNode) {
+ PTFInvocationSpec.OrderSpec oSpec = new PTFInvocationSpec.OrderSpec();
+ int exprCnt = sortNode.getChildCount();
+ for(int i=0; i < exprCnt; i++) {
+ PTFInvocationSpec.OrderExpression exprSpec = new PTFInvocationSpec.OrderExpression();
+ ASTNode orderSpec = (ASTNode) sortNode.getChild(i);
+ ASTNode nullOrderSpec = (ASTNode) orderSpec.getChild(0);
+ exprSpec.setExpression((ASTNode) nullOrderSpec.getChild(0));
+ if ( orderSpec.getType() == HiveParser.TOK_TABSORTCOLNAMEASC ) {
+ exprSpec.setOrder(org.apache.hadoop.hive.ql.parse.PTFInvocationSpec.Order.ASC);
Review Comment:
can we use static import of `org.apache.hadoop.hive.ql.parse.PTFInvocationSpec`
##########
ql/src/test/org/apache/hadoop/hive/ql/TestTxnCommands.java:
##########
@@ -2155,6 +2155,11 @@ public void testShowCompactions() throws Exception {
ShowCompactResponse rsp = txnHandler.showCompact(new ShowCompactRequest());
List<String> r = runStatementOnDriver("SHOW COMPACTIONS");
Assert.assertEquals(rsp.getCompacts().size()+1, r.size());//includes Header row
+ r = runStatementOnDriver("SHOW COMPACTIONS LIMIT 3");
+ Assert.assertEquals(4, r.size());
+
+ r = runStatementOnDriver("SHOW COMPACTIONS SCHEMA mydb1 TYPE 'MAJOR' ORDER BY CC_PARTITION ASC , CC_TABLE DESC");
Review Comment:
why do we need this if we have `testShowCompactionFilterSortingAndLimit`?
##########
ql/src/java/org/apache/hadoop/hive/ql/ddl/process/show/compactions/ShowCompactionsAnalyzer.java:
##########
@@ -80,15 +82,38 @@ public void analyzeInternal(ASTNode root) throws SemanticException {
case HiveParser.TOK_COMPACT_ID:
compactionId = Long.parseLong(child.getChild(0).getText());
break;
+ case HiveParser.TOK_LIMIT:
+ limit = Short.valueOf((child.getChild(0)).getText());
+ break;
+ case HiveParser.TOK_ORDERBY:
+ order = processSortOrderSpec(child);
+ break;
default:
dbName = stripQuotes(child.getText());
}
}
ShowCompactionsDesc desc = new ShowCompactionsDesc(ctx.getResFile(), compactionId, dbName, tblName, poolName, compactionType,
- compactionStatus, partitionSpec);
+ compactionStatus, partitionSpec, limit, order);
Task<DDLWork> task = TaskFactory.get(new DDLWork(getInputs(), getOutputs(), desc));
rootTasks.add(task);
task.setFetchSource(true);
setFetchTask(createFetchTask(ShowCompactionsDesc.SCHEMA));
}
+ private String processSortOrderSpec(ASTNode sortNode)
+ {
+ StringBuilder orderByExp = new StringBuilder();
+ ArrayList<PTFInvocationSpec.OrderExpression> orderExp = processOrderSpec(sortNode).getExpressions();
Review Comment:
use List interface
##########
ql/src/test/org/apache/hadoop/hive/ql/TestTxnCommands.java:
##########
@@ -2187,6 +2192,9 @@ public void testShowCompactions() throws Exception {
Assert.assertTrue(p.matcher(r.get(i)).matches());
}
+ r = runStatementOnDriver("SHOW COMPACTIONS SCHEMA mydb1 TYPE 'MAJOR' LIMIT 2");
Review Comment:
why do we need this if we have `testShowCompactionFilterSortingAndLimit`?
##########
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java:
##########
@@ -3881,15 +3881,18 @@ public boolean submitForCleanup(CompactionRequest rqst, long highestWriteId, lon
public ShowCompactResponse showCompact(ShowCompactRequest rqst) throws MetaException {
try {
ShowCompactResponse response = new ShowCompactResponse(new ArrayList<>());
- String query = TxnQueries.SHOW_COMPACTION_QUERY + getShowCompactFilterClause(rqst) +
- TxnQueries.SHOW_COMPACTION_ORDERBY_CLAUSE;
+ String query = TxnQueries.SHOW_COMPACTION_QUERY +
+ getShowCompactFilterClause(rqst) +
+ getShowCompactSortingOrderClause(rqst) ;
List<String> params = getShowCompactParamList(rqst);
-
Review Comment:
keep the new line here
##########
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java:
##########
@@ -3881,15 +3881,18 @@ public boolean submitForCleanup(CompactionRequest rqst, long highestWriteId, lon
public ShowCompactResponse showCompact(ShowCompactRequest rqst) throws MetaException {
try {
ShowCompactResponse response = new ShowCompactResponse(new ArrayList<>());
- String query = TxnQueries.SHOW_COMPACTION_QUERY + getShowCompactFilterClause(rqst) +
- TxnQueries.SHOW_COMPACTION_ORDERBY_CLAUSE;
+ String query = TxnQueries.SHOW_COMPACTION_QUERY +
+ getShowCompactFilterClause(rqst) +
+ getShowCompactSortingOrderClause(rqst) ;
List<String> params = getShowCompactParamList(rqst);
-
try (Connection dbConn = getDbConn(Connection.TRANSACTION_READ_COMMITTED);
PreparedStatement stmt = sqlGenerator.prepareStmtWithParameters(dbConn, query, params)) {
if (rqst.isSetId()) {
stmt.setLong(1, rqst.getId());
}
+ int rowLimit = (int) rqst.getLimit();
+ if (rowLimit > 0)
Review Comment:
wrap with braces
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sonarcloud[bot] commented on pull request #3708: HIVE-26580: SHOW COMPACTIONS should support ordering and limiting fun…
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on PR #3708:
URL: https://github.com/apache/hive/pull/3708#issuecomment-1293070734
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=3708)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3708&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3708&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3708&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3708&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3708&metric=duplicated_lines_density&view=list) No Duplication information
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #3708: HIVE-26580: SHOW COMPACTIONS should support ordering and limiting fun…
Posted by GitBox <gi...@apache.org>.
deniskuzZ commented on code in PR #3708:
URL: https://github.com/apache/hive/pull/3708#discussion_r1018185710
##########
ql/src/java/org/apache/hadoop/hive/ql/io/AcidUtils.java:
##########
@@ -230,6 +230,20 @@ public boolean accept(Path path) {
}
};
+ public enum CompactionColumn {
+ PARTITIONS("CC_PARTITION"),
Review Comment:
it's single
##########
ql/src/java/org/apache/hadoop/hive/ql/io/AcidUtils.java:
##########
@@ -230,6 +230,20 @@ public boolean accept(Path path) {
}
};
+ public enum CompactionColumn {
+ PARTITIONS("CC_PARTITION"),
+ DBNAME("CC_DATABASE"),
Review Comment:
DB_NAME
##########
ql/src/java/org/apache/hadoop/hive/ql/io/AcidUtils.java:
##########
@@ -230,6 +230,20 @@ public boolean accept(Path path) {
}
};
+ public enum CompactionColumn {
+ PARTITIONS("CC_PARTITION"),
+ DBNAME("CC_DATABASE"),
+ TBNAME("CC_TABLE"),
Review Comment:
TABLE_NAME
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #3708: HIVE-26580: SHOW COMPACTIONS should support ordering and limiting fun…
Posted by GitBox <gi...@apache.org>.
deniskuzZ commented on code in PR #3708:
URL: https://github.com/apache/hive/pull/3708#discussion_r1018198739
##########
ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java:
##########
@@ -1984,4 +1991,27 @@ public ParseContext getParseContext() {
return pCtx;
}
+ public PTFInvocationSpec.OrderSpec processOrderSpec(ASTNode sortNode) {
Review Comment:
why not keep it in ShowCompactionsAnalyzer?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #3708: HIVE-26580: SHOW COMPACTIONS should support ordering and limiting fun…
Posted by GitBox <gi...@apache.org>.
deniskuzZ commented on code in PR #3708:
URL: https://github.com/apache/hive/pull/3708#discussion_r1018194153
##########
ql/src/java/org/apache/hadoop/hive/ql/io/AcidUtils.java:
##########
@@ -230,6 +230,20 @@ public boolean accept(Path path) {
}
};
+ public enum CompactionColumn {
Review Comment:
If you want to create a column mapping function, please keep it localized (i.e. ShowCompactionsAnalyzer) + add mapping for "SCHEMA" -> "DATABASE"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #3708: HIVE-26580: SHOW COMPACTIONS should support ordering and limiting fun…
Posted by GitBox <gi...@apache.org>.
deniskuzZ commented on code in PR #3708:
URL: https://github.com/apache/hive/pull/3708#discussion_r1018188264
##########
ql/src/java/org/apache/hadoop/hive/ql/io/AcidUtils.java:
##########
@@ -230,6 +230,20 @@ public boolean accept(Path path) {
}
};
+ public enum CompactionColumn {
+ PARTITIONS("CC_PARTITION"),
+ DBNAME("CC_DATABASE"),
+ TBNAME("CC_TABLE"),
+ TYPE("CC_TYPE"),
+ STATE("CC_STATE");
+ private final String colVal;
+ CompactionColumn(String colVal) {
+ this.colVal = colVal;
+ }
+ public String getValue() {
Review Comment:
toString()
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #3708: HIVE-26580: SHOW COMPACTIONS should support ordering and limiting fun…
Posted by GitBox <gi...@apache.org>.
deniskuzZ commented on code in PR #3708:
URL: https://github.com/apache/hive/pull/3708#discussion_r1018224069
##########
ql/src/java/org/apache/hadoop/hive/ql/io/AcidUtils.java:
##########
@@ -230,6 +230,20 @@ public boolean accept(Path path) {
}
};
+ public enum CompactionColumn {
+ PARTITIONS("CC_PARTITION"),
+ DBNAME("CC_DATABASE"),
+ TBNAME("CC_TABLE"),
+ TYPE("CC_TYPE"),
+ STATE("CC_STATE");
+ private final String colVal;
+ CompactionColumn(String colVal) {
+ this.colVal = colVal;
+ }
+ public String getValue() {
Review Comment:
@Override toString()
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] rkirtir commented on a diff in pull request #3708: HIVE-26580: SHOW COMPACTIONS should support ordering and limiting fun…
Posted by GitBox <gi...@apache.org>.
rkirtir commented on code in PR #3708:
URL: https://github.com/apache/hive/pull/3708#discussion_r1007691772
##########
ql/src/test/org/apache/hadoop/hive/ql/TestTxnCommands.java:
##########
@@ -2187,6 +2192,9 @@ public void testShowCompactions() throws Exception {
Assert.assertTrue(p.matcher(r.get(i)).matches());
}
+ r = runStatementOnDriver("SHOW COMPACTIONS SCHEMA mydb1 TYPE 'MAJOR' LIMIT 2");
Review Comment:
fixed
##########
ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java:
##########
@@ -1984,4 +1984,28 @@ public ParseContext getParseContext() {
return pCtx;
}
+ public PTFInvocationSpec.OrderSpec processOrderSpec(ASTNode sortNode) {
+ PTFInvocationSpec.OrderSpec oSpec = new PTFInvocationSpec.OrderSpec();
+ int exprCnt = sortNode.getChildCount();
+ for(int i=0; i < exprCnt; i++) {
+ PTFInvocationSpec.OrderExpression exprSpec = new PTFInvocationSpec.OrderExpression();
+ ASTNode orderSpec = (ASTNode) sortNode.getChild(i);
+ ASTNode nullOrderSpec = (ASTNode) orderSpec.getChild(0);
+ exprSpec.setExpression((ASTNode) nullOrderSpec.getChild(0));
+ if ( orderSpec.getType() == HiveParser.TOK_TABSORTCOLNAMEASC ) {
+ exprSpec.setOrder(org.apache.hadoop.hive.ql.parse.PTFInvocationSpec.Order.ASC);
+ }
+ else {
+ exprSpec.setOrder(org.apache.hadoop.hive.ql.parse.PTFInvocationSpec.Order.DESC);
+ }
+ if ( nullOrderSpec.getType() == HiveParser.TOK_NULLS_FIRST ) {
Review Comment:
fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sonarcloud[bot] commented on pull request #3708: HIVE-26580: SHOW COMPACTIONS should support ordering and limiting fun…
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on PR #3708:
URL: https://github.com/apache/hive/pull/3708#issuecomment-1303052702
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=3708)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3708&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3708&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3708&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=CODE_SMELL) [1 Code Smell](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3708&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3708&metric=duplicated_lines_density&view=list) No Duplication information
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sonarcloud[bot] commented on pull request #3708: HIVE-26580: SHOW COMPACTIONS should support ordering and limiting fun…
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on PR #3708:
URL: https://github.com/apache/hive/pull/3708#issuecomment-1318074326
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=3708)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3708&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3708&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3708&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3708&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3708&metric=duplicated_lines_density&view=list) No Duplication information
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sonarcloud[bot] commented on pull request #3708: HIVE-26580: SHOW COMPACTIONS should support ordering and limiting fun…
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on PR #3708:
URL: https://github.com/apache/hive/pull/3708#issuecomment-1316459546
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=3708)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3708&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3708&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3708&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=CODE_SMELL) [2 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3708&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3708&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3708&metric=duplicated_lines_density&view=list) No Duplication information
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] rkirtir commented on a diff in pull request #3708: HIVE-26580: SHOW COMPACTIONS should support ordering and limiting fun…
Posted by GitBox <gi...@apache.org>.
rkirtir commented on code in PR #3708:
URL: https://github.com/apache/hive/pull/3708#discussion_r1023522209
##########
ql/src/java/org/apache/hadoop/hive/ql/ddl/process/show/compactions/ShowCompactionsAnalyzer.java:
##########
@@ -80,15 +83,64 @@ public void analyzeInternal(ASTNode root) throws SemanticException {
case HiveParser.TOK_COMPACT_ID:
compactionId = Long.parseLong(child.getChild(0).getText());
break;
+ case HiveParser.TOK_LIMIT:
+ limit = Short.valueOf((child.getChild(0)).getText());
+ break;
+ case HiveParser.TOK_ORDERBY:
+ orderBy = processSortOrderSpec(child);
+ break;
default:
dbName = stripQuotes(child.getText());
}
}
ShowCompactionsDesc desc = new ShowCompactionsDesc(ctx.getResFile(), compactionId, dbName, tblName, poolName, compactionType,
- compactionStatus, partitionSpec);
+ compactionStatus, partitionSpec, limit, orderBy);
Task<DDLWork> task = TaskFactory.get(new DDLWork(getInputs(), getOutputs(), desc));
rootTasks.add(task);
task.setFetchSource(true);
setFetchTask(createFetchTask(ShowCompactionsDesc.SCHEMA));
}
+
+
+ private String processSortOrderSpec(ASTNode sortNode) {
+ List<PTFInvocationSpec.OrderExpression> orderExp = processOrderSpec(sortNode).getExpressions();
+ Map<String, String> orderByAttributes = orderExp.stream().collect(
+ Collectors.toMap(x -> {
+ return x.getExpression().getChild(0)==null ?
+ CompactionColumn.valueOf(x.getExpression().getText().replaceAll("\'","").toUpperCase()).toString():
+ CompactionColumn.valueOf(x.getExpression().getChild(0).getText()).toString();
+ }, x -> x.getOrder().toString()));
+ return orderByAttributes.entrySet().stream().map(e -> e.getKey() + "\t" + e.getValue()).collect(Collectors.joining(","));
+ }
+ public enum CompactionColumn {
+ COMPACTIONID("CC_ID"),
+ PARTITION("CC_PARTITION"),
+ DATABASE("CC_DATABASE"),
+ TABLE("CC_TABLE"),
+ TYPE("CC_TYPE"),
+ STATE("CC_STATE"),
+ WORKER_HOST("CC_WORKER_ID"),
+ ENQUEUE_TIME("CC_ENQUEUE_TIME"),
+ POOLNAME("CC_POOL_NAME"),
+ WORKER("CC_WORKER_VERSION"),
+ START_TIME("CC_START"),
+ TXNID ("CC_TXN_ID"),
+ HADOOP_JOB_ID("CC_HADOOP_JOB_ID"),
+ NEXT_TXN_ID("CC_NEXT_TXN_ID"),
+ HIGHEST_WRITE_ID("CC_HIGHEST_WRITE_ID");
+
+ private final String colVal;
+ CompactionColumn(String colVal) {
+ this.colVal = colVal;
+ }
+ public String getColValue() {
Review Comment:
fixed
##########
ql/src/java/org/apache/hadoop/hive/ql/ddl/process/show/compactions/ShowCompactionsAnalyzer.java:
##########
@@ -80,15 +83,64 @@ public void analyzeInternal(ASTNode root) throws SemanticException {
case HiveParser.TOK_COMPACT_ID:
compactionId = Long.parseLong(child.getChild(0).getText());
break;
+ case HiveParser.TOK_LIMIT:
+ limit = Short.valueOf((child.getChild(0)).getText());
+ break;
+ case HiveParser.TOK_ORDERBY:
+ orderBy = processSortOrderSpec(child);
+ break;
default:
dbName = stripQuotes(child.getText());
}
}
ShowCompactionsDesc desc = new ShowCompactionsDesc(ctx.getResFile(), compactionId, dbName, tblName, poolName, compactionType,
- compactionStatus, partitionSpec);
+ compactionStatus, partitionSpec, limit, orderBy);
Task<DDLWork> task = TaskFactory.get(new DDLWork(getInputs(), getOutputs(), desc));
rootTasks.add(task);
task.setFetchSource(true);
setFetchTask(createFetchTask(ShowCompactionsDesc.SCHEMA));
}
+
+
+ private String processSortOrderSpec(ASTNode sortNode) {
+ List<PTFInvocationSpec.OrderExpression> orderExp = processOrderSpec(sortNode).getExpressions();
+ Map<String, String> orderByAttributes = orderExp.stream().collect(
+ Collectors.toMap(x -> {
+ return x.getExpression().getChild(0)==null ?
+ CompactionColumn.valueOf(x.getExpression().getText().replaceAll("\'","").toUpperCase()).toString():
+ CompactionColumn.valueOf(x.getExpression().getChild(0).getText()).toString();
+ }, x -> x.getOrder().toString()));
+ return orderByAttributes.entrySet().stream().map(e -> e.getKey() + "\t" + e.getValue()).collect(Collectors.joining(","));
+ }
+ public enum CompactionColumn {
Review Comment:
fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] rkirtir commented on a diff in pull request #3708: HIVE-26580: SHOW COMPACTIONS should support ordering and limiting fun…
Posted by GitBox <gi...@apache.org>.
rkirtir commented on code in PR #3708:
URL: https://github.com/apache/hive/pull/3708#discussion_r1023521855
##########
ql/src/java/org/apache/hadoop/hive/ql/ddl/process/show/compactions/ShowCompactionsAnalyzer.java:
##########
@@ -80,15 +83,64 @@ public void analyzeInternal(ASTNode root) throws SemanticException {
case HiveParser.TOK_COMPACT_ID:
compactionId = Long.parseLong(child.getChild(0).getText());
break;
+ case HiveParser.TOK_LIMIT:
+ limit = Short.valueOf((child.getChild(0)).getText());
+ break;
+ case HiveParser.TOK_ORDERBY:
+ orderBy = processSortOrderSpec(child);
+ break;
default:
dbName = stripQuotes(child.getText());
}
}
ShowCompactionsDesc desc = new ShowCompactionsDesc(ctx.getResFile(), compactionId, dbName, tblName, poolName, compactionType,
- compactionStatus, partitionSpec);
+ compactionStatus, partitionSpec, limit, orderBy);
Task<DDLWork> task = TaskFactory.get(new DDLWork(getInputs(), getOutputs(), desc));
rootTasks.add(task);
task.setFetchSource(true);
setFetchTask(createFetchTask(ShowCompactionsDesc.SCHEMA));
}
+
+
+ private String processSortOrderSpec(ASTNode sortNode) {
+ List<PTFInvocationSpec.OrderExpression> orderExp = processOrderSpec(sortNode).getExpressions();
+ Map<String, String> orderByAttributes = orderExp.stream().collect(
+ Collectors.toMap(x -> {
+ return x.getExpression().getChild(0)==null ?
+ CompactionColumn.valueOf(x.getExpression().getText().replaceAll("\'","").toUpperCase()).toString():
+ CompactionColumn.valueOf(x.getExpression().getChild(0).getText()).toString();
+ }, x -> x.getOrder().toString()));
+ return orderByAttributes.entrySet().stream().map(e -> e.getKey() + "\t" + e.getValue()).collect(Collectors.joining(","));
+ }
+ public enum CompactionColumn {
+ COMPACTIONID("CC_ID"),
Review Comment:
fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #3708: HIVE-26580: SHOW COMPACTIONS should support ordering and limiting fun…
Posted by GitBox <gi...@apache.org>.
deniskuzZ commented on code in PR #3708:
URL: https://github.com/apache/hive/pull/3708#discussion_r1022844493
##########
ql/src/java/org/apache/hadoop/hive/ql/ddl/process/show/compactions/ShowCompactionsAnalyzer.java:
##########
@@ -80,15 +83,64 @@ public void analyzeInternal(ASTNode root) throws SemanticException {
case HiveParser.TOK_COMPACT_ID:
compactionId = Long.parseLong(child.getChild(0).getText());
break;
+ case HiveParser.TOK_LIMIT:
+ limit = Short.valueOf((child.getChild(0)).getText());
+ break;
+ case HiveParser.TOK_ORDERBY:
+ orderBy = processSortOrderSpec(child);
+ break;
default:
dbName = stripQuotes(child.getText());
}
}
ShowCompactionsDesc desc = new ShowCompactionsDesc(ctx.getResFile(), compactionId, dbName, tblName, poolName, compactionType,
- compactionStatus, partitionSpec);
+ compactionStatus, partitionSpec, limit, orderBy);
Task<DDLWork> task = TaskFactory.get(new DDLWork(getInputs(), getOutputs(), desc));
rootTasks.add(task);
task.setFetchSource(true);
setFetchTask(createFetchTask(ShowCompactionsDesc.SCHEMA));
}
+
+
+ private String processSortOrderSpec(ASTNode sortNode) {
+ List<PTFInvocationSpec.OrderExpression> orderExp = processOrderSpec(sortNode).getExpressions();
+ Map<String, String> orderByAttributes = orderExp.stream().collect(
+ Collectors.toMap(x -> {
+ return x.getExpression().getChild(0)==null ?
+ CompactionColumn.valueOf(x.getExpression().getText().replaceAll("\'","").toUpperCase()).toString():
+ CompactionColumn.valueOf(x.getExpression().getChild(0).getText()).toString();
+ }, x -> x.getOrder().toString()));
+ return orderByAttributes.entrySet().stream().map(e -> e.getKey() + "\t" + e.getValue()).collect(Collectors.joining(","));
+ }
+ public enum CompactionColumn {
+ COMPACTIONID("CC_ID"),
+ PARTITION("CC_PARTITION"),
+ DATABASE("CC_DATABASE"),
Review Comment:
add SCHEMA("CC_DATABASE")
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #3708: HIVE-26580: SHOW COMPACTIONS should support ordering and limiting fun…
Posted by GitBox <gi...@apache.org>.
deniskuzZ commented on code in PR #3708:
URL: https://github.com/apache/hive/pull/3708#discussion_r1022843105
##########
ql/src/java/org/apache/hadoop/hive/ql/ddl/process/show/compactions/ShowCompactionsAnalyzer.java:
##########
@@ -80,15 +83,64 @@ public void analyzeInternal(ASTNode root) throws SemanticException {
case HiveParser.TOK_COMPACT_ID:
compactionId = Long.parseLong(child.getChild(0).getText());
break;
+ case HiveParser.TOK_LIMIT:
+ limit = Short.valueOf((child.getChild(0)).getText());
+ break;
+ case HiveParser.TOK_ORDERBY:
+ orderBy = processSortOrderSpec(child);
+ break;
default:
dbName = stripQuotes(child.getText());
}
}
ShowCompactionsDesc desc = new ShowCompactionsDesc(ctx.getResFile(), compactionId, dbName, tblName, poolName, compactionType,
- compactionStatus, partitionSpec);
+ compactionStatus, partitionSpec, limit, orderBy);
Task<DDLWork> task = TaskFactory.get(new DDLWork(getInputs(), getOutputs(), desc));
rootTasks.add(task);
task.setFetchSource(true);
setFetchTask(createFetchTask(ShowCompactionsDesc.SCHEMA));
}
+
+
+ private String processSortOrderSpec(ASTNode sortNode) {
+ List<PTFInvocationSpec.OrderExpression> orderExp = processOrderSpec(sortNode).getExpressions();
+ Map<String, String> orderByAttributes = orderExp.stream().collect(
+ Collectors.toMap(x -> {
+ return x.getExpression().getChild(0)==null ?
+ CompactionColumn.valueOf(x.getExpression().getText().replaceAll("\'","").toUpperCase()).toString():
+ CompactionColumn.valueOf(x.getExpression().getChild(0).getText()).toString();
+ }, x -> x.getOrder().toString()));
+ return orderByAttributes.entrySet().stream().map(e -> e.getKey() + "\t" + e.getValue()).collect(Collectors.joining(","));
+ }
+ public enum CompactionColumn {
Review Comment:
should it be public? new line is missing. Do we even need an enum?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org