You are viewing a plain text version of this content. The canonical link for it is here.
Posted to gitbox@hive.apache.org by GitBox <gi...@apache.org> on 2022/09/27 05:24:54 UTC
[GitHub] [hive] rkirtir opened a new pull request, #3625: Hive26563
rkirtir opened a new pull request, #3625:
URL: https://github.com/apache/hive/pull/3625
…output
<!--
Thanks for sending a pull request! Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://cwiki.apache.org/confluence/display/Hive/HowToContribute
2. Ensure that you have created an issue on the Hive project JIRA: https://issues.apache.org/jira/projects/HIVE/summary
3. Ensure you have added or run the appropriate tests for your PR:
4. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP]HIVE-XXXXX: Your PR title ...'.
5. Be sure to keep the PR description updated to reflect all changes.
6. Please write your PR title to summarize what this PR proposes.
7. If possible, provide a concise example to reproduce the issue for a faster review.
-->
### What changes were proposed in this pull request?
Show Compaction output is sorted so that most recent job will appear at top in the list and extra columns are added to the output
### Why are the changes needed?
Show Compaction output needed reformatting as it does not have consistency with
recent added columns to COMPACTION view
https://issues.apache.org/jira/browse/HIVE-26563
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Junit and Q tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sonarcloud[bot] commented on pull request #3625: Hive-26563: Add extra columns in Show Compactions output and sort the output
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on PR #3625:
URL: https://github.com/apache/hive/pull/3625#issuecomment-1278465503
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=3625)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3625&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3625&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3625&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=CODE_SMELL) [5 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3625&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3625&metric=duplicated_lines_density&view=list) No Duplication information
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #3625: Hive-26563: Add extra columns in Show Compactions output and sort the output
Posted by GitBox <gi...@apache.org>.
deniskuzZ commented on code in PR #3625:
URL: https://github.com/apache/hive/pull/3625#discussion_r992433770
##########
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java:
##########
@@ -3908,49 +3908,19 @@ public ShowCompactResponse showCompact(ShowCompactRequest rqst) throws MetaExcep
ShowCompactResponse response = new ShowCompactResponse(new ArrayList<>());
Connection dbConn = null;
PreparedStatement stmt = null;
+ StringBuilder query = new StringBuilder();
try {
try {
- StringBuilder sb =new StringBuilder(2048);
- sb.append(
- "SELECT " +
- " \"CQ_DATABASE\", \"CQ_TABLE\", \"CQ_PARTITION\", \"CQ_STATE\", \"CQ_TYPE\", \"CQ_WORKER_ID\", " +
- " \"CQ_START\", -1 \"CC_END\", \"CQ_RUN_AS\", \"CQ_HADOOP_JOB_ID\", \"CQ_ID\", \"CQ_ERROR_MESSAGE\", " +
- " \"CQ_ENQUEUE_TIME\", \"CQ_WORKER_VERSION\", \"CQ_INITIATOR_ID\", \"CQ_INITIATOR_VERSION\", " +
- " \"CQ_CLEANER_START\", \"CQ_POOL_NAME\"" +
- "FROM " +
- " \"COMPACTION_QUEUE\" "
- );
- if (org.apache.commons.lang3.StringUtils.isNotBlank(rqst.getPoolName())) {
- sb.append("WHERE \"CQ_POOL_NAME\" = ? ");
- }
- sb.append(
- "UNION ALL " +
- "SELECT " +
- " \"CC_DATABASE\", \"CC_TABLE\", \"CC_PARTITION\", \"CC_STATE\", \"CC_TYPE\", \"CC_WORKER_ID\", " +
- " \"CC_START\", \"CC_END\", \"CC_RUN_AS\", \"CC_HADOOP_JOB_ID\", \"CC_ID\", \"CC_ERROR_MESSAGE\", " +
- " \"CC_ENQUEUE_TIME\", \"CC_WORKER_VERSION\", \"CC_INITIATOR_ID\", \"CC_INITIATOR_VERSION\", " +
- " -1 , \"CC_POOL_NAME\"" +
- "FROM " +
- " \"COMPLETED_COMPACTIONS\" "
- );
- if (org.apache.commons.lang3.StringUtils.isNotBlank(rqst.getPoolName())) {
- sb.append("WHERE \"CC_POOL_NAME\" = ?");
- }
- //todo: sort by cq_id?
- //what I want is order by cc_end desc, cc_start asc (but derby has a bug https://issues.apache.org/jira/browse/DERBY-6013)
- //to sort so that currently running jobs are at the end of the list (bottom of screen)
- //and currently running ones are in sorted by start time
- //w/o order by likely currently running compactions will be first (LHS of Union)
-
- String query = sb.toString();
+ query.append(TxnQueries.SHOW_COMPACTION_QUERY);
+ String whereClause = getShowCompactionFilterClause(rqst);
+ if (!whereClause.isEmpty()) {
+ query.append(" where ");
Review Comment:
you are missing the `whereClause`
````
query
.append(getShowCompactionFilterClause(rqst))
.append(TxnQueries.SHOW_COMPACTION_ORDERBY_CLAUSE);
````
in getShowCompactionFilterClause
````
private String getShowCompactionFilterClause(ShowCompactRequest request) {
StringBuilder filter = new StringBuilder();
....
return filter.length() > 0 ? " where "+ filter.toString() : StringUtils.EMPTY
````
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sonarcloud[bot] commented on pull request #3625: Hive-26563: Add extra columns in Show Compactions output and sort the output
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on PR #3625:
URL: https://github.com/apache/hive/pull/3625#issuecomment-1277075634
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=3625)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3625&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3625&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3625&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=CODE_SMELL) [7 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3625&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3625&metric=duplicated_lines_density&view=list) No Duplication information
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sonarcloud[bot] commented on pull request #3625: Hive-26563: Add extra columns in Show Compactions output and sort the output
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on PR #3625:
URL: https://github.com/apache/hive/pull/3625#issuecomment-1274507532
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=3625)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3625&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3625&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3625&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=CODE_SMELL) [4 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3625&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3625&metric=duplicated_lines_density&view=list) No Duplication information
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sonarcloud[bot] commented on pull request #3625: Hive-26563: Add extra columns in Show Compactions output and sort the output
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on PR #3625:
URL: https://github.com/apache/hive/pull/3625#issuecomment-1281012885
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=3625)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3625&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3625&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3625&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=CODE_SMELL) [5 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3625&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3625&metric=duplicated_lines_density&view=list) No Duplication information
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #3625: Hive-26563: Add extra columns in Show Compactions output and sort the output
Posted by GitBox <gi...@apache.org>.
deniskuzZ commented on code in PR #3625:
URL: https://github.com/apache/hive/pull/3625#discussion_r992433770
##########
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java:
##########
@@ -3908,49 +3908,19 @@ public ShowCompactResponse showCompact(ShowCompactRequest rqst) throws MetaExcep
ShowCompactResponse response = new ShowCompactResponse(new ArrayList<>());
Connection dbConn = null;
PreparedStatement stmt = null;
+ StringBuilder query = new StringBuilder();
try {
try {
- StringBuilder sb =new StringBuilder(2048);
- sb.append(
- "SELECT " +
- " \"CQ_DATABASE\", \"CQ_TABLE\", \"CQ_PARTITION\", \"CQ_STATE\", \"CQ_TYPE\", \"CQ_WORKER_ID\", " +
- " \"CQ_START\", -1 \"CC_END\", \"CQ_RUN_AS\", \"CQ_HADOOP_JOB_ID\", \"CQ_ID\", \"CQ_ERROR_MESSAGE\", " +
- " \"CQ_ENQUEUE_TIME\", \"CQ_WORKER_VERSION\", \"CQ_INITIATOR_ID\", \"CQ_INITIATOR_VERSION\", " +
- " \"CQ_CLEANER_START\", \"CQ_POOL_NAME\"" +
- "FROM " +
- " \"COMPACTION_QUEUE\" "
- );
- if (org.apache.commons.lang3.StringUtils.isNotBlank(rqst.getPoolName())) {
- sb.append("WHERE \"CQ_POOL_NAME\" = ? ");
- }
- sb.append(
- "UNION ALL " +
- "SELECT " +
- " \"CC_DATABASE\", \"CC_TABLE\", \"CC_PARTITION\", \"CC_STATE\", \"CC_TYPE\", \"CC_WORKER_ID\", " +
- " \"CC_START\", \"CC_END\", \"CC_RUN_AS\", \"CC_HADOOP_JOB_ID\", \"CC_ID\", \"CC_ERROR_MESSAGE\", " +
- " \"CC_ENQUEUE_TIME\", \"CC_WORKER_VERSION\", \"CC_INITIATOR_ID\", \"CC_INITIATOR_VERSION\", " +
- " -1 , \"CC_POOL_NAME\"" +
- "FROM " +
- " \"COMPLETED_COMPACTIONS\" "
- );
- if (org.apache.commons.lang3.StringUtils.isNotBlank(rqst.getPoolName())) {
- sb.append("WHERE \"CC_POOL_NAME\" = ?");
- }
- //todo: sort by cq_id?
- //what I want is order by cc_end desc, cc_start asc (but derby has a bug https://issues.apache.org/jira/browse/DERBY-6013)
- //to sort so that currently running jobs are at the end of the list (bottom of screen)
- //and currently running ones are in sorted by start time
- //w/o order by likely currently running compactions will be first (LHS of Union)
-
- String query = sb.toString();
+ query.append(TxnQueries.SHOW_COMPACTION_QUERY);
+ String whereClause = getShowCompactionFilterClause(rqst);
+ if (!whereClause.isEmpty()) {
+ query.append(" where ");
Review Comment:
you are missing the `whereClause`
````
query
.append(getShowCompactionFilterClause(rqst))
.append(TxnQueries.SHOW_COMPACTION_ORDERBY_CLAUSE);
````
in getShowCompactionFilterClause
````
private String getShowCompactionFilterClause(ShowCompactRequest request) {
StringBuilder filter = new StringBuilder();
....
return filter.length()>0? " where "+ filter.toString() : StringUtils.EMPTY
````
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #3625: Hive-26563: Add extra columns in Show Compactions output and sort the output
Posted by GitBox <gi...@apache.org>.
deniskuzZ commented on code in PR #3625:
URL: https://github.com/apache/hive/pull/3625#discussion_r993410036
##########
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java:
##########
@@ -3908,49 +3908,19 @@ public ShowCompactResponse showCompact(ShowCompactRequest rqst) throws MetaExcep
ShowCompactResponse response = new ShowCompactResponse(new ArrayList<>());
Connection dbConn = null;
PreparedStatement stmt = null;
+ StringBuilder query = new StringBuilder();
try {
try {
- StringBuilder sb =new StringBuilder(2048);
- sb.append(
- "SELECT " +
- " \"CQ_DATABASE\", \"CQ_TABLE\", \"CQ_PARTITION\", \"CQ_STATE\", \"CQ_TYPE\", \"CQ_WORKER_ID\", " +
- " \"CQ_START\", -1 \"CC_END\", \"CQ_RUN_AS\", \"CQ_HADOOP_JOB_ID\", \"CQ_ID\", \"CQ_ERROR_MESSAGE\", " +
- " \"CQ_ENQUEUE_TIME\", \"CQ_WORKER_VERSION\", \"CQ_INITIATOR_ID\", \"CQ_INITIATOR_VERSION\", " +
- " \"CQ_CLEANER_START\", \"CQ_POOL_NAME\"" +
- "FROM " +
- " \"COMPACTION_QUEUE\" "
- );
- if (org.apache.commons.lang3.StringUtils.isNotBlank(rqst.getPoolName())) {
- sb.append("WHERE \"CQ_POOL_NAME\" = ? ");
- }
- sb.append(
- "UNION ALL " +
- "SELECT " +
- " \"CC_DATABASE\", \"CC_TABLE\", \"CC_PARTITION\", \"CC_STATE\", \"CC_TYPE\", \"CC_WORKER_ID\", " +
- " \"CC_START\", \"CC_END\", \"CC_RUN_AS\", \"CC_HADOOP_JOB_ID\", \"CC_ID\", \"CC_ERROR_MESSAGE\", " +
- " \"CC_ENQUEUE_TIME\", \"CC_WORKER_VERSION\", \"CC_INITIATOR_ID\", \"CC_INITIATOR_VERSION\", " +
- " -1 , \"CC_POOL_NAME\"" +
- "FROM " +
- " \"COMPLETED_COMPACTIONS\" "
- );
- if (org.apache.commons.lang3.StringUtils.isNotBlank(rqst.getPoolName())) {
- sb.append("WHERE \"CC_POOL_NAME\" = ?");
- }
- //todo: sort by cq_id?
- //what I want is order by cc_end desc, cc_start asc (but derby has a bug https://issues.apache.org/jira/browse/DERBY-6013)
- //to sort so that currently running jobs are at the end of the list (bottom of screen)
- //and currently running ones are in sorted by start time
- //w/o order by likely currently running compactions will be first (LHS of Union)
-
- String query = sb.toString();
+ query.append(TxnQueries.SHOW_COMPACTION_QUERY);
+ String whereClause = getShowCompactionFilterClause(rqst);
+ if (!whereClause.isEmpty()) {
+ query.append(whereClause);
+ query.append(TxnQueries.SHOW_COMPACTION_ORDERBY_CLAUSE);
+ } else
+ query.append(TxnQueries.SHOW_COMPACTION_ORDERBY_CLAUSE);
dbConn = getDbConn(Connection.TRANSACTION_READ_COMMITTED);
- stmt = dbConn.prepareStatement(query);
- if (org.apache.commons.lang3.StringUtils.isNotBlank(rqst.getPoolName())) {
- stmt.setString(1, rqst.getPoolName());
- stmt.setString(2, rqst.getPoolName());
- }
-
- LOG.debug("Going to execute query <{}>", query);
+ stmt = sqlGenerator.prepareStmtWithParameters(dbConn, query.toString(), getShowCompactionQueryParamList(rqst));
Review Comment:
please use try-with-resources, so we won't need to close them manually:
````
try (Connection dbConn = getDbConn(Connection.TRANSACTION_READ_COMMITTED);
PreparedStatement stmt = dbConn.prepareStatement(query)) {
...
}
````
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #3625: Hive-26563: Add extra columns in Show Compactions output and sort the output
Posted by GitBox <gi...@apache.org>.
deniskuzZ commented on code in PR #3625:
URL: https://github.com/apache/hive/pull/3625#discussion_r992431232
##########
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java:
##########
@@ -4009,6 +3983,25 @@ public ShowCompactResponse showCompact(ShowCompactRequest rqst) throws MetaExcep
}
}
+ private List<String> getShowCompactionQueryParamList(ShowCompactRequest request) throws MetaException {
+
+ String poolName = request.getPoolName();
+ List<String> params = new ArrayList<>();
+ if (org.apache.commons.lang3.StringUtils.isNotBlank(poolName)) {
+ params.add(poolName);
+ }
+ return params;
+ }
+
+ private String getShowCompactionFilterClause(ShowCompactRequest request) {
+ StringBuilder filter = new StringBuilder();
+ String poolName = request.getPoolName();
+ if(poolName != null && !poolName.isEmpty()){
Review Comment:
nit: space,
also you could use StringUtils.isEmpty instead of those 2 cheks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ merged pull request #3625: Hive-26563: Add extra columns in Show Compactions output and sort the output
Posted by GitBox <gi...@apache.org>.
deniskuzZ merged PR #3625:
URL: https://github.com/apache/hive/pull/3625
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sonarcloud[bot] commented on pull request #3625: Hive-26563: Add extra columns in Show Compactions output and sort the output
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on PR #3625:
URL: https://github.com/apache/hive/pull/3625#issuecomment-1259024867
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=3625)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3625&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3625&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3625&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=CODE_SMELL) [9 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3625&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3625&metric=duplicated_lines_density&view=list) No Duplication information
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sonarcloud[bot] commented on pull request #3625: Hive-26563: Add extra columns in Show Compactions output and sort the output
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on PR #3625:
URL: https://github.com/apache/hive/pull/3625#issuecomment-1265269462
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=3625)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=BUG) [5 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3625&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3625&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3625&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=CODE_SMELL) [13 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3625&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3625&metric=duplicated_lines_density&view=list) No Duplication information
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sonarcloud[bot] commented on pull request #3625: Hive-26563: Add extra columns in Show Compactions output and sort the output
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on PR #3625:
URL: https://github.com/apache/hive/pull/3625#issuecomment-1259185305
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=3625)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3625&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3625&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3625&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=CODE_SMELL) [9 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3625&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3625&metric=duplicated_lines_density&view=list) No Duplication information
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #3625: Hive-26563: Add extra columns in Show Compactions output and sort the output
Posted by GitBox <gi...@apache.org>.
deniskuzZ commented on code in PR #3625:
URL: https://github.com/apache/hive/pull/3625#discussion_r992433770
##########
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java:
##########
@@ -3908,49 +3908,19 @@ public ShowCompactResponse showCompact(ShowCompactRequest rqst) throws MetaExcep
ShowCompactResponse response = new ShowCompactResponse(new ArrayList<>());
Connection dbConn = null;
PreparedStatement stmt = null;
+ StringBuilder query = new StringBuilder();
try {
try {
- StringBuilder sb =new StringBuilder(2048);
- sb.append(
- "SELECT " +
- " \"CQ_DATABASE\", \"CQ_TABLE\", \"CQ_PARTITION\", \"CQ_STATE\", \"CQ_TYPE\", \"CQ_WORKER_ID\", " +
- " \"CQ_START\", -1 \"CC_END\", \"CQ_RUN_AS\", \"CQ_HADOOP_JOB_ID\", \"CQ_ID\", \"CQ_ERROR_MESSAGE\", " +
- " \"CQ_ENQUEUE_TIME\", \"CQ_WORKER_VERSION\", \"CQ_INITIATOR_ID\", \"CQ_INITIATOR_VERSION\", " +
- " \"CQ_CLEANER_START\", \"CQ_POOL_NAME\"" +
- "FROM " +
- " \"COMPACTION_QUEUE\" "
- );
- if (org.apache.commons.lang3.StringUtils.isNotBlank(rqst.getPoolName())) {
- sb.append("WHERE \"CQ_POOL_NAME\" = ? ");
- }
- sb.append(
- "UNION ALL " +
- "SELECT " +
- " \"CC_DATABASE\", \"CC_TABLE\", \"CC_PARTITION\", \"CC_STATE\", \"CC_TYPE\", \"CC_WORKER_ID\", " +
- " \"CC_START\", \"CC_END\", \"CC_RUN_AS\", \"CC_HADOOP_JOB_ID\", \"CC_ID\", \"CC_ERROR_MESSAGE\", " +
- " \"CC_ENQUEUE_TIME\", \"CC_WORKER_VERSION\", \"CC_INITIATOR_ID\", \"CC_INITIATOR_VERSION\", " +
- " -1 , \"CC_POOL_NAME\"" +
- "FROM " +
- " \"COMPLETED_COMPACTIONS\" "
- );
- if (org.apache.commons.lang3.StringUtils.isNotBlank(rqst.getPoolName())) {
- sb.append("WHERE \"CC_POOL_NAME\" = ?");
- }
- //todo: sort by cq_id?
- //what I want is order by cc_end desc, cc_start asc (but derby has a bug https://issues.apache.org/jira/browse/DERBY-6013)
- //to sort so that currently running jobs are at the end of the list (bottom of screen)
- //and currently running ones are in sorted by start time
- //w/o order by likely currently running compactions will be first (LHS of Union)
-
- String query = sb.toString();
+ query.append(TxnQueries.SHOW_COMPACTION_QUERY);
+ String whereClause = getShowCompactionFilterClause(rqst);
+ if (!whereClause.isEmpty()) {
+ query.append(" where ");
Review Comment:
you are missing the `whereClause`
````
if (!whereClause.isEmpty()) {
query.append(" where ").append(whereClause);
}
query.append(TxnQueries.SHOW_COMPACTION_ORDERBY_CLAUSE);
````
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sonarcloud[bot] commented on pull request #3625: Hive-26563: Add extra columns in Show Compactions output and sort the output
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on PR #3625:
URL: https://github.com/apache/hive/pull/3625#issuecomment-1264972036
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=3625)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=BUG) [5 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3625&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3625&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3625&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=CODE_SMELL) [13 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3625&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3625&metric=duplicated_lines_density&view=list) No Duplication information
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] rkirtir commented on a diff in pull request #3625: Hive-26563: Add extra columns in Show Compactions output and sort the output
Posted by GitBox <gi...@apache.org>.
rkirtir commented on code in PR #3625:
URL: https://github.com/apache/hive/pull/3625#discussion_r985625967
##########
ql/src/java/org/apache/hadoop/hive/ql/ddl/process/show/compactions/ShowCompactionsDesc.java:
##########
@@ -33,8 +33,8 @@ public class ShowCompactionsDesc implements DDLDesc, Serializable {
// @formatter:off
public static final String SCHEMA =
- "compactionid,dbname,tabname,partname,type,state,workerhost,workerid,enqueuetime,starttime,duration,hadoopjobid,errormessage,initiatorhost,initiatorid,poolname#" +
- "string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string";
+ "compactionid,dbname,tabname,partname,type,state,workerhost,workerid,enqueuetime,starttime,duration,hadoopjobid,errormessage,initiatorhost,initiatorid,poolname,txnid,nexttxnid,committime,hightestWriteID#" +
Review Comment:
resolved
##########
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java:
##########
@@ -314,6 +314,8 @@ abstract class TxnHandler implements TxnStore, TxnStore.MutexAPI {
private static final String EXCL_CTAS_ERR_MSG =
"Failed to initiate a concurrent CTAS operation with the same table name, lockInfo : %s";
private static final String ZERO_WAIT_READ_ERR_MSG = "Unable to acquire read lock due to an existing exclusive lock {%s}";
+ private static String SHOW_COMPACTION_ORDERBY_CLAUSE = " order by case when CC_END > CC_START and CC_END > CC_ENQUEUE_TIME then CC_END When CC_START > CC_ENQUEUE_TIME then CC_START Else CC_ENQUEUE_TIME end desc";
Review Comment:
resolved
##########
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java:
##########
@@ -3871,46 +3873,35 @@ public ShowCompactResponse showCompact(ShowCompactRequest rqst) throws MetaExcep
PreparedStatement stmt = null;
try {
try {
- StringBuilder sb =new StringBuilder(2048);
- sb.append(
- "SELECT " +
- " \"CQ_DATABASE\", \"CQ_TABLE\", \"CQ_PARTITION\", \"CQ_STATE\", \"CQ_TYPE\", \"CQ_WORKER_ID\", " +
- " \"CQ_START\", -1 \"CC_END\", \"CQ_RUN_AS\", \"CQ_HADOOP_JOB_ID\", \"CQ_ID\", \"CQ_ERROR_MESSAGE\", " +
- " \"CQ_ENQUEUE_TIME\", \"CQ_WORKER_VERSION\", \"CQ_INITIATOR_ID\", \"CQ_INITIATOR_VERSION\", " +
- " \"CQ_CLEANER_START\", \"CQ_POOL_NAME\"" +
- "FROM " +
- " \"COMPACTION_QUEUE\" "
- );
- if (org.apache.commons.lang3.StringUtils.isNotBlank(rqst.getPoolName())) {
- sb.append("WHERE \"CQ_POOL_NAME\" = ? ");
- }
- sb.append(
- "UNION ALL " +
- "SELECT " +
- " \"CC_DATABASE\", \"CC_TABLE\", \"CC_PARTITION\", \"CC_STATE\", \"CC_TYPE\", \"CC_WORKER_ID\", " +
- " \"CC_START\", \"CC_END\", \"CC_RUN_AS\", \"CC_HADOOP_JOB_ID\", \"CC_ID\", \"CC_ERROR_MESSAGE\", " +
- " \"CC_ENQUEUE_TIME\", \"CC_WORKER_VERSION\", \"CC_INITIATOR_ID\", \"CC_INITIATOR_VERSION\", " +
- " -1 , \"CC_POOL_NAME\"" +
- "FROM " +
- " \"COMPLETED_COMPACTIONS\" "
- );
- if (org.apache.commons.lang3.StringUtils.isNotBlank(rqst.getPoolName())) {
- sb.append("WHERE \"CC_POOL_NAME\" = ?");
- }
- //todo: sort by cq_id?
- //what I want is order by cc_end desc, cc_start asc (but derby has a bug https://issues.apache.org/jira/browse/DERBY-6013)
- //to sort so that currently running jobs are at the end of the list (bottom of screen)
- //and currently running ones are in sorted by start time
- //w/o order by likely currently running compactions will be first (LHS of Union)
-
- String query = sb.toString();
+ String query = "SELECT XX.* FROM ( SELECT " +
Review Comment:
Resolved. Added class TxnQueries for related queries
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #3625: Hive-26563: Add extra columns in Show Compactions output and sort the output
Posted by GitBox <gi...@apache.org>.
deniskuzZ commented on code in PR #3625:
URL: https://github.com/apache/hive/pull/3625#discussion_r993398046
##########
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java:
##########
@@ -4009,6 +3983,25 @@ public ShowCompactResponse showCompact(ShowCompactRequest rqst) throws MetaExcep
}
}
+ private List<String> getShowCompactionQueryParamList(ShowCompactRequest request) throws MetaException {
+
+ String poolName = request.getPoolName();
+ List<String> params = new ArrayList<>();
+ if (org.apache.commons.lang3.StringUtils.isNotBlank(poolName)) {
+ params.add(poolName);
+ }
+ return params;
+ }
+
+ private String getShowCompactionFilterClause(ShowCompactRequest request) {
+ StringBuilder filter = new StringBuilder();
+ String poolName = request.getPoolName();
+ if (org.apache.commons.lang3.StringUtils.isNotBlank(poolName)) {
+ filter.append("\"CC_POOL_NAME\"=?");
+ }
+ return filter.length() > 0 ? " where " + filter.toString() : org.apache.commons.lang3.StringUtils.EMPTY;
Review Comment:
can we do a static import for StringUtils to get rid of a package name?
##########
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java:
##########
@@ -3908,49 +3908,19 @@ public ShowCompactResponse showCompact(ShowCompactRequest rqst) throws MetaExcep
ShowCompactResponse response = new ShowCompactResponse(new ArrayList<>());
Connection dbConn = null;
PreparedStatement stmt = null;
+ StringBuilder query = new StringBuilder();
try {
try {
- StringBuilder sb =new StringBuilder(2048);
- sb.append(
- "SELECT " +
- " \"CQ_DATABASE\", \"CQ_TABLE\", \"CQ_PARTITION\", \"CQ_STATE\", \"CQ_TYPE\", \"CQ_WORKER_ID\", " +
- " \"CQ_START\", -1 \"CC_END\", \"CQ_RUN_AS\", \"CQ_HADOOP_JOB_ID\", \"CQ_ID\", \"CQ_ERROR_MESSAGE\", " +
- " \"CQ_ENQUEUE_TIME\", \"CQ_WORKER_VERSION\", \"CQ_INITIATOR_ID\", \"CQ_INITIATOR_VERSION\", " +
- " \"CQ_CLEANER_START\", \"CQ_POOL_NAME\"" +
- "FROM " +
- " \"COMPACTION_QUEUE\" "
- );
- if (org.apache.commons.lang3.StringUtils.isNotBlank(rqst.getPoolName())) {
- sb.append("WHERE \"CQ_POOL_NAME\" = ? ");
- }
- sb.append(
- "UNION ALL " +
- "SELECT " +
- " \"CC_DATABASE\", \"CC_TABLE\", \"CC_PARTITION\", \"CC_STATE\", \"CC_TYPE\", \"CC_WORKER_ID\", " +
- " \"CC_START\", \"CC_END\", \"CC_RUN_AS\", \"CC_HADOOP_JOB_ID\", \"CC_ID\", \"CC_ERROR_MESSAGE\", " +
- " \"CC_ENQUEUE_TIME\", \"CC_WORKER_VERSION\", \"CC_INITIATOR_ID\", \"CC_INITIATOR_VERSION\", " +
- " -1 , \"CC_POOL_NAME\"" +
- "FROM " +
- " \"COMPLETED_COMPACTIONS\" "
- );
- if (org.apache.commons.lang3.StringUtils.isNotBlank(rqst.getPoolName())) {
- sb.append("WHERE \"CC_POOL_NAME\" = ?");
- }
- //todo: sort by cq_id?
- //what I want is order by cc_end desc, cc_start asc (but derby has a bug https://issues.apache.org/jira/browse/DERBY-6013)
- //to sort so that currently running jobs are at the end of the list (bottom of screen)
- //and currently running ones are in sorted by start time
- //w/o order by likely currently running compactions will be first (LHS of Union)
-
- String query = sb.toString();
+ query.append(TxnQueries.SHOW_COMPACTION_QUERY);
+ String whereClause = getShowCompactionFilterClause(rqst);
+ if (!whereClause.isEmpty()) {
+ query.append(whereClause);
+ query.append(TxnQueries.SHOW_COMPACTION_ORDERBY_CLAUSE);
+ } else
+ query.append(TxnQueries.SHOW_COMPACTION_ORDERBY_CLAUSE);
dbConn = getDbConn(Connection.TRANSACTION_READ_COMMITTED);
- stmt = dbConn.prepareStatement(query);
- if (org.apache.commons.lang3.StringUtils.isNotBlank(rqst.getPoolName())) {
- stmt.setString(1, rqst.getPoolName());
- stmt.setString(2, rqst.getPoolName());
- }
-
- LOG.debug("Going to execute query <{}>", query);
+ stmt = sqlGenerator.prepareStmtWithParameters(dbConn, query.toString(), getShowCompactionQueryParamList(rqst));
Review Comment:
please use try-with-resources, so we won't need to close them manually:
````
try (Connection dbConn = getDbConn(Connection.TRANSACTION_READ_COMMITTED);
PreparedStatement stmt = dbConn.prepareStatement(query)) {
....
...
}
````
##########
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java:
##########
@@ -3908,49 +3908,19 @@ public ShowCompactResponse showCompact(ShowCompactRequest rqst) throws MetaExcep
ShowCompactResponse response = new ShowCompactResponse(new ArrayList<>());
Connection dbConn = null;
PreparedStatement stmt = null;
+ StringBuilder query = new StringBuilder();
try {
try {
- StringBuilder sb =new StringBuilder(2048);
- sb.append(
- "SELECT " +
- " \"CQ_DATABASE\", \"CQ_TABLE\", \"CQ_PARTITION\", \"CQ_STATE\", \"CQ_TYPE\", \"CQ_WORKER_ID\", " +
- " \"CQ_START\", -1 \"CC_END\", \"CQ_RUN_AS\", \"CQ_HADOOP_JOB_ID\", \"CQ_ID\", \"CQ_ERROR_MESSAGE\", " +
- " \"CQ_ENQUEUE_TIME\", \"CQ_WORKER_VERSION\", \"CQ_INITIATOR_ID\", \"CQ_INITIATOR_VERSION\", " +
- " \"CQ_CLEANER_START\", \"CQ_POOL_NAME\"" +
- "FROM " +
- " \"COMPACTION_QUEUE\" "
- );
- if (org.apache.commons.lang3.StringUtils.isNotBlank(rqst.getPoolName())) {
- sb.append("WHERE \"CQ_POOL_NAME\" = ? ");
- }
- sb.append(
- "UNION ALL " +
- "SELECT " +
- " \"CC_DATABASE\", \"CC_TABLE\", \"CC_PARTITION\", \"CC_STATE\", \"CC_TYPE\", \"CC_WORKER_ID\", " +
- " \"CC_START\", \"CC_END\", \"CC_RUN_AS\", \"CC_HADOOP_JOB_ID\", \"CC_ID\", \"CC_ERROR_MESSAGE\", " +
- " \"CC_ENQUEUE_TIME\", \"CC_WORKER_VERSION\", \"CC_INITIATOR_ID\", \"CC_INITIATOR_VERSION\", " +
- " -1 , \"CC_POOL_NAME\"" +
- "FROM " +
- " \"COMPLETED_COMPACTIONS\" "
- );
- if (org.apache.commons.lang3.StringUtils.isNotBlank(rqst.getPoolName())) {
- sb.append("WHERE \"CC_POOL_NAME\" = ?");
- }
- //todo: sort by cq_id?
- //what I want is order by cc_end desc, cc_start asc (but derby has a bug https://issues.apache.org/jira/browse/DERBY-6013)
- //to sort so that currently running jobs are at the end of the list (bottom of screen)
- //and currently running ones are in sorted by start time
- //w/o order by likely currently running compactions will be first (LHS of Union)
-
- String query = sb.toString();
+ query.append(TxnQueries.SHOW_COMPACTION_QUERY);
+ String whereClause = getShowCompactionFilterClause(rqst);
+ if (!whereClause.isEmpty()) {
Review Comment:
no need for if-else checks, you could simply do the following:
````
StringBuilder query = new StringBuilder(TxnQueries.SHOW_COMPACTION_QUERY)
.append(getShowCompactionFilterClause(rqst));
.append(TxnQueries.SHOW_COMPACTION_ORDERBY_CLAUSE);
````
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #3625: Hive-26563: Add extra columns in Show Compactions output and sort the output
Posted by GitBox <gi...@apache.org>.
deniskuzZ commented on code in PR #3625:
URL: https://github.com/apache/hive/pull/3625#discussion_r984573898
##########
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java:
##########
@@ -314,6 +314,8 @@ abstract class TxnHandler implements TxnStore, TxnStore.MutexAPI {
private static final String EXCL_CTAS_ERR_MSG =
"Failed to initiate a concurrent CTAS operation with the same table name, lockInfo : %s";
private static final String ZERO_WAIT_READ_ERR_MSG = "Unable to acquire read lock due to an existing exclusive lock {%s}";
+ private static String SHOW_COMPACTION_ORDERBY_CLAUSE = " order by case when CC_END > CC_START and CC_END > CC_ENQUEUE_TIME then CC_END When CC_START > CC_ENQUEUE_TIME then CC_START Else CC_ENQUEUE_TIME end desc";
Review Comment:
CC_ENQUEUE_TIME - inserted in a queue;
CC_START - worker started;
CC_COMMIT_TIME - end of worker/compaction
CQ_CLEANER_START - cleaner started
CC_END - end of cleaner
shouldn't it be something like:
````
order by
case
when CC_END > CC_START and CC_END > CQ_CLEANER_START
then CC_END
when CC_START > CQ_CLEANER_START
then CC_START
else CQ_CLEANER_START
end desc,
CC_ENQUEUE_TIME asc;
````
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sonarcloud[bot] commented on pull request #3625: Hive-26563: Add extra columns in Show Compactions output and sort the output
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on PR #3625:
URL: https://github.com/apache/hive/pull/3625#issuecomment-1272012868
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=3625)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=BUG) [6 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3625&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3625&resolved=false&types=SECURITY_HOTSPOT) [1 Security Hotspot](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3625&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=CODE_SMELL) [75 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3625&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3625&metric=duplicated_lines_density&view=list) No Duplication information
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sonarcloud[bot] commented on pull request #3625: Hive-26563: Add extra columns in Show Compactions output and sort the output
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on PR #3625:
URL: https://github.com/apache/hive/pull/3625#issuecomment-1277385826
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=3625)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3625&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3625&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3625&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=CODE_SMELL) [7 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3625&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3625&metric=duplicated_lines_density&view=list) No Duplication information
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sonarcloud[bot] commented on pull request #3625: Hive-26563: Add extra columns in Show Compactions output and sort the output
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on PR #3625:
URL: https://github.com/apache/hive/pull/3625#issuecomment-1280300329
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=3625)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3625&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3625&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3625&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=CODE_SMELL) [5 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3625&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3625&metric=duplicated_lines_density&view=list) No Duplication information
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sonarcloud[bot] commented on pull request #3625: Hive-26563: Add extra columns in Show Compactions output and sort the output
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on PR #3625:
URL: https://github.com/apache/hive/pull/3625#issuecomment-1281843738
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=3625)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3625&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3625&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3625&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=CODE_SMELL) [5 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3625&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3625&metric=duplicated_lines_density&view=list) No Duplication information
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #3625: Hive-26563: Add extra columns in Show Compactions output and sort the output
Posted by GitBox <gi...@apache.org>.
deniskuzZ commented on code in PR #3625:
URL: https://github.com/apache/hive/pull/3625#discussion_r994539733
##########
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java:
##########
@@ -3905,110 +3908,95 @@ protected static String compactorStateToResponse(char s) {
@RetrySemantics.ReadOnly
@SuppressWarnings("squid:S2095")
public ShowCompactResponse showCompact(ShowCompactRequest rqst) throws MetaException {
- ShowCompactResponse response = new ShowCompactResponse(new ArrayList<>());
- Connection dbConn = null;
- PreparedStatement stmt = null;
try {
- try {
- StringBuilder sb =new StringBuilder(2048);
- sb.append(
- "SELECT " +
- " \"CQ_DATABASE\", \"CQ_TABLE\", \"CQ_PARTITION\", \"CQ_STATE\", \"CQ_TYPE\", \"CQ_WORKER_ID\", " +
- " \"CQ_START\", -1 \"CC_END\", \"CQ_RUN_AS\", \"CQ_HADOOP_JOB_ID\", \"CQ_ID\", \"CQ_ERROR_MESSAGE\", " +
- " \"CQ_ENQUEUE_TIME\", \"CQ_WORKER_VERSION\", \"CQ_INITIATOR_ID\", \"CQ_INITIATOR_VERSION\", " +
- " \"CQ_CLEANER_START\", \"CQ_POOL_NAME\"" +
- "FROM " +
- " \"COMPACTION_QUEUE\" "
- );
- if (org.apache.commons.lang3.StringUtils.isNotBlank(rqst.getPoolName())) {
- sb.append("WHERE \"CQ_POOL_NAME\" = ? ");
- }
- sb.append(
- "UNION ALL " +
- "SELECT " +
- " \"CC_DATABASE\", \"CC_TABLE\", \"CC_PARTITION\", \"CC_STATE\", \"CC_TYPE\", \"CC_WORKER_ID\", " +
- " \"CC_START\", \"CC_END\", \"CC_RUN_AS\", \"CC_HADOOP_JOB_ID\", \"CC_ID\", \"CC_ERROR_MESSAGE\", " +
- " \"CC_ENQUEUE_TIME\", \"CC_WORKER_VERSION\", \"CC_INITIATOR_ID\", \"CC_INITIATOR_VERSION\", " +
- " -1 , \"CC_POOL_NAME\"" +
- "FROM " +
- " \"COMPLETED_COMPACTIONS\" "
- );
- if (org.apache.commons.lang3.StringUtils.isNotBlank(rqst.getPoolName())) {
- sb.append("WHERE \"CC_POOL_NAME\" = ?");
- }
- //todo: sort by cq_id?
- //what I want is order by cc_end desc, cc_start asc (but derby has a bug https://issues.apache.org/jira/browse/DERBY-6013)
- //to sort so that currently running jobs are at the end of the list (bottom of screen)
- //and currently running ones are in sorted by start time
- //w/o order by likely currently running compactions will be first (LHS of Union)
-
- String query = sb.toString();
- dbConn = getDbConn(Connection.TRANSACTION_READ_COMMITTED);
- stmt = dbConn.prepareStatement(query);
- if (org.apache.commons.lang3.StringUtils.isNotBlank(rqst.getPoolName())) {
- stmt.setString(1, rqst.getPoolName());
- stmt.setString(2, rqst.getPoolName());
- }
-
- LOG.debug("Going to execute query <{}>", query);
- ResultSet rs = stmt.executeQuery();
- while (rs.next()) {
- ShowCompactResponseElement e = new ShowCompactResponseElement();
- e.setDbname(rs.getString(1));
- e.setTablename(rs.getString(2));
- e.setPartitionname(rs.getString(3));
- e.setState(compactorStateToResponse(rs.getString(4).charAt(0)));
- try {
- e.setType(dbCompactionType2ThriftType(rs.getString(5).charAt(0)));
- } catch (MetaException ex) {
- //do nothing to handle RU/D if we add another status
- }
- e.setWorkerid(rs.getString(6));
- long start = rs.getLong(7);
- if (!rs.wasNull()) {
- e.setStart(start);
- }
- long endTime = rs.getLong(8);
- if (endTime != -1) {
- e.setEndTime(endTime);
- }
- e.setRunAs(rs.getString(9));
- e.setHadoopJobId(rs.getString(10));
- e.setId(rs.getLong(11));
- e.setErrorMessage(rs.getString(12));
- long enqueueTime = rs.getLong(13);
- if (!rs.wasNull()) {
- e.setEnqueueTime(enqueueTime);
- }
- e.setWorkerVersion(rs.getString(14));
- e.setInitiatorId(rs.getString(15));
- e.setInitiatorVersion(rs.getString(16));
- long cleanerStart = rs.getLong(17);
- if (!rs.wasNull() && (cleanerStart != -1)) {
- e.setCleanerStart(cleanerStart);
- }
- String poolName = rs.getString(18);
- if (org.apache.commons.lang3.StringUtils.isBlank(poolName)) {
- e.setPoolName(DEFAULT_POOL_NAME);
- } else {
- e.setPoolName(poolName);
+ ShowCompactResponse response = new ShowCompactResponse(new ArrayList<>());
+ try (Connection dbConn = getDbConn(Connection.TRANSACTION_READ_COMMITTED)) {
+ StringBuilder query = new StringBuilder(TxnQueries.SHOW_COMPACTION_QUERY).
+ append(getShowCompactionFilterClause(rqst)).append(TxnQueries.SHOW_COMPACTION_ORDERBY_CLAUSE);
+ try (PreparedStatement stmt = sqlGenerator.prepareStmtWithParameters(dbConn, query.toString(),
Review Comment:
as mentioned before, please move dbConn in the same try-with resources block, ie
````
try (Connection dbConn = getDbConn(Connection.TRANSACTION_READ_COMMITTED);
PreparedStatement stmt = sqlGenerator.prepareStmtWithParameters(dbConn, query.toString(),...) {
...
}
````
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sonarcloud[bot] commented on pull request #3625: Hive-26563: Add extra columns in Show Compactions output and sort the output
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on PR #3625:
URL: https://github.com/apache/hive/pull/3625#issuecomment-1277821386
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=3625)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3625&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3625&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3625&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=CODE_SMELL) [5 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3625&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3625&metric=duplicated_lines_density&view=list) No Duplication information
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sonarcloud[bot] commented on pull request #3625: Hive-26563: Add extra columns in Show Compactions output and sort the output
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on PR #3625:
URL: https://github.com/apache/hive/pull/3625#issuecomment-1271696062
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=3625)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=BUG) [6 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3625&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3625&resolved=false&types=SECURITY_HOTSPOT) [1 Security Hotspot](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3625&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=CODE_SMELL) [75 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3625&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3625&metric=duplicated_lines_density&view=list) No Duplication information
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sonarcloud[bot] commented on pull request #3625: Hive-26563: Add extra columns in Show Compactions output and sort the output
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on PR #3625:
URL: https://github.com/apache/hive/pull/3625#issuecomment-1272494275
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=3625)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=BUG) [6 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3625&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3625&resolved=false&types=SECURITY_HOTSPOT) [1 Security Hotspot](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3625&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=CODE_SMELL) [75 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3625&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3625&metric=duplicated_lines_density&view=list) No Duplication information
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sonarcloud[bot] commented on pull request #3625: Hive-26563: Add extra columns in Show Compactions output and sort the output
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on PR #3625:
URL: https://github.com/apache/hive/pull/3625#issuecomment-1276809021
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=3625)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3625&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3625&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3625&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=CODE_SMELL) [4 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3625&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3625&metric=duplicated_lines_density&view=list) No Duplication information
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #3625: Hive-26563: Add extra columns in Show Compactions output and sort the output
Posted by GitBox <gi...@apache.org>.
deniskuzZ commented on code in PR #3625:
URL: https://github.com/apache/hive/pull/3625#discussion_r984573898
##########
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java:
##########
@@ -314,6 +314,8 @@ abstract class TxnHandler implements TxnStore, TxnStore.MutexAPI {
private static final String EXCL_CTAS_ERR_MSG =
"Failed to initiate a concurrent CTAS operation with the same table name, lockInfo : %s";
private static final String ZERO_WAIT_READ_ERR_MSG = "Unable to acquire read lock due to an existing exclusive lock {%s}";
+ private static String SHOW_COMPACTION_ORDERBY_CLAUSE = " order by case when CC_END > CC_START and CC_END > CC_ENQUEUE_TIME then CC_END When CC_START > CC_ENQUEUE_TIME then CC_START Else CC_ENQUEUE_TIME end desc";
Review Comment:
CC_ENQUEUE_TIME - inserted in a queue;
CC_START - worker started;
CC_COMMIT_TIME - end of worker/compaction
CQ_CLEANER_START - cleaner started
CC_END - end of cleaner
shouldn't it be something like:
````
order by
case
when CC_END > CC_START and CC_END > CC_COMMIT_TIME
then CC_END
when CC_START > CC_COMMIT_TIME
then CC_START
else CC_COMMIT_TIME
end desc,
CC_ENQUEUE_TIME asc;
````
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #3625: Hive-26563: Add extra columns in Show Compactions output and sort the output
Posted by GitBox <gi...@apache.org>.
deniskuzZ commented on code in PR #3625:
URL: https://github.com/apache/hive/pull/3625#discussion_r984504870
##########
ql/src/java/org/apache/hadoop/hive/ql/ddl/process/show/compactions/ShowCompactionsDesc.java:
##########
@@ -33,8 +33,8 @@ public class ShowCompactionsDesc implements DDLDesc, Serializable {
// @formatter:off
public static final String SCHEMA =
- "compactionid,dbname,tabname,partname,type,state,workerhost,workerid,enqueuetime,starttime,duration,hadoopjobid,errormessage,initiatorhost,initiatorid,poolname#" +
- "string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string";
+ "compactionid,dbname,tabname,partname,type,state,workerhost,workerid,enqueuetime,starttime,duration,hadoopjobid,errormessage,initiatorhost,initiatorid,poolname,txnid,nexttxnid,committime,hightestWriteID#" +
Review Comment:
should `hightestWriteID` be lowercased?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #3625: Hive-26563: Add extra columns in Show Compactions output and sort the output
Posted by GitBox <gi...@apache.org>.
deniskuzZ commented on code in PR #3625:
URL: https://github.com/apache/hive/pull/3625#discussion_r984599301
##########
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java:
##########
@@ -3871,46 +3873,35 @@ public ShowCompactResponse showCompact(ShowCompactRequest rqst) throws MetaExcep
PreparedStatement stmt = null;
try {
try {
- StringBuilder sb =new StringBuilder(2048);
- sb.append(
- "SELECT " +
- " \"CQ_DATABASE\", \"CQ_TABLE\", \"CQ_PARTITION\", \"CQ_STATE\", \"CQ_TYPE\", \"CQ_WORKER_ID\", " +
- " \"CQ_START\", -1 \"CC_END\", \"CQ_RUN_AS\", \"CQ_HADOOP_JOB_ID\", \"CQ_ID\", \"CQ_ERROR_MESSAGE\", " +
- " \"CQ_ENQUEUE_TIME\", \"CQ_WORKER_VERSION\", \"CQ_INITIATOR_ID\", \"CQ_INITIATOR_VERSION\", " +
- " \"CQ_CLEANER_START\", \"CQ_POOL_NAME\"" +
- "FROM " +
- " \"COMPACTION_QUEUE\" "
- );
- if (org.apache.commons.lang3.StringUtils.isNotBlank(rqst.getPoolName())) {
- sb.append("WHERE \"CQ_POOL_NAME\" = ? ");
- }
- sb.append(
- "UNION ALL " +
- "SELECT " +
- " \"CC_DATABASE\", \"CC_TABLE\", \"CC_PARTITION\", \"CC_STATE\", \"CC_TYPE\", \"CC_WORKER_ID\", " +
- " \"CC_START\", \"CC_END\", \"CC_RUN_AS\", \"CC_HADOOP_JOB_ID\", \"CC_ID\", \"CC_ERROR_MESSAGE\", " +
- " \"CC_ENQUEUE_TIME\", \"CC_WORKER_VERSION\", \"CC_INITIATOR_ID\", \"CC_INITIATOR_VERSION\", " +
- " -1 , \"CC_POOL_NAME\"" +
- "FROM " +
- " \"COMPLETED_COMPACTIONS\" "
- );
- if (org.apache.commons.lang3.StringUtils.isNotBlank(rqst.getPoolName())) {
- sb.append("WHERE \"CC_POOL_NAME\" = ?");
- }
- //todo: sort by cq_id?
- //what I want is order by cc_end desc, cc_start asc (but derby has a bug https://issues.apache.org/jira/browse/DERBY-6013)
- //to sort so that currently running jobs are at the end of the list (bottom of screen)
- //and currently running ones are in sorted by start time
- //w/o order by likely currently running compactions will be first (LHS of Union)
-
- String query = sb.toString();
+ String query = "SELECT XX.* FROM ( SELECT " +
Review Comment:
We should move the query into the constants, maybe create some TxnQueries class
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #3625: Hive-26563: Add extra columns in Show Compactions output and sort the output
Posted by GitBox <gi...@apache.org>.
deniskuzZ commented on code in PR #3625:
URL: https://github.com/apache/hive/pull/3625#discussion_r991080110
##########
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java:
##########
@@ -3910,47 +3911,15 @@ public ShowCompactResponse showCompact(ShowCompactRequest rqst) throws MetaExcep
PreparedStatement stmt = null;
try {
try {
- StringBuilder sb =new StringBuilder(2048);
- sb.append(
- "SELECT " +
- " \"CQ_DATABASE\", \"CQ_TABLE\", \"CQ_PARTITION\", \"CQ_STATE\", \"CQ_TYPE\", \"CQ_WORKER_ID\", " +
- " \"CQ_START\", -1 \"CC_END\", \"CQ_RUN_AS\", \"CQ_HADOOP_JOB_ID\", \"CQ_ID\", \"CQ_ERROR_MESSAGE\", " +
- " \"CQ_ENQUEUE_TIME\", \"CQ_WORKER_VERSION\", \"CQ_INITIATOR_ID\", \"CQ_INITIATOR_VERSION\", " +
- " \"CQ_CLEANER_START\", \"CQ_POOL_NAME\"" +
- "FROM " +
- " \"COMPACTION_QUEUE\" "
- );
- if (org.apache.commons.lang3.StringUtils.isNotBlank(rqst.getPoolName())) {
- sb.append("WHERE \"CQ_POOL_NAME\" = ? ");
- }
- sb.append(
- "UNION ALL " +
- "SELECT " +
- " \"CC_DATABASE\", \"CC_TABLE\", \"CC_PARTITION\", \"CC_STATE\", \"CC_TYPE\", \"CC_WORKER_ID\", " +
- " \"CC_START\", \"CC_END\", \"CC_RUN_AS\", \"CC_HADOOP_JOB_ID\", \"CC_ID\", \"CC_ERROR_MESSAGE\", " +
- " \"CC_ENQUEUE_TIME\", \"CC_WORKER_VERSION\", \"CC_INITIATOR_ID\", \"CC_INITIATOR_VERSION\", " +
- " -1 , \"CC_POOL_NAME\"" +
- "FROM " +
- " \"COMPLETED_COMPACTIONS\" "
- );
- if (org.apache.commons.lang3.StringUtils.isNotBlank(rqst.getPoolName())) {
- sb.append("WHERE \"CC_POOL_NAME\" = ?");
- }
- //todo: sort by cq_id?
- //what I want is order by cc_end desc, cc_start asc (but derby has a bug https://issues.apache.org/jira/browse/DERBY-6013)
- //to sort so that currently running jobs are at the end of the list (bottom of screen)
- //and currently running ones are in sorted by start time
- //w/o order by likely currently running compactions will be first (LHS of Union)
-
- String query = sb.toString();
+ String query = TxnQueries.SHOW_COMPACTION_QUERY;
+ String whereClause = getFilterClause(rqst);
Review Comment:
Could we name the getFilterClause with some qualifier, not clear what is that filter for. maybe `getShowCompactionFilterClause`.
Result could already contain the 'where' keyword, so no extra if is needed to construct the final query.
````
query += whereClause + TxnQueries.SHOW_COMPACTION_ORDERBY_CLAUSE;
````
Also consider using `StringBuilder`
##########
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/utils/TxnQueries.java:
##########
@@ -0,0 +1,51 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ * <p/>
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * <p/>
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hive.metastore.txn;
+
+public class TxnQueries {
Review Comment:
TxnQueries is duplicated in `metastore-server` and `metastore-common`. Could we please keep just 1 instance?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #3625: Hive-26563: Add extra columns in Show Compactions output and sort the output
Posted by GitBox <gi...@apache.org>.
deniskuzZ commented on code in PR #3625:
URL: https://github.com/apache/hive/pull/3625#discussion_r994544425
##########
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java:
##########
@@ -3905,110 +3908,95 @@ protected static String compactorStateToResponse(char s) {
@RetrySemantics.ReadOnly
@SuppressWarnings("squid:S2095")
public ShowCompactResponse showCompact(ShowCompactRequest rqst) throws MetaException {
- ShowCompactResponse response = new ShowCompactResponse(new ArrayList<>());
- Connection dbConn = null;
- PreparedStatement stmt = null;
try {
- try {
- StringBuilder sb =new StringBuilder(2048);
- sb.append(
- "SELECT " +
- " \"CQ_DATABASE\", \"CQ_TABLE\", \"CQ_PARTITION\", \"CQ_STATE\", \"CQ_TYPE\", \"CQ_WORKER_ID\", " +
- " \"CQ_START\", -1 \"CC_END\", \"CQ_RUN_AS\", \"CQ_HADOOP_JOB_ID\", \"CQ_ID\", \"CQ_ERROR_MESSAGE\", " +
- " \"CQ_ENQUEUE_TIME\", \"CQ_WORKER_VERSION\", \"CQ_INITIATOR_ID\", \"CQ_INITIATOR_VERSION\", " +
- " \"CQ_CLEANER_START\", \"CQ_POOL_NAME\"" +
- "FROM " +
- " \"COMPACTION_QUEUE\" "
- );
- if (org.apache.commons.lang3.StringUtils.isNotBlank(rqst.getPoolName())) {
- sb.append("WHERE \"CQ_POOL_NAME\" = ? ");
- }
- sb.append(
- "UNION ALL " +
- "SELECT " +
- " \"CC_DATABASE\", \"CC_TABLE\", \"CC_PARTITION\", \"CC_STATE\", \"CC_TYPE\", \"CC_WORKER_ID\", " +
- " \"CC_START\", \"CC_END\", \"CC_RUN_AS\", \"CC_HADOOP_JOB_ID\", \"CC_ID\", \"CC_ERROR_MESSAGE\", " +
- " \"CC_ENQUEUE_TIME\", \"CC_WORKER_VERSION\", \"CC_INITIATOR_ID\", \"CC_INITIATOR_VERSION\", " +
- " -1 , \"CC_POOL_NAME\"" +
- "FROM " +
- " \"COMPLETED_COMPACTIONS\" "
- );
- if (org.apache.commons.lang3.StringUtils.isNotBlank(rqst.getPoolName())) {
- sb.append("WHERE \"CC_POOL_NAME\" = ?");
- }
- //todo: sort by cq_id?
- //what I want is order by cc_end desc, cc_start asc (but derby has a bug https://issues.apache.org/jira/browse/DERBY-6013)
- //to sort so that currently running jobs are at the end of the list (bottom of screen)
- //and currently running ones are in sorted by start time
- //w/o order by likely currently running compactions will be first (LHS of Union)
-
- String query = sb.toString();
- dbConn = getDbConn(Connection.TRANSACTION_READ_COMMITTED);
- stmt = dbConn.prepareStatement(query);
- if (org.apache.commons.lang3.StringUtils.isNotBlank(rqst.getPoolName())) {
- stmt.setString(1, rqst.getPoolName());
- stmt.setString(2, rqst.getPoolName());
- }
-
- LOG.debug("Going to execute query <{}>", query);
- ResultSet rs = stmt.executeQuery();
- while (rs.next()) {
- ShowCompactResponseElement e = new ShowCompactResponseElement();
- e.setDbname(rs.getString(1));
- e.setTablename(rs.getString(2));
- e.setPartitionname(rs.getString(3));
- e.setState(compactorStateToResponse(rs.getString(4).charAt(0)));
- try {
- e.setType(dbCompactionType2ThriftType(rs.getString(5).charAt(0)));
- } catch (MetaException ex) {
- //do nothing to handle RU/D if we add another status
- }
- e.setWorkerid(rs.getString(6));
- long start = rs.getLong(7);
- if (!rs.wasNull()) {
- e.setStart(start);
- }
- long endTime = rs.getLong(8);
- if (endTime != -1) {
- e.setEndTime(endTime);
- }
- e.setRunAs(rs.getString(9));
- e.setHadoopJobId(rs.getString(10));
- e.setId(rs.getLong(11));
- e.setErrorMessage(rs.getString(12));
- long enqueueTime = rs.getLong(13);
- if (!rs.wasNull()) {
- e.setEnqueueTime(enqueueTime);
- }
- e.setWorkerVersion(rs.getString(14));
- e.setInitiatorId(rs.getString(15));
- e.setInitiatorVersion(rs.getString(16));
- long cleanerStart = rs.getLong(17);
- if (!rs.wasNull() && (cleanerStart != -1)) {
- e.setCleanerStart(cleanerStart);
- }
- String poolName = rs.getString(18);
- if (org.apache.commons.lang3.StringUtils.isBlank(poolName)) {
- e.setPoolName(DEFAULT_POOL_NAME);
- } else {
- e.setPoolName(poolName);
+ ShowCompactResponse response = new ShowCompactResponse(new ArrayList<>());
+ try (Connection dbConn = getDbConn(Connection.TRANSACTION_READ_COMMITTED)) {
+ StringBuilder query = new StringBuilder(TxnQueries.SHOW_COMPACTION_QUERY).
+ append(getShowCompactionFilterClause(rqst)).append(TxnQueries.SHOW_COMPACTION_ORDERBY_CLAUSE);
+ try (PreparedStatement stmt = sqlGenerator.prepareStmtWithParameters(dbConn, query.toString(),
+ getShowCompactionQueryParamList(rqst))) {
+ LOG.debug("Going to execute query <" + query + ">");
+ ResultSet rs = stmt.executeQuery();
Review Comment:
also wrap rs with try:
````
try (ResultSet rs = pstmt.executeQuery()) {
while (rs.next()) {
....
}
}
````
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a diff in pull request #3625: Hive-26563: Add extra columns in Show Compactions output and sort the output
Posted by GitBox <gi...@apache.org>.
deniskuzZ commented on code in PR #3625:
URL: https://github.com/apache/hive/pull/3625#discussion_r994545742
##########
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java:
##########
@@ -3905,110 +3908,95 @@ protected static String compactorStateToResponse(char s) {
@RetrySemantics.ReadOnly
@SuppressWarnings("squid:S2095")
public ShowCompactResponse showCompact(ShowCompactRequest rqst) throws MetaException {
- ShowCompactResponse response = new ShowCompactResponse(new ArrayList<>());
- Connection dbConn = null;
- PreparedStatement stmt = null;
try {
- try {
- StringBuilder sb =new StringBuilder(2048);
- sb.append(
- "SELECT " +
- " \"CQ_DATABASE\", \"CQ_TABLE\", \"CQ_PARTITION\", \"CQ_STATE\", \"CQ_TYPE\", \"CQ_WORKER_ID\", " +
- " \"CQ_START\", -1 \"CC_END\", \"CQ_RUN_AS\", \"CQ_HADOOP_JOB_ID\", \"CQ_ID\", \"CQ_ERROR_MESSAGE\", " +
- " \"CQ_ENQUEUE_TIME\", \"CQ_WORKER_VERSION\", \"CQ_INITIATOR_ID\", \"CQ_INITIATOR_VERSION\", " +
- " \"CQ_CLEANER_START\", \"CQ_POOL_NAME\"" +
- "FROM " +
- " \"COMPACTION_QUEUE\" "
- );
- if (org.apache.commons.lang3.StringUtils.isNotBlank(rqst.getPoolName())) {
- sb.append("WHERE \"CQ_POOL_NAME\" = ? ");
- }
- sb.append(
- "UNION ALL " +
- "SELECT " +
- " \"CC_DATABASE\", \"CC_TABLE\", \"CC_PARTITION\", \"CC_STATE\", \"CC_TYPE\", \"CC_WORKER_ID\", " +
- " \"CC_START\", \"CC_END\", \"CC_RUN_AS\", \"CC_HADOOP_JOB_ID\", \"CC_ID\", \"CC_ERROR_MESSAGE\", " +
- " \"CC_ENQUEUE_TIME\", \"CC_WORKER_VERSION\", \"CC_INITIATOR_ID\", \"CC_INITIATOR_VERSION\", " +
- " -1 , \"CC_POOL_NAME\"" +
- "FROM " +
- " \"COMPLETED_COMPACTIONS\" "
- );
- if (org.apache.commons.lang3.StringUtils.isNotBlank(rqst.getPoolName())) {
- sb.append("WHERE \"CC_POOL_NAME\" = ?");
- }
- //todo: sort by cq_id?
- //what I want is order by cc_end desc, cc_start asc (but derby has a bug https://issues.apache.org/jira/browse/DERBY-6013)
- //to sort so that currently running jobs are at the end of the list (bottom of screen)
- //and currently running ones are in sorted by start time
- //w/o order by likely currently running compactions will be first (LHS of Union)
-
- String query = sb.toString();
- dbConn = getDbConn(Connection.TRANSACTION_READ_COMMITTED);
- stmt = dbConn.prepareStatement(query);
- if (org.apache.commons.lang3.StringUtils.isNotBlank(rqst.getPoolName())) {
- stmt.setString(1, rqst.getPoolName());
- stmt.setString(2, rqst.getPoolName());
- }
-
- LOG.debug("Going to execute query <{}>", query);
- ResultSet rs = stmt.executeQuery();
- while (rs.next()) {
- ShowCompactResponseElement e = new ShowCompactResponseElement();
- e.setDbname(rs.getString(1));
- e.setTablename(rs.getString(2));
- e.setPartitionname(rs.getString(3));
- e.setState(compactorStateToResponse(rs.getString(4).charAt(0)));
- try {
- e.setType(dbCompactionType2ThriftType(rs.getString(5).charAt(0)));
- } catch (MetaException ex) {
- //do nothing to handle RU/D if we add another status
- }
- e.setWorkerid(rs.getString(6));
- long start = rs.getLong(7);
- if (!rs.wasNull()) {
- e.setStart(start);
- }
- long endTime = rs.getLong(8);
- if (endTime != -1) {
- e.setEndTime(endTime);
- }
- e.setRunAs(rs.getString(9));
- e.setHadoopJobId(rs.getString(10));
- e.setId(rs.getLong(11));
- e.setErrorMessage(rs.getString(12));
- long enqueueTime = rs.getLong(13);
- if (!rs.wasNull()) {
- e.setEnqueueTime(enqueueTime);
- }
- e.setWorkerVersion(rs.getString(14));
- e.setInitiatorId(rs.getString(15));
- e.setInitiatorVersion(rs.getString(16));
- long cleanerStart = rs.getLong(17);
- if (!rs.wasNull() && (cleanerStart != -1)) {
- e.setCleanerStart(cleanerStart);
- }
- String poolName = rs.getString(18);
- if (org.apache.commons.lang3.StringUtils.isBlank(poolName)) {
- e.setPoolName(DEFAULT_POOL_NAME);
- } else {
- e.setPoolName(poolName);
+ ShowCompactResponse response = new ShowCompactResponse(new ArrayList<>());
+ try (Connection dbConn = getDbConn(Connection.TRANSACTION_READ_COMMITTED)) {
+ StringBuilder query = new StringBuilder(TxnQueries.SHOW_COMPACTION_QUERY).
+ append(getShowCompactionFilterClause(rqst)).append(TxnQueries.SHOW_COMPACTION_ORDERBY_CLAUSE);
Review Comment:
put appends on new lines
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sonarcloud[bot] commented on pull request #3625: Hive-26563: Add extra columns in Show Compactions output and sort the output
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on PR #3625:
URL: https://github.com/apache/hive/pull/3625#issuecomment-1280715912
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=3625)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3625&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3625&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3625&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=CODE_SMELL) [5 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3625&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3625&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3625&metric=duplicated_lines_density&view=list) No Duplication information
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org