You are viewing a plain text version of this content. The canonical link for it is here.
Posted to gitbox@hive.apache.org by GitBox <gi...@apache.org> on 2021/04/06 11:17:06 UTC
[GitHub] [hive] ayushtkn opened a new pull request #2154: HIVE-24978. Optimise number of DROP_PARTITION events created.
ayushtkn opened a new pull request #2154:
URL: https://github.com/apache/hive/pull/2154
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] aasha commented on a change in pull request #2154: HIVE-24978. Optimise number of DROP_PARTITION events created.
Posted by GitBox <gi...@apache.org>.
aasha commented on a change in pull request #2154:
URL: https://github.com/apache/hive/pull/2154#discussion_r612939857
##########
File path: itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/parse/TestReplicationScenarios.java
##########
@@ -4294,6 +4295,9 @@ public void testMoveOptimizationIncremental() throws IOException {
@Test
public void testDatabaseInJobName() throws Throwable {
+ // Clean up configurations
+ driver.getConf().set(JobContext.JOB_NAME, "");
Review comment:
can this clean up be done at tear down?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] aasha merged pull request #2154: HIVE-24978. Optimise number of DROP_PARTITION events created.
Posted by GitBox <gi...@apache.org>.
aasha merged pull request #2154:
URL: https://github.com/apache/hive/pull/2154
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] ayushtkn commented on a change in pull request #2154: HIVE-24978. Optimise number of DROP_PARTITION events created.
Posted by GitBox <gi...@apache.org>.
ayushtkn commented on a change in pull request #2154:
URL: https://github.com/apache/hive/pull/2154#discussion_r613045613
##########
File path: itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/parse/TestReplicationScenarios.java
##########
@@ -4294,6 +4295,9 @@ public void testMoveOptimizationIncremental() throws IOException {
@Test
public void testDatabaseInJobName() throws Throwable {
+ // Clean up configurations
+ driver.getConf().set(JobContext.JOB_NAME, "");
Review comment:
came from HIVE-25000, Removed itself from my code
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] ayushtkn commented on a change in pull request #2154: HIVE-24978. Optimise number of DROP_PARTITION events created.
Posted by GitBox <gi...@apache.org>.
ayushtkn commented on a change in pull request #2154:
URL: https://github.com/apache/hive/pull/2154#discussion_r611252750
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/ddl/table/partition/drop/AlterTableDropPartitionOperation.java
##########
@@ -120,6 +126,12 @@ private void dropPartitions() throws HiveException {
List<Partition> droppedPartitions = context.getDb().dropPartitions(tablenName.getDb(), tablenName.getTable(),
partitionExpressions, options);
+ if (isRepl) {
Review comment:
It does a lot of stuff below related to ``llap`` and printing to console, which I thought would be irrelevant here, so I added this check, So in case of replication, I skip the below part and return from here itself
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] ayushtkn commented on a change in pull request #2154: HIVE-24978. Optimise number of DROP_PARTITION events created.
Posted by GitBox <gi...@apache.org>.
ayushtkn commented on a change in pull request #2154:
URL: https://github.com/apache/hive/pull/2154#discussion_r611252750
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/ddl/table/partition/drop/AlterTableDropPartitionOperation.java
##########
@@ -120,6 +126,12 @@ private void dropPartitions() throws HiveException {
List<Partition> droppedPartitions = context.getDb().dropPartitions(tablenName.getDb(), tablenName.getTable(),
partitionExpressions, options);
+ if (isRepl) {
Review comment:
It does a lot of stuff below related to ``llap`` and printing to console, which I thought would be irrelevant here, so I added this check, So in case of replication, I skip the below part and return from here itself
There is something like this below:
``
// We have already locked the table, don't lock the partitions.
DDLUtils.addIfAbsentByName(new WriteEntity(partition, WriteEntity.WriteType.DDL_NO_LOCK), context);
``
This ddlUtils stuff only, I am not very sure, Let me know if you find it relevant, I will add it in my block as well
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] aasha commented on a change in pull request #2154: HIVE-24978. Optimise number of DROP_PARTITION events created.
Posted by GitBox <gi...@apache.org>.
aasha commented on a change in pull request #2154:
URL: https://github.com/apache/hive/pull/2154#discussion_r609321577
##########
File path: standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/events/DropPartitionEvent.java
##########
@@ -45,6 +45,14 @@ public DropPartitionEvent (Table table,
this.deleteData = deleteData;
}
+ public DropPartitionEvent(Table table, Iterable<Partition> partition, boolean status, boolean deleteData,
+ IHMSHandler handler) {
+ super(status, handler);
+ this.table = table;
+ this.partitions = partition;
Review comment:
rename to partitions
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/ddl/table/partition/drop/AlterTableDropPartitionOperation.java
##########
@@ -120,6 +126,12 @@ private void dropPartitions() throws HiveException {
List<Partition> droppedPartitions = context.getDb().dropPartitions(tablenName.getDb(), tablenName.getTable(),
partitionExpressions, options);
+ if (isRepl) {
Review comment:
what is this check
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org