[GitHub] [hive] abstractdog commented on a change in pull request #1556: HIVE-24207: LimitOperator can leverage ObjectCache to bail out quickly

[GitBox <gi...@apache.org>]

abstractdog commented on a change in pull request #1556:
URL: https://github.com/apache/hive/pull/1556#discussion_r540004959



##########
File path: ql/src/test/queries/clientpositive/authorization_view_1.q
##########
@@ -1,5 +1,6 @@
 --! qt:dataset:src
 set hive.security.authorization.manager=org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider;
+set hive.exec.reducers.max=1;

Review comment:
       yes, as this patch changes the behavior of all queries having limit, basically, all queries with >1 reducers are subject to intermittent result changes, because of making reducers exit early
   that's what I experienced in early precommit runs, and I didn't want to introduce ORDER BY clauses

##########
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/LimitOperator.java
##########
@@ -19,11 +19,18 @@
 package org.apache.hadoop.hive.ql.exec;
 
 import java.io.Serializable;
+import java.util.concurrent.Callable;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicInteger;
 
 import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.hive.conf.HiveConf;
 import org.apache.hadoop.hive.ql.CompilationOpContext;
+import org.apache.hadoop.hive.ql.exec.tez.DagUtils;
+import org.apache.hadoop.hive.ql.exec.tez.LlapObjectCache;
 import org.apache.hadoop.hive.ql.metadata.HiveException;
 import org.apache.hadoop.hive.ql.plan.LimitDesc;
+import org.apache.hadoop.hive.ql.plan.OperatorDesc;

Review comment:
       I'll do




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org