[jira] [Work logged] (HIVE-25159) Remove support for ordered results in llap external client library

["ASF GitHub Bot (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HIVE-25159?focusedWorklogId=602091&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-602091 ]

ASF GitHub Bot logged work on HIVE-25159:
-----------------------------------------

                Author: ASF GitHub Bot
            Created on: 26/May/21 04:48
            Start Date: 26/May/21 04:48
    Worklog Time Spent: 10m 
      Work Description: ShubhamChaurasia opened a new pull request #2318:
URL: https://github.com/apache/hive/pull/2318


   The issue with order by in llap external client is described in https://issues.apache.org/jira/browse/HIVE-25159
   
   ### What changes were proposed in this pull request?
   To resolve the above issue, we propose removing of order by handling in the llap external client library and let the clients do it at their end.
   
   ### Why are the changes needed?
   1. These changes are needed to make llap external client queries more performant, earlier they used to generate only a single split for order by queries.
   2. Removing order by support also helps us to get rid of exception arising to due false detection of order by clause (it was specific to spark-llap and was done for that only).
   
   
   ### Does this PR introduce _any_ user-facing change?
   It removes a config - hive.llap.external.splits.order.by.force.single.split
   
   
   ### How was this patch tested?
   Through existing unit tests 
   - org.apache.hive.jdbc.TestJdbcGenericUDTFGetSplits
   - org.apache.hive.jdbc.TestJdbcGenericUDTFGetSplits2


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Issue Time Tracking
-------------------

            Worklog Id:     (was: 602091)
    Remaining Estimate: 0h
            Time Spent: 10m

> Remove support for ordered results in llap external client library
> ------------------------------------------------------------------
>
>                 Key: HIVE-25159
>                 URL: https://issues.apache.org/jira/browse/HIVE-25159
>             Project: Hive
>          Issue Type: Bug
>          Components: Clients, Hive
>            Reporter: Shubham Chaurasia
>            Assignee: Shubham Chaurasia
>            Priority: Major
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> Currently when querying via llap external client framework, in case of order by queries -
> 1. Due to the fact that spark-llap used to wrap actual query in a subquery as mentioned in [HIVE-19794|https://issues.apache.org/jira/browse/HIVE-19794]
> a) We had to detect order by like - 
> {code}
> orderByQuery = plan.getQueryProperties().hasOrderBy() || plan.getQueryProperties().hasOuterOrderBy();
> {code}
> Due to this we recently saw an exception like below for one of the queries that did not have an outer order by (It was having an order by in a subquery)
> {code}
> org.apache.hive.service.cli.HiveSQLException: java.io.IOException: org.apache.hadoop.hive.ql.metadata.HiveException: java.io.IOException:
> java.lang.IllegalStateException: Requested to generate single split. Paths and fileStatuses are expected to be 1. Got paths: 1 fileStatuses: 7
> {code}
> b) Also we had to disable following optimization - 
> {code}
> HiveConf.setBoolVar(conf, ConfVars.HIVE_REMOVE_ORDERBY_IN_SUBQUERY, false);
> {code}
> 2. By default we have {{hive.llap.external.splits.order.by.force.single.split=true}} which forces us to generate single split leading to performance bottleneck. 
> We should remove ordering support altogether from llap external client repo and let clients handle it at their end.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)