[jira] [Commented] (HIVE-8311) Driver is encoding transaction information too late

["Alan Gates (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HIVE-8311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14154090#comment-14154090 ] 

Alan Gates commented on HIVE-8311:
----------------------------------

bq. So now it's possible that txn list is computed...
If another txn runs between when a query generates its txn list and when it acquires locks that's fine because this txn won't see the results of that txn.  I think this is more reasonable then having the query wait and generate its txn state after locks, because that makes the results of the query variable based on when locks are obtained instead of when it is submitted.

(Note that this issue exists no matter what.  Even once locks are acquired it may take some time for the engine (Tez or MR) to submit the job.  Also, once we support cross query transactions we'll have multiple queries where we are acquiring new sets of locks.)

If the compactor runs between when the query gets its txn list and when it obtains locks we should still be fine.  By definition the compactor does not change the set of valid data.  Some txns listed as aborted in the query's list may now be removed from the file, but that's not a problem.

Have I missed anything?

bq. Does hive cache query plans? 
Not at the moment.

> Driver is encoding transaction information too late
> ---------------------------------------------------
>
>                 Key: HIVE-8311
>                 URL: https://issues.apache.org/jira/browse/HIVE-8311
>             Project: Hive
>          Issue Type: Bug
>          Components: Transactions
>    Affects Versions: 0.14.0
>            Reporter: Alan Gates
>            Assignee: Alan Gates
>            Priority: Blocker
>             Fix For: 0.14.0
>
>         Attachments: HIVE-8311.patch
>
>
> Currently Driver is obtaining the transaction information and encoding it in the conf in runInternal.  But this is too late, as the query has already been planned.  Either we need to change the plan when this info is obtained or we need to obtain it at compile time.  This bug was introduced by HIVE-8203.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)