You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Jason Dere <jd...@hortonworks.com> on 2017/07/13 00:09:30 UTC
Re: Review Request 60259: HIVE-16926 LlapTaskUmbilicalExternalClient
should not start new umbilical server for every fragment request
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60259/
-----------------------------------------------------------
(Updated July 13, 2017, 12:09 a.m.)
Review request for hive, Sergey Shelukhin and Siddharth Seth.
Changes
-------
Updating patch per feedback from Sid
Bugs: HIVE-16926
https://issues.apache.org/jira/browse/HIVE-16926
Repository: hive-git
Description
-------
Initial patch, restructured the LlapTaskUmbilicalExternalClient code a bit.
- Uses shared LLAP umbilical server rather than a new server per external client
- Retries rejected submissions (WorkSubmitter helper class)
- No more deferred cleanup (from HIVE-16652). One thing about this is that once clients are closed/unregistered, communicator.stop() is called and it's removed from the registered list of clients. So we might get a few warning messages about untracked taskAttemptIds coming in during heartbeat() .. if this is undesirable we might be able to leave them in the registeredClients list (but ignore heartbeats to them as they are tagged as closed), and remove them using the HeartbeatCheckTask once they get too old.
Diffs (updated)
-----
llap-client/src/java/org/apache/hadoop/hive/llap/LlapBaseRecordReader.java 7fff147
llap-client/src/java/org/apache/hadoop/hive/llap/SubmitWorkInfo.java 95b0ffc
llap-client/src/java/org/apache/hadoop/hive/llap/ext/LlapTaskUmbilicalExternalClient.java 406bdda
llap-client/src/java/org/apache/hadoop/hive/llap/tezplugins/helpers/LlapTaskUmbilicalServer.java 403381d
llap-ext-client/src/java/org/apache/hadoop/hive/llap/LlapBaseInputFormat.java eb93241
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDTFGetSplits.java d4ec44e
Diff: https://reviews.apache.org/r/60259/diff/4/
Changes: https://reviews.apache.org/r/60259/diff/3-4/
Testing
-------
Thanks,
Jason Dere