You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Siddharth Seth <ss...@apache.org> on 2016/06/16 21:13:58 UTC
Review Request 48818: See jira
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48818/
-----------------------------------------------------------
Review request for hive and Sergey Shelukhin.
Bugs: HIVE-14023
https://issues.apache.org/jira/browse/HIVE-14023
Repository: hive-git
Description
-------
HIVE-14023. LLAP: Make the Hive query id available in ContainerRunner
Diffs
-----
llap-client/src/java/org/apache/hadoop/hive/llap/ext/LlapTaskUmbilicalExternalClient.java 5f250b4
llap-common/src/gen/protobuf/gen-java/org/apache/hadoop/hive/llap/daemon/rpc/LlapDaemonProtocolProtos.java 56a1361
llap-common/src/java/org/apache/hadoop/hive/llap/tez/Converters.java 01dc2e1
llap-common/src/protobuf/LlapDaemonProtocol.proto 92dda21
llap-common/src/test/org/apache/hadoop/hive/llap/tez/TestConverters.java 85c6091
llap-ext-client/src/java/org/apache/hadoop/hive/llap/LlapBaseInputFormat.java 46030ec
llap-server/src/java/org/apache/hadoop/hive/llap/daemon/impl/ContainerRunnerImpl.java ded84c1
llap-server/src/java/org/apache/hadoop/hive/llap/daemon/impl/QueryTracker.java a965872
llap-server/src/java/org/apache/hadoop/hive/llap/daemon/impl/TaskRunnerCallable.java 6c853a6
llap-server/src/test/org/apache/hadoop/hive/llap/daemon/impl/TaskExecutorTestHelpers.java fe2ced5
llap-server/src/test/org/apache/hadoop/hive/llap/daemon/impl/comparator/TestFirstInFirstOutComparator.java ac48a3a
llap-tez/src/java/org/apache/hadoop/hive/llap/tezplugins/LlapTaskCommunicator.java fcf3378
llap-tez/src/java/org/apache/hadoop/hive/llap/tezplugins/helpers/SourceStateTracker.java 3dd73f6
llap-tez/src/test/org/apache/hadoop/hive/llap/tezplugins/TestLlapTaskCommunicator.java 1901328
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDTFGetSplits.java bdf254b
Diff: https://reviews.apache.org/r/48818/diff/
Testing
-------
Thanks,
Siddharth Seth
Re: Review Request 48818: HIVE-14023. LLAP: Make the Hive query id
available in ContainerRunner
Posted by Siddharth Seth <ss...@apache.org>.
> On June 17, 2016, 12:55 a.m., Sergey Shelukhin wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDTFGetSplits.java, line 436
> > <https://reviews.apache.org/r/48818/diff/1/?file=1421793#file1421793line436>
> >
> > dag index always 0?
Changed to pick this up from TaskSpec. (that sets it to 0 - the assumption being the appId is unique)
> On June 17, 2016, 12:55 a.m., Sergey Shelukhin wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDTFGetSplits.java, line 378
> > <https://reviews.apache.org/r/48818/diff/1/?file=1421793#file1421793line378>
> >
> > is makeQueryId unique? Should we just use the same thing we use for generated applicationId? Makes for less things for the user to track.
Supposed to be unique. "sseth_20160616180332_06e3b03e-a9ec-4e36-be54-17374c18d339" is an example.
I'll change it to the appId though - single construct does work better.
> On June 17, 2016, 12:55 a.m., Sergey Shelukhin wrote:
> > llap-tez/src/java/org/apache/hadoop/hive/llap/tezplugins/LlapTaskCommunicator.java, line 269
> > <https://reviews.apache.org/r/48818/diff/1/?file=1421790#file1421790line269>
> >
> > follow up JIRA? probably under bulk upgrade-to-Tez-v.X one
Will create one.
> On June 17, 2016, 12:55 a.m., Sergey Shelukhin wrote:
> > llap-server/src/java/org/apache/hadoop/hive/llap/daemon/impl/ContainerRunnerImpl.java, line 313
> > <https://reviews.apache.org/r/48818/diff/1/?file=1421785#file1421785line313>
> >
> > why is this removed? it can be useful for other communication purposes
There's nothing for it to do. It can be re-introduced if required ? (The Hook interface still exists in TezProcessor).
- Siddharth
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48818/#review138141
-----------------------------------------------------------
On June 16, 2016, 9:16 p.m., Siddharth Seth wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48818/
> -----------------------------------------------------------
>
> (Updated June 16, 2016, 9:16 p.m.)
>
>
> Review request for hive and Sergey Shelukhin.
>
>
> Bugs: HIVE-14023
> https://issues.apache.org/jira/browse/HIVE-14023
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> HIVE-14023. LLAP: Make the Hive query id available in ContainerRunner
>
>
> Diffs
> -----
>
> llap-client/src/java/org/apache/hadoop/hive/llap/ext/LlapTaskUmbilicalExternalClient.java 5f250b4
> llap-common/src/gen/protobuf/gen-java/org/apache/hadoop/hive/llap/daemon/rpc/LlapDaemonProtocolProtos.java 56a1361
> llap-common/src/java/org/apache/hadoop/hive/llap/tez/Converters.java 01dc2e1
> llap-common/src/protobuf/LlapDaemonProtocol.proto 92dda21
> llap-common/src/test/org/apache/hadoop/hive/llap/tez/TestConverters.java 85c6091
> llap-ext-client/src/java/org/apache/hadoop/hive/llap/LlapBaseInputFormat.java 46030ec
> llap-server/src/java/org/apache/hadoop/hive/llap/daemon/impl/ContainerRunnerImpl.java ded84c1
> llap-server/src/java/org/apache/hadoop/hive/llap/daemon/impl/QueryTracker.java a965872
> llap-server/src/java/org/apache/hadoop/hive/llap/daemon/impl/TaskRunnerCallable.java 6c853a6
> llap-server/src/test/org/apache/hadoop/hive/llap/daemon/impl/TaskExecutorTestHelpers.java fe2ced5
> llap-server/src/test/org/apache/hadoop/hive/llap/daemon/impl/comparator/TestFirstInFirstOutComparator.java ac48a3a
> llap-tez/src/java/org/apache/hadoop/hive/llap/tezplugins/LlapTaskCommunicator.java fcf3378
> llap-tez/src/java/org/apache/hadoop/hive/llap/tezplugins/helpers/SourceStateTracker.java 3dd73f6
> llap-tez/src/test/org/apache/hadoop/hive/llap/tezplugins/TestLlapTaskCommunicator.java 1901328
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDTFGetSplits.java bdf254b
>
> Diff: https://reviews.apache.org/r/48818/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Siddharth Seth
>
>
Re: Review Request 48818: HIVE-14023. LLAP: Make the Hive query id
available in ContainerRunner
Posted by Sergey Shelukhin <se...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48818/#review138141
-----------------------------------------------------------
llap-server/src/java/org/apache/hadoop/hive/llap/daemon/impl/ContainerRunnerImpl.java
<https://reviews.apache.org/r/48818/#comment203354>
why is this removed? it can be useful for other communication purposes
llap-tez/src/java/org/apache/hadoop/hive/llap/tezplugins/LlapTaskCommunicator.java (line 266)
<https://reviews.apache.org/r/48818/#comment203355>
follow up JIRA? probably under bulk upgrade-to-Tez-v.X one
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDTFGetSplits.java (line 378)
<https://reviews.apache.org/r/48818/#comment203357>
is makeQueryId unique? Should we just use the same thing we use for generated applicationId? Makes for less things for the user to track.
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDTFGetSplits.java (line 436)
<https://reviews.apache.org/r/48818/#comment203356>
dag index always 0?
- Sergey Shelukhin
On June 16, 2016, 9:16 p.m., Siddharth Seth wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48818/
> -----------------------------------------------------------
>
> (Updated June 16, 2016, 9:16 p.m.)
>
>
> Review request for hive and Sergey Shelukhin.
>
>
> Bugs: HIVE-14023
> https://issues.apache.org/jira/browse/HIVE-14023
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> HIVE-14023. LLAP: Make the Hive query id available in ContainerRunner
>
>
> Diffs
> -----
>
> llap-client/src/java/org/apache/hadoop/hive/llap/ext/LlapTaskUmbilicalExternalClient.java 5f250b4
> llap-common/src/gen/protobuf/gen-java/org/apache/hadoop/hive/llap/daemon/rpc/LlapDaemonProtocolProtos.java 56a1361
> llap-common/src/java/org/apache/hadoop/hive/llap/tez/Converters.java 01dc2e1
> llap-common/src/protobuf/LlapDaemonProtocol.proto 92dda21
> llap-common/src/test/org/apache/hadoop/hive/llap/tez/TestConverters.java 85c6091
> llap-ext-client/src/java/org/apache/hadoop/hive/llap/LlapBaseInputFormat.java 46030ec
> llap-server/src/java/org/apache/hadoop/hive/llap/daemon/impl/ContainerRunnerImpl.java ded84c1
> llap-server/src/java/org/apache/hadoop/hive/llap/daemon/impl/QueryTracker.java a965872
> llap-server/src/java/org/apache/hadoop/hive/llap/daemon/impl/TaskRunnerCallable.java 6c853a6
> llap-server/src/test/org/apache/hadoop/hive/llap/daemon/impl/TaskExecutorTestHelpers.java fe2ced5
> llap-server/src/test/org/apache/hadoop/hive/llap/daemon/impl/comparator/TestFirstInFirstOutComparator.java ac48a3a
> llap-tez/src/java/org/apache/hadoop/hive/llap/tezplugins/LlapTaskCommunicator.java fcf3378
> llap-tez/src/java/org/apache/hadoop/hive/llap/tezplugins/helpers/SourceStateTracker.java 3dd73f6
> llap-tez/src/test/org/apache/hadoop/hive/llap/tezplugins/TestLlapTaskCommunicator.java 1901328
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDTFGetSplits.java bdf254b
>
> Diff: https://reviews.apache.org/r/48818/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Siddharth Seth
>
>
Re: Review Request 48818: HIVE-14023. LLAP: Make the Hive query id
available in ContainerRunner
Posted by Siddharth Seth <ss...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48818/
-----------------------------------------------------------
(Updated June 17, 2016, 5:44 a.m.)
Review request for hive and Sergey Shelukhin.
Changes
-------
Without generated code.
Bugs: HIVE-14023
https://issues.apache.org/jira/browse/HIVE-14023
Repository: hive-git
Description
-------
HIVE-14023. LLAP: Make the Hive query id available in ContainerRunner
Diffs (updated)
-----
llap-client/src/java/org/apache/hadoop/hive/llap/ext/LlapTaskUmbilicalExternalClient.java 5f250b4
llap-common/src/java/org/apache/hadoop/hive/llap/tez/Converters.java 01dc2e1
llap-common/src/protobuf/LlapDaemonProtocol.proto 92dda21
llap-common/src/test/org/apache/hadoop/hive/llap/tez/TestConverters.java 85c6091
llap-ext-client/src/java/org/apache/hadoop/hive/llap/LlapBaseInputFormat.java 46030ec
llap-server/src/java/org/apache/hadoop/hive/llap/daemon/impl/ContainerRunnerImpl.java ded84c1
llap-server/src/java/org/apache/hadoop/hive/llap/daemon/impl/QueryTracker.java a965872
llap-server/src/java/org/apache/hadoop/hive/llap/daemon/impl/TaskRunnerCallable.java 6c853a6
llap-server/src/test/org/apache/hadoop/hive/llap/daemon/impl/TaskExecutorTestHelpers.java fe2ced5
llap-server/src/test/org/apache/hadoop/hive/llap/daemon/impl/comparator/TestFirstInFirstOutComparator.java ac48a3a
llap-tez/src/java/org/apache/hadoop/hive/llap/tezplugins/LlapTaskCommunicator.java fcf3378
llap-tez/src/java/org/apache/hadoop/hive/llap/tezplugins/helpers/SourceStateTracker.java 3dd73f6
llap-tez/src/test/org/apache/hadoop/hive/llap/tezplugins/TestLlapTaskCommunicator.java 1901328
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDTFGetSplits.java bdf254b
Diff: https://reviews.apache.org/r/48818/diff/
Testing
-------
Thanks,
Siddharth Seth
Re: Review Request 48818: HIVE-14023. LLAP: Make the Hive query id
available in ContainerRunner
Posted by Siddharth Seth <ss...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48818/
-----------------------------------------------------------
(Updated June 17, 2016, 5:43 a.m.)
Review request for hive and Sergey Shelukhin.
Bugs: HIVE-14023
https://issues.apache.org/jira/browse/HIVE-14023
Repository: hive-git
Description
-------
HIVE-14023. LLAP: Make the Hive query id available in ContainerRunner
Diffs (updated)
-----
llap-client/src/java/org/apache/hadoop/hive/llap/ext/LlapTaskUmbilicalExternalClient.java 5f250b4
llap-common/src/gen/protobuf/gen-java/org/apache/hadoop/hive/llap/daemon/rpc/LlapDaemonProtocolProtos.java 56a1361
llap-common/src/java/org/apache/hadoop/hive/llap/tez/Converters.java 01dc2e1
llap-common/src/protobuf/LlapDaemonProtocol.proto 92dda21
llap-common/src/test/org/apache/hadoop/hive/llap/tez/TestConverters.java 85c6091
llap-ext-client/src/java/org/apache/hadoop/hive/llap/LlapBaseInputFormat.java 46030ec
llap-server/src/java/org/apache/hadoop/hive/llap/daemon/impl/ContainerRunnerImpl.java ded84c1
llap-server/src/java/org/apache/hadoop/hive/llap/daemon/impl/QueryTracker.java a965872
llap-server/src/java/org/apache/hadoop/hive/llap/daemon/impl/TaskRunnerCallable.java 6c853a6
llap-server/src/test/org/apache/hadoop/hive/llap/daemon/impl/TaskExecutorTestHelpers.java fe2ced5
llap-server/src/test/org/apache/hadoop/hive/llap/daemon/impl/comparator/TestFirstInFirstOutComparator.java ac48a3a
llap-tez/src/java/org/apache/hadoop/hive/llap/tezplugins/LlapTaskCommunicator.java fcf3378
llap-tez/src/java/org/apache/hadoop/hive/llap/tezplugins/helpers/SourceStateTracker.java 3dd73f6
llap-tez/src/test/org/apache/hadoop/hive/llap/tezplugins/TestLlapTaskCommunicator.java 1901328
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDTFGetSplits.java bdf254b
Diff: https://reviews.apache.org/r/48818/diff/
Testing
-------
Thanks,
Siddharth Seth
Re: Review Request 48818: HIVE-14023. LLAP: Make the Hive query id
available in ContainerRunner
Posted by Siddharth Seth <ss...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48818/
-----------------------------------------------------------
(Updated June 16, 2016, 9:16 p.m.)
Review request for hive and Sergey Shelukhin.
Summary (updated)
-----------------
HIVE-14023. LLAP: Make the Hive query id available in ContainerRunner
Bugs: HIVE-14023
https://issues.apache.org/jira/browse/HIVE-14023
Repository: hive-git
Description
-------
HIVE-14023. LLAP: Make the Hive query id available in ContainerRunner
Diffs
-----
llap-client/src/java/org/apache/hadoop/hive/llap/ext/LlapTaskUmbilicalExternalClient.java 5f250b4
llap-common/src/gen/protobuf/gen-java/org/apache/hadoop/hive/llap/daemon/rpc/LlapDaemonProtocolProtos.java 56a1361
llap-common/src/java/org/apache/hadoop/hive/llap/tez/Converters.java 01dc2e1
llap-common/src/protobuf/LlapDaemonProtocol.proto 92dda21
llap-common/src/test/org/apache/hadoop/hive/llap/tez/TestConverters.java 85c6091
llap-ext-client/src/java/org/apache/hadoop/hive/llap/LlapBaseInputFormat.java 46030ec
llap-server/src/java/org/apache/hadoop/hive/llap/daemon/impl/ContainerRunnerImpl.java ded84c1
llap-server/src/java/org/apache/hadoop/hive/llap/daemon/impl/QueryTracker.java a965872
llap-server/src/java/org/apache/hadoop/hive/llap/daemon/impl/TaskRunnerCallable.java 6c853a6
llap-server/src/test/org/apache/hadoop/hive/llap/daemon/impl/TaskExecutorTestHelpers.java fe2ced5
llap-server/src/test/org/apache/hadoop/hive/llap/daemon/impl/comparator/TestFirstInFirstOutComparator.java ac48a3a
llap-tez/src/java/org/apache/hadoop/hive/llap/tezplugins/LlapTaskCommunicator.java fcf3378
llap-tez/src/java/org/apache/hadoop/hive/llap/tezplugins/helpers/SourceStateTracker.java 3dd73f6
llap-tez/src/test/org/apache/hadoop/hive/llap/tezplugins/TestLlapTaskCommunicator.java 1901328
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDTFGetSplits.java bdf254b
Diff: https://reviews.apache.org/r/48818/diff/
Testing
-------
Thanks,
Siddharth Seth