You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@tez.apache.org by "Hadoop QA (Jira)" <ji...@apache.org> on 2020/10/09 09:48:00 UTC
[jira] [Commented] (TEZ-4236) Investigate if DAGAppMaster's
DAGClientServer is really need to be started/used in local mode
[ https://issues.apache.org/jira/browse/TEZ-4236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17210770#comment-17210770 ]
Hadoop QA commented on TEZ-4236:
--------------------------------
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 19m 10s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m 0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 1 new or modified test files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 28s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m 25s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 24s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m 46s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 50s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 0m 47s{color} | {color:blue} Used deprecated FindBugs config; considering switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 23s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 8s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 58s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange} 0m 14s{color} | {color:orange} tez-api: The patch generated 7 new + 130 unchanged - 1 fixed = 137 total (was 131) {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange} 0m 21s{color} | {color:orange} tez-dag: The patch generated 10 new + 105 unchanged - 0 fixed = 115 total (was 105) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 35s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 1m 47s{color} | {color:red} tez-api in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 4m 28s{color} | {color:green} tez-dag in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 39m 44s{color} | {color:red} tez-tests in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 27s{color} | {color:green} The patch does not generate ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 88m 55s{color} | {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | tez.client.TestTezClient |
| | tez.test.TestDAGRecovery2 |
| | tez.test.TestLocalMode |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/PreCommit-TEZ-Build/35/artifact/out/Dockerfile |
| JIRA Issue | TEZ-4236 |
| JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/13013290/TEZ-4236.01.patch |
| Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs checkstyle compile |
| uname | Linux dd059eb3337c 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | personality/tez.sh |
| git revision | master / 6e9c1b2d6 |
| Default Java | Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| checkstyle | https://ci-hadoop.apache.org/job/PreCommit-TEZ-Build/35/artifact/out/diff-checkstyle-tez-api.txt |
| checkstyle | https://ci-hadoop.apache.org/job/PreCommit-TEZ-Build/35/artifact/out/diff-checkstyle-tez-dag.txt |
| unit | https://ci-hadoop.apache.org/job/PreCommit-TEZ-Build/35/artifact/out/patch-unit-tez-api.txt |
| unit | https://ci-hadoop.apache.org/job/PreCommit-TEZ-Build/35/artifact/out/patch-unit-tez-tests.txt |
| Test Results | https://ci-hadoop.apache.org/job/PreCommit-TEZ-Build/35/testReport/ |
| Max. process+thread count | 1228 (vs. ulimit of 5500) |
| modules | C: tez-api tez-dag tez-tests U: . |
| Console output | https://ci-hadoop.apache.org/job/PreCommit-TEZ-Build/35/console |
| versions | git=2.17.1 maven=3.6.0 findbugs=3.0.1 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
> Investigate if DAGAppMaster's DAGClientServer is really need to be started/used in local mode
> ---------------------------------------------------------------------------------------------
>
> Key: TEZ-4236
> URL: https://issues.apache.org/jira/browse/TEZ-4236
> Project: Apache Tez
> Issue Type: Improvement
> Reporter: László Bodor
> Assignee: László Bodor
> Priority: Major
> Attachments: TEZ-4236.01.patch
>
>
> For a customer, we found that they try to run hive on tez local mode tests, and due to strict firewall rules and security policy, the DagClientServer struggles to start or respond to connections (think about corporate laptops here). Instead of investigating the network-related workarounds, let's find another way and decide if DAGAppMaster really needs this server.
> LocalClient has an embedded [DAGAppMaster|https://github.com/apache/tez/blob/master/tez-dag/src/main/java/org/apache/tez/client/LocalClient.java#L65], so at the first sight, it doesn't make sense to rely on the network environment in terms of AM usage. Hopefully, AM can become more lightweight in the long term if we try to eliminate pointless network usage.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)