You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@tez.apache.org by "TezQA (JIRA)" <ji...@apache.org> on 2019/04/02 19:41:00 UTC
[jira] [Commented] (TEZ-3991) Unmanaged tez sessions
[ https://issues.apache.org/jira/browse/TEZ-3991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16808092#comment-16808092 ]
TezQA commented on TEZ-3991:
----------------------------
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 53m 26s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 8 new or modified test files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 3m 19s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m 48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m 40s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m 45s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s{color} | {color:blue} Skipped patched modules with no Java source: . {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 58s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 4m 1s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 24s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m 40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m 18s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 35s{color} | {color:red} tez-dag generated 1 new + 7 unchanged - 1 fixed = 8 total (was 8) {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 1m 32s{color} | {color:red} root generated 1 new + 39 unchanged - 1 fixed = 40 total (was 40) {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange} 0m 22s{color} | {color:orange} tez-api: The patch generated 47 new + 130 unchanged - 4 fixed = 177 total (was 134) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 17s{color} | {color:green} tez-common: The patch generated 0 new + 69 unchanged - 5 fixed = 69 total (was 74) {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange} 0m 27s{color} | {color:orange} tez-dag: The patch generated 49 new + 138 unchanged - 0 fixed = 187 total (was 138) {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange} 0m 20s{color} | {color:orange} tez-tests: The patch generated 14 new + 5 unchanged - 0 fixed = 19 total (was 5) {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange} 0m 48s{color} | {color:orange} root: The patch generated 110 new + 342 unchanged - 9 fixed = 452 total (was 351) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s{color} | {color:red} The patch 11 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 6s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s{color} | {color:blue} Skipped patched modules with no Java source: . {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 1s{color} | {color:red} tez-api generated 5 new + 0 unchanged - 0 fixed = 5 total (was 0) {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 5s{color} | {color:red} tez-dag generated 10 new + 0 unchanged - 0 fixed = 10 total (was 0) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m 7s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 1m 56s{color} | {color:red} tez-api in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 29s{color} | {color:green} tez-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 3m 41s{color} | {color:red} tez-dag in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 3m 53s{color} | {color:red} tez-tests in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 3m 31s{color} | {color:red} root in the patch failed. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 1m 41s{color} | {color:red} The patch generated 14 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black}119m 12s{color} | {color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:tez-api |
| | Found reliance on default encoding in org.apache.tez.client.registry.zookeeper.ZkAMRegistryClient.getAMRecord(ChildData):in org.apache.tez.client.registry.zookeeper.ZkAMRegistryClient.getAMRecord(ChildData): new String(byte[]) At ZkAMRegistryClient.java:[line 100] |
| | Unread field:ZkFrameworkClient.java:[line 46] |
| | Found reliance on default encoding in org.apache.tez.frameworkplugins.local.LocalAmRegistryClient.getRecord(String):in org.apache.tez.frameworkplugins.local.LocalAmRegistryClient.getRecord(String): new java.io.InputStreamReader(InputStream) At LocalAmRegistryClient.java:[line 39] |
| | Possible null pointer dereference in org.apache.tez.frameworkplugins.local.LocalAmRegistryClient.getAllRecords() due to return value of called method Dereferenced at LocalAmRegistryClient.java:org.apache.tez.frameworkplugins.local.LocalAmRegistryClient.getAllRecords() due to return value of called method Dereferenced at LocalAmRegistryClient.java:[line 51] |
| | org.apache.tez.frameworkplugins.local.LocalAmRegistryClient.getRecord(String) may fail to close stream At LocalAmRegistryClient.java:stream At LocalAmRegistryClient.java:[line 39] |
| FindBugs | module:tez-dag |
| | Found reliance on default encoding in org.apache.tez.dag.api.client.registry.LocalStandaloneAmRegistry.add(AMRecord):in org.apache.tez.dag.api.client.registry.LocalStandaloneAmRegistry.add(AMRecord): new java.io.PrintWriter(File) At LocalStandaloneAmRegistry.java:[line 27] |
| | Exceptional return value of java.io.File.delete() ignored in org.apache.tez.dag.api.client.registry.LocalStandaloneAmRegistry.add(AMRecord) At LocalStandaloneAmRegistry.java:ignored in org.apache.tez.dag.api.client.registry.LocalStandaloneAmRegistry.add(AMRecord) At LocalStandaloneAmRegistry.java:[line 25] |
| | Exceptional return value of java.io.File.mkdir() ignored in org.apache.tez.dag.api.client.registry.LocalStandaloneAmRegistry.add(AMRecord) At LocalStandaloneAmRegistry.java:ignored in org.apache.tez.dag.api.client.registry.LocalStandaloneAmRegistry.add(AMRecord) At LocalStandaloneAmRegistry.java:[line 21] |
| | Exceptional return value of java.io.File.delete() ignored in org.apache.tez.dag.api.client.registry.LocalStandaloneAmRegistry.remove(AMRecord) At LocalStandaloneAmRegistry.java:ignored in org.apache.tez.dag.api.client.registry.LocalStandaloneAmRegistry.remove(AMRecord) At LocalStandaloneAmRegistry.java:[line 39] |
| | Found reliance on default encoding in org.apache.tez.dag.api.client.registry.LocalUnmanagedAmRegistry.add(AMRecord):in org.apache.tez.dag.api.client.registry.LocalUnmanagedAmRegistry.add(AMRecord): new java.io.PrintWriter(File) At LocalUnmanagedAmRegistry.java:[line 29] |
| | Exceptional return value of java.io.File.delete() ignored in org.apache.tez.dag.api.client.registry.LocalUnmanagedAmRegistry.add(AMRecord) At LocalUnmanagedAmRegistry.java:ignored in org.apache.tez.dag.api.client.registry.LocalUnmanagedAmRegistry.add(AMRecord) At LocalUnmanagedAmRegistry.java:[line 27] |
| | Exceptional return value of java.io.File.mkdir() ignored in org.apache.tez.dag.api.client.registry.LocalUnmanagedAmRegistry.add(AMRecord) At LocalUnmanagedAmRegistry.java:ignored in org.apache.tez.dag.api.client.registry.LocalUnmanagedAmRegistry.add(AMRecord) At LocalUnmanagedAmRegistry.java:[line 23] |
| | Exceptional return value of java.io.File.delete() ignored in org.apache.tez.dag.api.client.registry.LocalUnmanagedAmRegistry.remove(AMRecord) At LocalUnmanagedAmRegistry.java:ignored in org.apache.tez.dag.api.client.registry.LocalUnmanagedAmRegistry.remove(AMRecord) At LocalUnmanagedAmRegistry.java:[line 41] |
| | Found reliance on default encoding in org.apache.tez.dag.api.client.registry.zookeeper.ZkAMRegistry.add(AMRecord):in org.apache.tez.dag.api.client.registry.zookeeper.ZkAMRegistry.add(AMRecord): String.getBytes() At ZkAMRegistry.java:[line 98] |
| | org.apache.tez.dag.app.DAGAppMaster.amExts isn't final but should be At DAGAppMaster.java:be At DAGAppMaster.java:[line 230] |
| Failed junit tests | tez.dag.api.TestTezConfiguration |
| | tez.dag.app.dag.impl.TestVertexImpl2 |
| | tez.mapreduce.TestMRRJobs |
| | tez.test.TestExceptionPropagation |
| | tez.test.TestTezJobs |
| | tez.test.TestRecovery |
| | tez.test.TestSecureShuffle |
| | tez.test.TestAMRecovery |
| | tez.test.TestDAGRecovery |
| | tez.test.TestDAGRecovery2 |
| | tez.mapreduce.TestMRRJobsDAGApi |
| | tez.test.TestFaultTolerance |
| | tez.mapreduce.TestMRRJobs |
| | tez.test.TestExceptionPropagation |
| | tez.test.TestTezJobs |
| | tez.test.TestRecovery |
| | tez.test.TestSecureShuffle |
| | tez.test.TestAMRecovery |
| | tez.test.TestDAGRecovery |
| | tez.test.TestDAGRecovery2 |
| | tez.mapreduce.TestMRRJobsDAGApi |
| | tez.test.TestFaultTolerance |
| | tez.dag.app.dag.impl.TestVertexImpl2 |
| | tez.dag.api.TestTezConfiguration |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/tez:d4a62de |
| JIRA Issue | TEZ-3991 |
| JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12964601/TEZ-3991.1.patch |
| Optional Tests | dupname asflicense javac javadoc unit xml compile findbugs checkstyle |
| uname | Linux 03ac3308993b 4.4.0-138-generic #164~14.04.1-Ubuntu SMP Fri Oct 5 08:56:16 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | master / 8395a95 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_191 |
| findbugs | v3.0.1 |
| javac | https://builds.apache.org/job/PreCommit-TEZ-Build/132/artifact/out/diff-compile-javac-tez-dag.txt |
| javac | https://builds.apache.org/job/PreCommit-TEZ-Build/132/artifact/out/diff-compile-javac-root.txt |
| checkstyle | https://builds.apache.org/job/PreCommit-TEZ-Build/132/artifact/out/diff-checkstyle-tez-api.txt |
| checkstyle | https://builds.apache.org/job/PreCommit-TEZ-Build/132/artifact/out/diff-checkstyle-tez-dag.txt |
| checkstyle | https://builds.apache.org/job/PreCommit-TEZ-Build/132/artifact/out/diff-checkstyle-tez-tests.txt |
| checkstyle | https://builds.apache.org/job/PreCommit-TEZ-Build/132/artifact/out/diff-checkstyle-root.txt |
| whitespace | https://builds.apache.org/job/PreCommit-TEZ-Build/132/artifact/out/whitespace-tabs.txt |
| findbugs | https://builds.apache.org/job/PreCommit-TEZ-Build/132/artifact/out/new-findbugs-tez-api.html |
| findbugs | https://builds.apache.org/job/PreCommit-TEZ-Build/132/artifact/out/new-findbugs-tez-dag.html |
| unit | https://builds.apache.org/job/PreCommit-TEZ-Build/132/artifact/out/patch-unit-tez-api.txt |
| unit | https://builds.apache.org/job/PreCommit-TEZ-Build/132/artifact/out/patch-unit-tez-dag.txt |
| unit | https://builds.apache.org/job/PreCommit-TEZ-Build/132/artifact/out/patch-unit-tez-tests.txt |
| unit | https://builds.apache.org/job/PreCommit-TEZ-Build/132/artifact/out/patch-unit-root.txt |
| Test Results | https://builds.apache.org/job/PreCommit-TEZ-Build/132/testReport/ |
| asflicense | https://builds.apache.org/job/PreCommit-TEZ-Build/132/artifact/out/patch-asflicense-problems.txt |
| Max. process+thread count | 626 (vs. ulimit of 10000) |
| modules | C: tez-api tez-common tez-dag tez-tests . U: . |
| Console output | https://builds.apache.org/job/PreCommit-TEZ-Build/132/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> Unmanaged tez sessions
> ----------------------
>
> Key: TEZ-3991
> URL: https://issues.apache.org/jira/browse/TEZ-3991
> Project: Apache Tez
> Issue Type: New Feature
> Affects Versions: 0.10.0
> Reporter: Prasanth Jayachandran
> Assignee: Eric Wohlstadter
> Priority: Major
> Labels: Kubernetes
> Attachments: TEZ-3991.1.patch
>
>
> Provide an option for launching tez AM in unmanaged mode. In unmanaged mode, tez AMs can register itself with Zookeeper which clients (like HiveServer2) can discover via zk registry client.
> HiveServer2 currently manages the lifecycle of tez AMs. The unmanaged mode will let AM come up on their own (can be via simple java launcher) and be discoverable for others.
> Example use case for this is, HiveServer2 can discover already running AMs and can attach to it for DAG submission and detach when done executing queries. AMs can similarly discover LLAP daemons via task scheduler plugin for submitting tasks.
> A mode to cut off interactions with RM will also useful since for LLAP no on-demand containers are required.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)