You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tez.apache.org by GitBox <gi...@apache.org> on 2021/01/28 16:58:45 UTC
[GitHub] [tez] belugabehr opened a new pull request #94: TEZ-4274: Log Credential Tokens
belugabehr opened a new pull request #94:
URL: https://github.com/apache/tez/pull/94
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tez] belugabehr commented on pull request #94: TEZ-4274: Log Credential Tokens
Posted by GitBox <gi...@apache.org>.
belugabehr commented on pull request #94:
URL: https://github.com/apache/tez/pull/94#issuecomment-769271540
What message would that be? Happy to update it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tez] abstractdog edited a comment on pull request #94: TEZ-4274: Log Credential Tokens
Posted by GitBox <gi...@apache.org>.
abstractdog edited a comment on pull request #94:
URL: https://github.com/apache/tez/pull/94#issuecomment-888056943
nice catch!
credentials are already logged in TezChild constructor by:
https://github.com/apache/tez/blob/master/tez-runtime-internals/src/main/java/org/apache/tez/runtime/task/TezChild.java#L203
```
TezCommonUtils.logCredentials(LOG, credentials, "tezChildInit");
```
wouldn't this be enough? not 100% in line with hadoop, but adds a useful small context message
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@tez.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tez] hadoop-yetus commented on pull request #94: TEZ-4274: Log Credential Tokens
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #94:
URL: https://github.com/apache/tez/pull/94#issuecomment-888968449
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 12m 45s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| -1 :x: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 12m 53s | master passed |
| +1 :green_heart: | compile | 0m 29s | master passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 0m 29s | master passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | checkstyle | 1m 0s | master passed |
| +1 :green_heart: | javadoc | 0m 40s | master passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 0m 27s | master passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +0 :ok: | spotbugs | 1m 11s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 1m 8s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 16s | the patch passed |
| +1 :green_heart: | compile | 0m 16s | the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javac | 0m 16s | the patch passed |
| +1 :green_heart: | compile | 0m 14s | the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | javac | 0m 14s | the patch passed |
| +1 :green_heart: | checkstyle | 0m 9s | tez-runtime-internals: The patch generated 0 new + 8 unchanged - 1 fixed = 8 total (was 9) |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | javadoc | 0m 14s | the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 0m 13s | the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | findbugs | 0m 37s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 0m 37s | tez-runtime-internals in the patch passed. |
| +1 :green_heart: | asflicense | 0m 14s | The patch does not generate ASF License warnings. |
| | | 33m 34s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-94/3/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/tez/pull/94 |
| JIRA Issue | TEZ-4274 |
| Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs checkstyle compile |
| uname | Linux 4802f1882e82 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | personality/tez.sh |
| git revision | master / 464d86d8f |
| Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Test Results | https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-94/3/testReport/ |
| Max. process+thread count | 102 (vs. ulimit of 5500) |
| modules | C: tez-runtime-internals U: tez-runtime-internals |
| Console output | https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-94/3/console |
| versions | git=2.25.1 maven=3.6.3 findbugs=3.0.1 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@tez.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tez] jteagles commented on pull request #94: TEZ-4274: Log Credential Tokens
Posted by GitBox <gi...@apache.org>.
jteagles commented on pull request #94:
URL: https://github.com/apache/tez/pull/94#issuecomment-769346972
I wasn't clear in my last reply. [MAPREDUCE-6971](https://issues.apache.org/jira/browse/MAPREDUCE-6971) simplified the way logging is done. In Tez we prefer to log credentials only in debug, but the concept is the same. TezChild was modeled after YarnChild, so it may make sense to adopt similar change. As show in the link below.
https://github.com/apache/hadoop/commit/453d48bdfbb67ed3e66c33c4aef239c3d7bdd3bc#diff-9191ed128249f1e8510e92cc7c1fa1160d1743f7ffebe82dd5dab0b2cb38ad79L103
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tez] tez-yetus commented on pull request #94: TEZ-4274: Log Credential Tokens
Posted by GitBox <gi...@apache.org>.
tez-yetus commented on pull request #94:
URL: https://github.com/apache/tez/pull/94#issuecomment-895501757
Thanks, @abstractdog. I totally missed this change. @belugabehr is the current credential logging enough for your needs?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@tez.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tez] abstractdog commented on pull request #94: TEZ-4274: Log Credential Tokens
Posted by GitBox <gi...@apache.org>.
abstractdog commented on pull request #94:
URL: https://github.com/apache/tez/pull/94#issuecomment-888056943
nice catch!
credentials are already logged in TezChild constructor by:
https://github.com/apache/tez/blob/master/tez-runtime-internals/src/main/java/org/apache/tez/runtime/task/TezChild.java#L203
```
TezCommonUtils.logCredentials(LOG, credentials, "tezChildInit");
```
wouldn't that be enough? if so, let's just remove this part
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@tez.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tez] hadoop-yetus commented on pull request #94: TEZ-4274: Log Credential Tokens
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #94:
URL: https://github.com/apache/tez/pull/94#issuecomment-887717026
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 19m 57s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| -1 :x: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 14m 49s | master passed |
| +1 :green_heart: | compile | 0m 30s | master passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 0m 29s | master passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | checkstyle | 1m 2s | master passed |
| +1 :green_heart: | javadoc | 0m 38s | master passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 0m 24s | master passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +0 :ok: | spotbugs | 1m 16s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 1m 14s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 19s | the patch passed |
| +1 :green_heart: | compile | 0m 20s | the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javac | 0m 20s | the patch passed |
| +1 :green_heart: | compile | 0m 19s | the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | javac | 0m 19s | the patch passed |
| +1 :green_heart: | checkstyle | 0m 11s | tez-runtime-internals: The patch generated 0 new + 8 unchanged - 1 fixed = 8 total (was 9) |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | javadoc | 0m 18s | the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 0m 16s | the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | findbugs | 0m 53s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 0m 45s | tez-runtime-internals in the patch passed. |
| +1 :green_heart: | asflicense | 0m 15s | The patch does not generate ASF License warnings. |
| | | 43m 33s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-94/2/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/tez/pull/94 |
| JIRA Issue | TEZ-4274 |
| Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs checkstyle compile |
| uname | Linux b093570f9306 4.15.0-128-generic #131-Ubuntu SMP Wed Dec 9 06:57:35 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | personality/tez.sh |
| git revision | master / 464d86d8f |
| Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Test Results | https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-94/2/testReport/ |
| Max. process+thread count | 90 (vs. ulimit of 5500) |
| modules | C: tez-runtime-internals U: tez-runtime-internals |
| Console output | https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-94/2/console |
| versions | git=2.25.1 maven=3.6.3 findbugs=3.0.1 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@tez.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tez] jteagles commented on pull request #94: TEZ-4274: Log Credential Tokens
Posted by GitBox <gi...@apache.org>.
jteagles commented on pull request #94:
URL: https://github.com/apache/tez/pull/94#issuecomment-769231485
Was accidentally dropped as part of TEZ-2176. MapReduce logs credentials with this message.
""Executing with tokens: {}", credentials.getAllTokens()"
I wonder if we want to Update the message to better reflect purpose.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tez] jteagles commented on pull request #94: TEZ-4274: Log Credential Tokens
Posted by GitBox <gi...@apache.org>.
jteagles commented on pull request #94:
URL: https://github.com/apache/tez/pull/94#issuecomment-895529119
Sorry, the above message was from me.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@tez.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tez] belugabehr commented on pull request #94: TEZ-4274: Log Credential Tokens
Posted by GitBox <gi...@apache.org>.
belugabehr commented on pull request #94:
URL: https://github.com/apache/tez/pull/94#issuecomment-887639903
@jteagles Updated as requested. Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@tez.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org