You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@tez.apache.org by "ayushtkn (via GitHub)" <gi...@apache.org> on 2023/06/21 07:56:36 UTC
[GitHub] [tez] ayushtkn opened a new pull request, #295: TEZ-4250: Optimise TaskImpl::getCounters.
ayushtkn opened a new pull request, #295:
URL: https://github.com/apache/tez/pull/295
(no comment)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@tez.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tez] tez-yetus commented on pull request #295: TEZ-4250: Optimise TaskImpl::getCounters.
Posted by "tez-yetus (via GitHub)" <gi...@apache.org>.
tez-yetus commented on PR #295:
URL: https://github.com/apache/tez/pull/295#issuecomment-1600545266
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 18s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| -1 :x: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 12m 8s | master passed |
| +1 :green_heart: | compile | 0m 48s | master passed with JDK Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu122.04.1 |
| +1 :green_heart: | compile | 0m 43s | master passed with JDK Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~22.04-b09 |
| +1 :green_heart: | checkstyle | 1m 3s | master passed |
| +1 :green_heart: | javadoc | 0m 25s | master passed with JDK Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu122.04.1 |
| +1 :green_heart: | javadoc | 0m 24s | master passed with JDK Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~22.04-b09 |
| +0 :ok: | spotbugs | 1m 33s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 1m 29s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 31s | the patch passed |
| +1 :green_heart: | compile | 0m 35s | the patch passed with JDK Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu122.04.1 |
| +1 :green_heart: | javac | 0m 35s | the patch passed |
| +1 :green_heart: | compile | 0m 29s | the patch passed with JDK Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~22.04-b09 |
| +1 :green_heart: | javac | 0m 29s | the patch passed |
| +1 :green_heart: | checkstyle | 0m 24s | tez-dag: The patch generated 0 new + 127 unchanged - 1 fixed = 127 total (was 128) |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | javadoc | 0m 11s | the patch passed with JDK Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu122.04.1 |
| +1 :green_heart: | javadoc | 0m 10s | the patch passed with JDK Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~22.04-b09 |
| +1 :green_heart: | findbugs | 1m 19s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 5m 18s | tez-dag in the patch passed. |
| +1 :green_heart: | asflicense | 0m 9s | The patch does not generate ASF License warnings. |
| | | 27m 38s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-295/4/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/tez/pull/295 |
| JIRA Issue | TEZ-4250 |
| Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs checkstyle compile |
| uname | Linux e82e71a30ab5 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3 19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | personality/tez.sh |
| git revision | master / d20f3347d |
| Default Java | Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~22.04-b09 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu122.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~22.04-b09 |
| Test Results | https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-295/4/testReport/ |
| Max. process+thread count | 254 (vs. ulimit of 5500) |
| modules | C: tez-dag U: tez-dag |
| Console output | https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-295/4/console |
| versions | git=2.34.1 maven=3.6.3 findbugs=3.0.1 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@tez.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tez] tez-yetus commented on pull request #295: TEZ-4250: Optimise TaskImpl::getCounters.
Posted by "tez-yetus (via GitHub)" <gi...@apache.org>.
tez-yetus commented on PR #295:
URL: https://github.com/apache/tez/pull/295#issuecomment-1600428728
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 18s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| -1 :x: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 5m 0s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 7m 43s | master passed |
| +1 :green_heart: | compile | 1m 33s | master passed with JDK Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu122.04.1 |
| +1 :green_heart: | compile | 1m 20s | master passed with JDK Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~22.04-b09 |
| +1 :green_heart: | checkstyle | 1m 9s | master passed |
| +1 :green_heart: | javadoc | 1m 9s | master passed with JDK Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu122.04.1 |
| +1 :green_heart: | javadoc | 1m 5s | master passed with JDK Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~22.04-b09 |
| +0 :ok: | spotbugs | 1m 25s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 2m 41s | master passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 10s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 0m 53s | the patch passed |
| +1 :green_heart: | compile | 0m 59s | the patch passed with JDK Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu122.04.1 |
| +1 :green_heart: | javac | 0m 59s | the patch passed |
| +1 :green_heart: | compile | 0m 52s | the patch passed with JDK Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~22.04-b09 |
| +1 :green_heart: | javac | 0m 52s | the patch passed |
| -0 :warning: | checkstyle | 0m 13s | tez-api: The patch generated 1 new + 7 unchanged - 0 fixed = 8 total (was 7) |
| -0 :warning: | checkstyle | 0m 25s | tez-dag: The patch generated 1 new + 127 unchanged - 1 fixed = 128 total (was 128) |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | javadoc | 0m 38s | the patch passed with JDK Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu122.04.1 |
| +1 :green_heart: | javadoc | 0m 39s | the patch passed with JDK Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~22.04-b09 |
| +1 :green_heart: | findbugs | 2m 28s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 2m 11s | tez-api in the patch passed. |
| +1 :green_heart: | unit | 5m 15s | tez-dag in the patch passed. |
| +1 :green_heart: | asflicense | 0m 19s | The patch does not generate ASF License warnings. |
| | | 38m 32s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-295/2/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/tez/pull/295 |
| JIRA Issue | TEZ-4250 |
| Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs checkstyle compile |
| uname | Linux dfa640d7716b 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3 19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | personality/tez.sh |
| git revision | master / d20f3347d |
| Default Java | Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~22.04-b09 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu122.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~22.04-b09 |
| checkstyle | https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-295/2/artifact/out/diff-checkstyle-tez-api.txt |
| checkstyle | https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-295/2/artifact/out/diff-checkstyle-tez-dag.txt |
| Test Results | https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-295/2/testReport/ |
| Max. process+thread count | 691 (vs. ulimit of 5500) |
| modules | C: tez-api tez-dag U: . |
| Console output | https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-295/2/console |
| versions | git=2.34.1 maven=3.6.3 findbugs=3.0.1 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@tez.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tez] ayushtkn commented on a diff in pull request #295: TEZ-4250: Optimise TaskImpl::getCounters.
Posted by "ayushtkn (via GitHub)" <gi...@apache.org>.
ayushtkn commented on code in PR #295:
URL: https://github.com/apache/tez/pull/295#discussion_r1239543151
##########
tez-dag/src/main/java/org/apache/tez/dag/app/dag/impl/TaskImpl.java:
##########
@@ -465,15 +465,19 @@ public TaskReport getReport() {
@Override
public TezCounters getCounters() {
- TezCounters counters = new TezCounters();
- counters.incrAllCounters(this.counters);
+ TezCounters tezCounters = null;
+ if (getVertex().isSpeculationEnabled()) {
+ tezCounters = new TezCounters();
+ tezCounters.incrAllCounters(this.counters);
+ }
readLock.lock();
try {
TaskAttempt bestAttempt = selectBestAttempt();
- if (bestAttempt != null) {
- counters.incrAllCounters(bestAttempt.getCounters());
+ if (bestAttempt != null && tezCounters != null) {
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@tez.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tez] abstractdog merged pull request #295: TEZ-4250: Optimise TaskImpl::getCounters.
Posted by "abstractdog (via GitHub)" <gi...@apache.org>.
abstractdog merged PR #295:
URL: https://github.com/apache/tez/pull/295
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@tez.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tez] tez-yetus commented on pull request #295: TEZ-4250: Optimise TaskImpl::getCounters.
Posted by "tez-yetus (via GitHub)" <gi...@apache.org>.
tez-yetus commented on PR #295:
URL: https://github.com/apache/tez/pull/295#issuecomment-1600428584
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 19s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| -1 :x: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 5m 44s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 9m 3s | master passed |
| +1 :green_heart: | compile | 1m 33s | master passed with JDK Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu122.04.1 |
| +1 :green_heart: | compile | 1m 23s | master passed with JDK Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~22.04-b09 |
| +1 :green_heart: | checkstyle | 1m 29s | master passed |
| +1 :green_heart: | javadoc | 1m 14s | master passed with JDK Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu122.04.1 |
| +1 :green_heart: | javadoc | 1m 6s | master passed with JDK Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~22.04-b09 |
| +0 :ok: | spotbugs | 1m 28s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 2m 45s | master passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 0m 54s | the patch passed |
| +1 :green_heart: | compile | 1m 1s | the patch passed with JDK Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu122.04.1 |
| +1 :green_heart: | javac | 1m 1s | the patch passed |
| +1 :green_heart: | compile | 0m 51s | the patch passed with JDK Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~22.04-b09 |
| +1 :green_heart: | javac | 0m 51s | the patch passed |
| -0 :warning: | checkstyle | 0m 13s | tez-api: The patch generated 1 new + 7 unchanged - 0 fixed = 8 total (was 7) |
| -0 :warning: | checkstyle | 0m 24s | tez-dag: The patch generated 1 new + 127 unchanged - 1 fixed = 128 total (was 128) |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | javadoc | 0m 38s | the patch passed with JDK Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu122.04.1 |
| +1 :green_heart: | javadoc | 0m 39s | the patch passed with JDK Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~22.04-b09 |
| +1 :green_heart: | findbugs | 2m 29s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 2m 14s | tez-api in the patch passed. |
| +1 :green_heart: | unit | 5m 14s | tez-dag in the patch passed. |
| +1 :green_heart: | asflicense | 0m 23s | The patch does not generate ASF License warnings. |
| | | 41m 17s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-295/1/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/tez/pull/295 |
| JIRA Issue | TEZ-4250 |
| Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs checkstyle compile |
| uname | Linux efbbd1ff4bc6 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3 19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | personality/tez.sh |
| git revision | master / d20f3347d |
| Default Java | Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~22.04-b09 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu122.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~22.04-b09 |
| checkstyle | https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-295/1/artifact/out/diff-checkstyle-tez-api.txt |
| checkstyle | https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-295/1/artifact/out/diff-checkstyle-tez-dag.txt |
| Test Results | https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-295/1/testReport/ |
| Max. process+thread count | 390 (vs. ulimit of 5500) |
| modules | C: tez-api tez-dag U: . |
| Console output | https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-295/1/console |
| versions | git=2.34.1 maven=3.6.3 findbugs=3.0.1 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@tez.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tez] abstractdog commented on pull request #295: TEZ-4250: Optimise TaskImpl::getCounters.
Posted by "abstractdog (via GitHub)" <gi...@apache.org>.
abstractdog commented on PR #295:
URL: https://github.com/apache/tez/pull/295#issuecomment-1603887973
thanks for the review @rbalamohan
LGTM also, only a minor nit, let me know if it makes sense
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@tez.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tez] rbalamohan commented on pull request #295: TEZ-4250: Optimise TaskImpl::getCounters.
Posted by "rbalamohan (via GitHub)" <gi...@apache.org>.
rbalamohan commented on PR #295:
URL: https://github.com/apache/tez/pull/295#issuecomment-1603735866
LGTM. +1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@tez.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tez] tez-yetus commented on pull request #295: TEZ-4250: Optimise TaskImpl::getCounters.
Posted by "tez-yetus (via GitHub)" <gi...@apache.org>.
tez-yetus commented on PR #295:
URL: https://github.com/apache/tez/pull/295#issuecomment-1600550158
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 19s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| -1 :x: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 12m 2s | master passed |
| +1 :green_heart: | compile | 0m 50s | master passed with JDK Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu122.04.1 |
| +1 :green_heart: | compile | 0m 42s | master passed with JDK Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~22.04-b09 |
| +1 :green_heart: | checkstyle | 0m 43s | master passed |
| +1 :green_heart: | javadoc | 0m 26s | master passed with JDK Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu122.04.1 |
| +1 :green_heart: | javadoc | 0m 23s | master passed with JDK Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~22.04-b09 |
| +0 :ok: | spotbugs | 1m 35s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 1m 33s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 29s | the patch passed |
| +1 :green_heart: | compile | 0m 34s | the patch passed with JDK Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu122.04.1 |
| +1 :green_heart: | javac | 0m 34s | the patch passed |
| +1 :green_heart: | compile | 0m 28s | the patch passed with JDK Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~22.04-b09 |
| +1 :green_heart: | javac | 0m 28s | the patch passed |
| +1 :green_heart: | checkstyle | 0m 24s | tez-dag: The patch generated 0 new + 127 unchanged - 1 fixed = 127 total (was 128) |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | javadoc | 0m 11s | the patch passed with JDK Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu122.04.1 |
| +1 :green_heart: | javadoc | 0m 10s | the patch passed with JDK Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~22.04-b09 |
| +1 :green_heart: | findbugs | 1m 18s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 5m 16s | tez-dag in the patch passed. |
| +1 :green_heart: | asflicense | 0m 11s | The patch does not generate ASF License warnings. |
| | | 27m 14s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-295/5/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/tez/pull/295 |
| JIRA Issue | TEZ-4250 |
| Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs checkstyle compile |
| uname | Linux 8a4f0e695c1d 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3 19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | personality/tez.sh |
| git revision | master / d20f3347d |
| Default Java | Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~22.04-b09 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu122.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~22.04-b09 |
| Test Results | https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-295/5/testReport/ |
| Max. process+thread count | 219 (vs. ulimit of 5500) |
| modules | C: tez-dag U: tez-dag |
| Console output | https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-295/5/console |
| versions | git=2.34.1 maven=3.6.3 findbugs=3.0.1 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@tez.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tez] abstractdog commented on a diff in pull request #295: TEZ-4250: Optimise TaskImpl::getCounters.
Posted by "abstractdog (via GitHub)" <gi...@apache.org>.
abstractdog commented on code in PR #295:
URL: https://github.com/apache/tez/pull/295#discussion_r1239500780
##########
tez-dag/src/main/java/org/apache/tez/dag/app/dag/impl/TaskImpl.java:
##########
@@ -465,15 +465,19 @@ public TaskReport getReport() {
@Override
public TezCounters getCounters() {
- TezCounters counters = new TezCounters();
- counters.incrAllCounters(this.counters);
+ TezCounters tezCounters = null;
+ if (getVertex().isSpeculationEnabled()) {
+ tezCounters = new TezCounters();
+ tezCounters.incrAllCounters(this.counters);
+ }
readLock.lock();
try {
TaskAttempt bestAttempt = selectBestAttempt();
- if (bestAttempt != null) {
- counters.incrAllCounters(bestAttempt.getCounters());
+ if (bestAttempt != null && tezCounters != null) {
Review Comment:
tezCounters != null only if speculation is enabled due to the recent patch, would it make sense to use that condition here for better readability? I guess it would clarify that we only use "this.counter" and this whole stuff in case of speculation
also, we're returning a non-null counter anyway, would this be clearer as:
```
TezCounters taskCounters = (bestAttempt != null) ? bestAttempt.getCounters() : TaskAttemptImpl.EMPTY_COUNTERS;
if (speculationEnabled){
tezCounters.incrAllCounters(taskCounters);
return tezCounters;
}
return taskCounters;
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@tez.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tez] abstractdog commented on a diff in pull request #295: TEZ-4250: Optimise TaskImpl::getCounters.
Posted by "abstractdog (via GitHub)" <gi...@apache.org>.
abstractdog commented on code in PR #295:
URL: https://github.com/apache/tez/pull/295#discussion_r1239500780
##########
tez-dag/src/main/java/org/apache/tez/dag/app/dag/impl/TaskImpl.java:
##########
@@ -465,15 +465,19 @@ public TaskReport getReport() {
@Override
public TezCounters getCounters() {
- TezCounters counters = new TezCounters();
- counters.incrAllCounters(this.counters);
+ TezCounters tezCounters = null;
+ if (getVertex().isSpeculationEnabled()) {
+ tezCounters = new TezCounters();
+ tezCounters.incrAllCounters(this.counters);
+ }
readLock.lock();
try {
TaskAttempt bestAttempt = selectBestAttempt();
- if (bestAttempt != null) {
- counters.incrAllCounters(bestAttempt.getCounters());
+ if (bestAttempt != null && tezCounters != null) {
Review Comment:
tezCounters != null only if speculation is enabled due to the recent patch, would it make sense to use that condition here for better readability? I guess it would clarify that we only use "this.counter" and this whole stuff in case of speculation
also, we're returning a non-null counter anyway, would this be clearer as:
```
TezCounters taskCounters = (bestAttempt != null) ? bestAttempt.getCounters() : TaskAttemptImpl.EMPTY_COUNTERS;
if (isSpeculationEnabled){
tezCounters.incrAllCounters(taskCounters);
return tezCounters;
}
return taskCounters;
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@tez.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tez] tez-yetus commented on pull request #295: TEZ-4250: Optimise TaskImpl::getCounters.
Posted by "tez-yetus (via GitHub)" <gi...@apache.org>.
tez-yetus commented on PR #295:
URL: https://github.com/apache/tez/pull/295#issuecomment-1603979054
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 18s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| -1 :x: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 12m 50s | master passed |
| +1 :green_heart: | compile | 0m 46s | master passed with JDK Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu122.04.1 |
| +1 :green_heart: | compile | 0m 43s | master passed with JDK Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~22.04-b09 |
| +1 :green_heart: | checkstyle | 0m 48s | master passed |
| +1 :green_heart: | javadoc | 0m 36s | master passed with JDK Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu122.04.1 |
| +1 :green_heart: | javadoc | 0m 27s | master passed with JDK Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~22.04-b09 |
| +0 :ok: | spotbugs | 1m 47s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 1m 45s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 27s | the patch passed |
| +1 :green_heart: | compile | 0m 30s | the patch passed with JDK Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu122.04.1 |
| +1 :green_heart: | javac | 0m 30s | the patch passed |
| +1 :green_heart: | compile | 0m 27s | the patch passed with JDK Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~22.04-b09 |
| +1 :green_heart: | javac | 0m 27s | the patch passed |
| +1 :green_heart: | checkstyle | 0m 22s | tez-dag: The patch generated 0 new + 127 unchanged - 1 fixed = 127 total (was 128) |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | javadoc | 0m 10s | the patch passed with JDK Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu122.04.1 |
| +1 :green_heart: | javadoc | 0m 10s | the patch passed with JDK Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~22.04-b09 |
| +1 :green_heart: | findbugs | 1m 10s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 5m 14s | tez-dag in the patch passed. |
| +1 :green_heart: | asflicense | 0m 16s | The patch does not generate ASF License warnings. |
| | | 28m 15s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-295/6/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/tez/pull/295 |
| JIRA Issue | TEZ-4250 |
| Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs checkstyle compile |
| uname | Linux aee43b4351c2 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3 19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | personality/tez.sh |
| git revision | master / d20f3347d |
| Default Java | Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~22.04-b09 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu122.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~22.04-b09 |
| Test Results | https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-295/6/testReport/ |
| Max. process+thread count | 229 (vs. ulimit of 5500) |
| modules | C: tez-dag U: tez-dag |
| Console output | https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-295/6/console |
| versions | git=2.34.1 maven=3.6.3 findbugs=3.0.1 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@tez.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tez] tez-yetus commented on pull request #295: TEZ-4250: Optimise TaskImpl::getCounters.
Posted by "tez-yetus (via GitHub)" <gi...@apache.org>.
tez-yetus commented on PR #295:
URL: https://github.com/apache/tez/pull/295#issuecomment-1600462538
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 25m 43s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| -1 :x: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 17m 44s | master passed |
| +1 :green_heart: | compile | 0m 43s | master passed with JDK Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu122.04.1 |
| +1 :green_heart: | compile | 0m 37s | master passed with JDK Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~22.04-b09 |
| +1 :green_heart: | checkstyle | 1m 19s | master passed |
| +1 :green_heart: | javadoc | 0m 33s | master passed with JDK Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu122.04.1 |
| +1 :green_heart: | javadoc | 0m 21s | master passed with JDK Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~22.04-b09 |
| +0 :ok: | spotbugs | 1m 46s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 1m 44s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 29s | the patch passed |
| +1 :green_heart: | compile | 0m 32s | the patch passed with JDK Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu122.04.1 |
| +1 :green_heart: | javac | 0m 32s | the patch passed |
| +1 :green_heart: | compile | 0m 27s | the patch passed with JDK Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~22.04-b09 |
| +1 :green_heart: | javac | 0m 27s | the patch passed |
| -0 :warning: | checkstyle | 0m 22s | tez-dag: The patch generated 1 new + 127 unchanged - 1 fixed = 128 total (was 128) |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | javadoc | 0m 10s | the patch passed with JDK Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu122.04.1 |
| +1 :green_heart: | javadoc | 0m 9s | the patch passed with JDK Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~22.04-b09 |
| +1 :green_heart: | findbugs | 1m 15s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 5m 5s | tez-dag in the patch passed. |
| +1 :green_heart: | asflicense | 0m 10s | The patch does not generate ASF License warnings. |
| | | 58m 30s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-295/3/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/tez/pull/295 |
| JIRA Issue | TEZ-4250 |
| Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs checkstyle compile |
| uname | Linux 5704737df9fc 4.15.0-212-generic #223-Ubuntu SMP Tue May 23 13:09:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | personality/tez.sh |
| git revision | master / d20f3347d |
| Default Java | Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~22.04-b09 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu122.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~22.04-b09 |
| checkstyle | https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-295/3/artifact/out/diff-checkstyle-tez-dag.txt |
| Test Results | https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-295/3/testReport/ |
| Max. process+thread count | 243 (vs. ulimit of 5500) |
| modules | C: tez-dag U: tez-dag |
| Console output | https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-295/3/console |
| versions | git=2.34.1 maven=3.6.3 findbugs=3.0.1 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@tez.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org