You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "bhattmanish98 (via GitHub)" <gi...@apache.org> on 2024/09/23 05:43:53 UTC
[PR] Hadoop-19280. Initialize ABFS client timer only if metric collection is enabled [hadoop]
bhattmanish98 opened a new pull request, #7061:
URL: https://github.com/apache/hadoop/pull/7061
**Description of PR**
**JIRA**: https://issues.apache.org/jira/browse/HADOOP-19280
**Current Flow**: In the current flow, we are initializing the timer of the abfs-timer-client outside the metric collection enable check. As a result, for each file system, when the AbfsClient object is initialized, it spawns a thread to evaluate the time of the ABFS client. Since we are purging/closing the timer inside the metric collection check, these threads are not being closed, causing them to persist in a long-lived state.
**Changes Made**: This PR contains the changes related to moving the initialization of the timer inside the metric collection check. Also includes check on metric collection if we are accessing the timer variable to avoid the null pointer exception.
Created two test suites to check the behaviour of timer variable and thread spawn in case of metric collection enabled and disabled.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] Hadoop-19280. Initialize ABFS client timer only if metric collection is enabled [hadoop]
Posted by "saikatroy038 (via GitHub)" <gi...@apache.org>.
saikatroy038 commented on code in PR #7061:
URL: https://github.com/apache/hadoop/pull/7061#discussion_r1776546954
##########
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/services/TestAbfsClient.java:
##########
@@ -0,0 +1,118 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.fs.azurebfs.services;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.azurebfs.AbfsConfiguration;
+import org.apache.hadoop.fs.azurebfs.AbfsCountersImpl;
+import org.apache.hadoop.fs.azurebfs.oauth2.AccessTokenProvider;
+import org.apache.hadoop.fs.azurebfs.utils.Base64;
+import org.apache.hadoop.fs.azurebfs.utils.MetricFormat;
+import org.assertj.core.api.Assertions;
+import org.junit.Test;
+import org.mockito.Mockito;
+
+import java.net.URI;
+import java.net.URL;
+import java.util.Map;
+
+import static org.apache.hadoop.fs.azurebfs.constants.ConfigurationKeys.FS_AZURE_METRIC_ACCOUNT_KEY;
+import static org.apache.hadoop.fs.azurebfs.constants.ConfigurationKeys.FS_AZURE_METRIC_ACCOUNT_NAME;
+import static org.apache.hadoop.fs.azurebfs.constants.ConfigurationKeys.FS_AZURE_METRIC_FORMAT;
+import static org.apache.hadoop.fs.azurebfs.services.AbfsClient.ABFS_CLIENT_TIMER_THREAD_NAME;
+
+public class TestAbfsClient {
+ private static final String ACCOUNT_NAME = "bogusAccountName.dfs.core.windows.net";
+ private static final String ACCOUNT_KEY = "testKey";
+
+ @Test
+ public void testTimerNotInitialize() throws Exception {
+ final Configuration configuration = new Configuration();
+ AbfsConfiguration abfsConfiguration = new AbfsConfiguration(configuration, ACCOUNT_NAME);
+
+ AbfsCounters abfsCounters = Mockito.spy(new AbfsCountersImpl(new URI("abcd")));
+ AbfsClientContext abfsClientContext = new AbfsClientContextBuilder().withAbfsCounters(abfsCounters).build();
+
+ // Get an instance of AbfsClient.
+ AbfsClient client = new AbfsDfsClient(new URL("https://azure.com"),
+ null,
+ abfsConfiguration,
+ (AccessTokenProvider) null,
+ null,
+ abfsClientContext);
+
+ Assertions.assertThat(client.getTimer())
+ .describedAs("Timer should not be initialized")
+ .isNull();
+
+ // Check if a thread with the name "abfs-timer-client" exists
+ Assertions.assertThat(isThreadRunning(ABFS_CLIENT_TIMER_THREAD_NAME))
+ .describedAs("Expected thread 'abfs-timer-client' not found")
+ .isEqualTo(false);
+ client.close();
+ }
+
+ @Test
+ public void testTimerInitialize() throws Exception {
+ final Configuration configuration = new Configuration();
+ configuration.set(FS_AZURE_METRIC_FORMAT, String.valueOf(MetricFormat.INTERNAL_BACKOFF_METRIC_FORMAT));
+ configuration.set(FS_AZURE_METRIC_ACCOUNT_NAME, ACCOUNT_NAME);
+ configuration.set(FS_AZURE_METRIC_ACCOUNT_KEY, Base64.encode(ACCOUNT_KEY.getBytes()));
+ AbfsConfiguration abfsConfiguration = new AbfsConfiguration(configuration, ACCOUNT_NAME);
+
+ AbfsCounters abfsCounters = Mockito.spy(new AbfsCountersImpl(new URI("abcd")));
+ AbfsClientContext abfsClientContext = new AbfsClientContextBuilder().withAbfsCounters(abfsCounters).build();
+
+ // Get an instance of AbfsClient.
+ AbfsClient client = new AbfsDfsClient(new URL("https://azure.com"),
+ null,
+ abfsConfiguration,
+ (AccessTokenProvider) null,
+ null,
+ abfsClientContext);
+
+ Assertions.assertThat(client.getTimer())
+ .describedAs("Timer should be initialized")
+ .isNotNull();
+
+ // Check if a thread with the name "abfs-timer-client" exists
+ Assertions.assertThat(isThreadRunning(ABFS_CLIENT_TIMER_THREAD_NAME))
+ .describedAs("Expected thread 'abfs-timer-client' not found")
+ .isEqualTo(true);
+ client.close();
+
+ // Check if the thread is removed after closing the client
+ Assertions.assertThat(isThreadRunning("abfs-timer-client"))
Review Comment:
There is no guarantee that the thread will be removed immediately after calling client.close(). This can make the test fail intermittently.
Maybe add a small delay (Thread.sleep(500)) after client.close() to handle that?
##########
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/services/TestAbfsClient.java:
##########
@@ -0,0 +1,118 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.fs.azurebfs.services;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.azurebfs.AbfsConfiguration;
+import org.apache.hadoop.fs.azurebfs.AbfsCountersImpl;
+import org.apache.hadoop.fs.azurebfs.oauth2.AccessTokenProvider;
+import org.apache.hadoop.fs.azurebfs.utils.Base64;
+import org.apache.hadoop.fs.azurebfs.utils.MetricFormat;
+import org.assertj.core.api.Assertions;
+import org.junit.Test;
+import org.mockito.Mockito;
+
+import java.net.URI;
+import java.net.URL;
+import java.util.Map;
+
+import static org.apache.hadoop.fs.azurebfs.constants.ConfigurationKeys.FS_AZURE_METRIC_ACCOUNT_KEY;
+import static org.apache.hadoop.fs.azurebfs.constants.ConfigurationKeys.FS_AZURE_METRIC_ACCOUNT_NAME;
+import static org.apache.hadoop.fs.azurebfs.constants.ConfigurationKeys.FS_AZURE_METRIC_FORMAT;
+import static org.apache.hadoop.fs.azurebfs.services.AbfsClient.ABFS_CLIENT_TIMER_THREAD_NAME;
+
+public class TestAbfsClient {
+ private static final String ACCOUNT_NAME = "bogusAccountName.dfs.core.windows.net";
+ private static final String ACCOUNT_KEY = "testKey";
+
+ @Test
+ public void testTimerNotInitialize() throws Exception {
+ final Configuration configuration = new Configuration();
+ AbfsConfiguration abfsConfiguration = new AbfsConfiguration(configuration, ACCOUNT_NAME);
+
+ AbfsCounters abfsCounters = Mockito.spy(new AbfsCountersImpl(new URI("abcd")));
+ AbfsClientContext abfsClientContext = new AbfsClientContextBuilder().withAbfsCounters(abfsCounters).build();
+
+ // Get an instance of AbfsClient.
+ AbfsClient client = new AbfsDfsClient(new URL("https://azure.com"),
+ null,
+ abfsConfiguration,
+ (AccessTokenProvider) null,
+ null,
+ abfsClientContext);
+
+ Assertions.assertThat(client.getTimer())
+ .describedAs("Timer should not be initialized")
+ .isNull();
+
+ // Check if a thread with the name "abfs-timer-client" exists
+ Assertions.assertThat(isThreadRunning(ABFS_CLIENT_TIMER_THREAD_NAME))
+ .describedAs("Expected thread 'abfs-timer-client' not found")
+ .isEqualTo(false);
+ client.close();
+ }
+
+ @Test
+ public void testTimerInitialize() throws Exception {
+ final Configuration configuration = new Configuration();
+ configuration.set(FS_AZURE_METRIC_FORMAT, String.valueOf(MetricFormat.INTERNAL_BACKOFF_METRIC_FORMAT));
+ configuration.set(FS_AZURE_METRIC_ACCOUNT_NAME, ACCOUNT_NAME);
+ configuration.set(FS_AZURE_METRIC_ACCOUNT_KEY, Base64.encode(ACCOUNT_KEY.getBytes()));
+ AbfsConfiguration abfsConfiguration = new AbfsConfiguration(configuration, ACCOUNT_NAME);
+
+ AbfsCounters abfsCounters = Mockito.spy(new AbfsCountersImpl(new URI("abcd")));
+ AbfsClientContext abfsClientContext = new AbfsClientContextBuilder().withAbfsCounters(abfsCounters).build();
+
+ // Get an instance of AbfsClient.
+ AbfsClient client = new AbfsDfsClient(new URL("https://azure.com"),
+ null,
+ abfsConfiguration,
+ (AccessTokenProvider) null,
+ null,
+ abfsClientContext);
+
+ Assertions.assertThat(client.getTimer())
+ .describedAs("Timer should be initialized")
+ .isNotNull();
+
+ // Check if a thread with the name "abfs-timer-client" exists
+ Assertions.assertThat(isThreadRunning(ABFS_CLIENT_TIMER_THREAD_NAME))
+ .describedAs("Expected thread 'abfs-timer-client' not found")
+ .isEqualTo(true);
+ client.close();
+
+ // Check if the thread is removed after closing the client
+ Assertions.assertThat(isThreadRunning("abfs-timer-client"))
Review Comment:
Assertions.assertThat(isThreadRunning(ABFS_CLIENT_TIMER_THREAD_NAME))
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] Hadoop-19280. Initialize ABFS client timer only if metric collection is enabled [hadoop]
Posted by "hadoop-yetus (via GitHub)" <gi...@apache.org>.
hadoop-yetus commented on PR #7061:
URL: https://github.com/apache/hadoop/pull/7061#issuecomment-2371671052
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 51s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 50m 0s | | trunk passed |
| +1 :green_heart: | compile | 0m 41s | | trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 |
| +1 :green_heart: | compile | 0m 37s | | trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05 |
| +1 :green_heart: | checkstyle | 0m 31s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 41s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 41s | | trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 |
| +1 :green_heart: | javadoc | 0m 33s | | trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05 |
| +1 :green_heart: | spotbugs | 1m 8s | | trunk passed |
| +1 :green_heart: | shadedclient | 39m 54s | | branch has no errors when building and testing our client artifacts. |
| -0 :warning: | patch | 40m 16s | | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 29s | | the patch passed |
| +1 :green_heart: | compile | 0m 32s | | the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 |
| +1 :green_heart: | javac | 0m 32s | | the patch passed |
| +1 :green_heart: | compile | 0m 28s | | the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05 |
| +1 :green_heart: | javac | 0m 28s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | checkstyle | 0m 20s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 31s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 29s | | the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 |
| +1 :green_heart: | javadoc | 0m 25s | | the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05 |
| +1 :green_heart: | spotbugs | 1m 9s | | the patch passed |
| +1 :green_heart: | shadedclient | 39m 40s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 2m 33s | | hadoop-azure in the patch passed. |
| +1 :green_heart: | asflicense | 0m 37s | | The patch does not generate ASF License warnings. |
| | | 144m 2s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7061/6/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/7061 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux f2ff62965326 5.15.0-118-generic #128-Ubuntu SMP Fri Jul 5 09:28:59 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 99e1cc9fea8bf936e518667282e199771dcc3200 |
| Default Java | Private Build-1.8.0_422-8u422-b05-1~20.04-b05 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_422-8u422-b05-1~20.04-b05 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7061/6/testReport/ |
| Max. process+thread count | 577 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7061/6/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] Hadoop-19280. Initialize ABFS client timer only if metric collection is enabled [hadoop]
Posted by "bhattmanish98 (via GitHub)" <gi...@apache.org>.
bhattmanish98 commented on code in PR #7061:
URL: https://github.com/apache/hadoop/pull/7061#discussion_r1776415026
##########
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClient.java:
##########
@@ -1418,7 +1419,7 @@ private TracingContext getMetricTracingContext() {
boolean timerOrchestrator(TimerFunctionality timerFunctionality, TimerTask timerTask) {
switch (timerFunctionality) {
case RESUME:
- if (isMetricCollectionStopped.get()) {
+ if (isMetricCollectionEnabled) {
Review Comment:
The check is already included within the synchronization block, which is why I didn't add it here again.
synchronized (this) {
if (isMetricCollectionStopped.get()) {
resumeTimer();
}
}
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] Hadoop-19280. Initialize ABFS client timer only if metric collection is enabled [hadoop]
Posted by "bhattmanish98 (via GitHub)" <gi...@apache.org>.
bhattmanish98 commented on code in PR #7061:
URL: https://github.com/apache/hadoop/pull/7061#discussion_r1776465494
##########
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClient.java:
##########
@@ -1418,7 +1419,7 @@ private TracingContext getMetricTracingContext() {
boolean timerOrchestrator(TimerFunctionality timerFunctionality, TimerTask timerTask) {
switch (timerFunctionality) {
case RESUME:
- if (isMetricCollectionStopped.get()) {
+ if (isMetricCollectionEnabled) {
Review Comment:
@anmolanmol1234 @saikatroy038 Added the check on isMetricCollectionStopped.get(). Please check
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] Hadoop-19280. Initialize ABFS client timer only if metric collection is enabled [hadoop]
Posted by "bhattmanish98 (via GitHub)" <gi...@apache.org>.
bhattmanish98 commented on PR #7061:
URL: https://github.com/apache/hadoop/pull/7061#issuecomment-2367400821
@anujmodi2021 @anmolanmol1234 Please review the PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] Hadoop-19280. Initialize ABFS client timer only if metric collection is enabled [hadoop]
Posted by "bhattmanish98 (via GitHub)" <gi...@apache.org>.
bhattmanish98 commented on PR #7061:
URL: https://github.com/apache/hadoop/pull/7061#issuecomment-2375869827
@mukund-thakur @steveloughran @saikatroy038 Could you please review the PR?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] Hadoop-19280. Initialize ABFS client timer only if metric collection is enabled [hadoop]
Posted by "hadoop-yetus (via GitHub)" <gi...@apache.org>.
hadoop-yetus commented on PR #7061:
URL: https://github.com/apache/hadoop/pull/7061#issuecomment-2376342595
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 51s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 50m 5s | | trunk passed |
| +1 :green_heart: | compile | 0m 41s | | trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 |
| +1 :green_heart: | compile | 0m 36s | | trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05 |
| +1 :green_heart: | checkstyle | 0m 32s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 42s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 41s | | trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 |
| +1 :green_heart: | javadoc | 0m 34s | | trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05 |
| +1 :green_heart: | spotbugs | 1m 8s | | trunk passed |
| +1 :green_heart: | shadedclient | 39m 44s | | branch has no errors when building and testing our client artifacts. |
| -0 :warning: | patch | 40m 6s | | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 30s | | the patch passed |
| +1 :green_heart: | compile | 0m 32s | | the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 |
| +1 :green_heart: | javac | 0m 32s | | the patch passed |
| +1 :green_heart: | compile | 0m 29s | | the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05 |
| +1 :green_heart: | javac | 0m 29s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | checkstyle | 0m 19s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 31s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 29s | | the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 |
| +1 :green_heart: | javadoc | 0m 25s | | the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05 |
| +1 :green_heart: | spotbugs | 1m 7s | | the patch passed |
| +1 :green_heart: | shadedclient | 39m 48s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 2m 34s | | hadoop-azure in the patch passed. |
| +1 :green_heart: | asflicense | 0m 39s | | The patch does not generate ASF License warnings. |
| | | 144m 10s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7061/8/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/7061 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux f53280e42010 5.15.0-118-generic #128-Ubuntu SMP Fri Jul 5 09:28:59 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 69365575e86834ee382d7e927b8cc9566a1021e9 |
| Default Java | Private Build-1.8.0_422-8u422-b05-1~20.04-b05 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_422-8u422-b05-1~20.04-b05 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7061/8/testReport/ |
| Max. process+thread count | 586 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7061/8/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HADOOP-19280. [ABFS] Initialize client timer only if metric collection is enabled [hadoop]
Posted by "bhattmanish98 (via GitHub)" <gi...@apache.org>.
bhattmanish98 commented on PR #7061:
URL: https://github.com/apache/hadoop/pull/7061#issuecomment-2603860423
> Manish, thank you!
>
> Please can you do a backport PR for branch-3.4; this is simply for testing rather than any more code reviews.
@steveloughran Applogies for the delay. I have raised a PR (https://github.com/apache/hadoop/pull/7307) to backport these changes to branch-3.4.
Thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] Hadoop-19280. Initialize ABFS client timer only if metric collection is enabled [hadoop]
Posted by "hadoop-yetus (via GitHub)" <gi...@apache.org>.
hadoop-yetus commented on PR #7061:
URL: https://github.com/apache/hadoop/pull/7061#issuecomment-2367851941
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 24s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 32m 30s | | trunk passed |
| +1 :green_heart: | compile | 0m 26s | | trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 |
| +1 :green_heart: | compile | 0m 21s | | trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05 |
| +1 :green_heart: | checkstyle | 0m 22s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 26s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 27s | | trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 |
| +1 :green_heart: | javadoc | 0m 24s | | trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05 |
| +1 :green_heart: | spotbugs | 0m 46s | | trunk passed |
| +1 :green_heart: | shadedclient | 20m 10s | | branch has no errors when building and testing our client artifacts. |
| -0 :warning: | patch | 20m 23s | | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 19s | | the patch passed |
| +1 :green_heart: | compile | 0m 18s | | the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 |
| +1 :green_heart: | javac | 0m 18s | | the patch passed |
| +1 :green_heart: | compile | 0m 18s | | the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05 |
| +1 :green_heart: | javac | 0m 18s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| -0 :warning: | checkstyle | 0m 14s | [/results-checkstyle-hadoop-tools_hadoop-azure.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7061/3/artifact/out/results-checkstyle-hadoop-tools_hadoop-azure.txt) | hadoop-tools/hadoop-azure: The patch generated 2 new + 2 unchanged - 0 fixed = 4 total (was 2) |
| +1 :green_heart: | mvnsite | 0m 19s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 18s | | the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 |
| +1 :green_heart: | javadoc | 0m 17s | | the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05 |
| +1 :green_heart: | spotbugs | 0m 42s | | the patch passed |
| +1 :green_heart: | shadedclient | 20m 11s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 1m 58s | | hadoop-azure in the patch passed. |
| +1 :green_heart: | asflicense | 0m 24s | | The patch does not generate ASF License warnings. |
| | | 82m 41s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7061/3/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/7061 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux e6f4362d8941 5.15.0-117-generic #127-Ubuntu SMP Fri Jul 5 20:13:28 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 7a16f6f7ca0a03c0e2840c0cfd91e7a0e08e743d |
| Default Java | Private Build-1.8.0_422-8u422-b05-1~20.04-b05 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_422-8u422-b05-1~20.04-b05 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7061/3/testReport/ |
| Max. process+thread count | 717 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7061/3/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] Hadoop-19280. Initialize ABFS client timer only if metric collection is enabled [hadoop]
Posted by "anmolanmol1234 (via GitHub)" <gi...@apache.org>.
anmolanmol1234 commented on code in PR #7061:
URL: https://github.com/apache/hadoop/pull/7061#discussion_r1776408271
##########
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClient.java:
##########
@@ -1418,7 +1419,7 @@ private TracingContext getMetricTracingContext() {
boolean timerOrchestrator(TimerFunctionality timerFunctionality, TimerTask timerTask) {
switch (timerFunctionality) {
case RESUME:
- if (isMetricCollectionStopped.get()) {
+ if (isMetricCollectionEnabled) {
Review Comment:
The if condition for stopped should also be added here
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] Hadoop-19280. Initialize ABFS client timer only if metric collection is enabled [hadoop]
Posted by "bhattmanish98 (via GitHub)" <gi...@apache.org>.
bhattmanish98 commented on code in PR #7061:
URL: https://github.com/apache/hadoop/pull/7061#discussion_r1776616147
##########
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/services/TestAbfsClient.java:
##########
@@ -0,0 +1,118 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.fs.azurebfs.services;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.azurebfs.AbfsConfiguration;
+import org.apache.hadoop.fs.azurebfs.AbfsCountersImpl;
+import org.apache.hadoop.fs.azurebfs.oauth2.AccessTokenProvider;
+import org.apache.hadoop.fs.azurebfs.utils.Base64;
+import org.apache.hadoop.fs.azurebfs.utils.MetricFormat;
+import org.assertj.core.api.Assertions;
+import org.junit.Test;
+import org.mockito.Mockito;
+
+import java.net.URI;
+import java.net.URL;
+import java.util.Map;
+
+import static org.apache.hadoop.fs.azurebfs.constants.ConfigurationKeys.FS_AZURE_METRIC_ACCOUNT_KEY;
+import static org.apache.hadoop.fs.azurebfs.constants.ConfigurationKeys.FS_AZURE_METRIC_ACCOUNT_NAME;
+import static org.apache.hadoop.fs.azurebfs.constants.ConfigurationKeys.FS_AZURE_METRIC_FORMAT;
+import static org.apache.hadoop.fs.azurebfs.services.AbfsClient.ABFS_CLIENT_TIMER_THREAD_NAME;
+
+public class TestAbfsClient {
+ private static final String ACCOUNT_NAME = "bogusAccountName.dfs.core.windows.net";
+ private static final String ACCOUNT_KEY = "testKey";
+
+ @Test
+ public void testTimerNotInitialize() throws Exception {
+ final Configuration configuration = new Configuration();
+ AbfsConfiguration abfsConfiguration = new AbfsConfiguration(configuration, ACCOUNT_NAME);
+
+ AbfsCounters abfsCounters = Mockito.spy(new AbfsCountersImpl(new URI("abcd")));
+ AbfsClientContext abfsClientContext = new AbfsClientContextBuilder().withAbfsCounters(abfsCounters).build();
+
+ // Get an instance of AbfsClient.
+ AbfsClient client = new AbfsDfsClient(new URL("https://azure.com"),
+ null,
+ abfsConfiguration,
+ (AccessTokenProvider) null,
+ null,
+ abfsClientContext);
+
+ Assertions.assertThat(client.getTimer())
+ .describedAs("Timer should not be initialized")
+ .isNull();
+
+ // Check if a thread with the name "abfs-timer-client" exists
+ Assertions.assertThat(isThreadRunning(ABFS_CLIENT_TIMER_THREAD_NAME))
+ .describedAs("Expected thread 'abfs-timer-client' not found")
+ .isEqualTo(false);
+ client.close();
+ }
+
+ @Test
+ public void testTimerInitialize() throws Exception {
+ final Configuration configuration = new Configuration();
+ configuration.set(FS_AZURE_METRIC_FORMAT, String.valueOf(MetricFormat.INTERNAL_BACKOFF_METRIC_FORMAT));
+ configuration.set(FS_AZURE_METRIC_ACCOUNT_NAME, ACCOUNT_NAME);
+ configuration.set(FS_AZURE_METRIC_ACCOUNT_KEY, Base64.encode(ACCOUNT_KEY.getBytes()));
+ AbfsConfiguration abfsConfiguration = new AbfsConfiguration(configuration, ACCOUNT_NAME);
+
+ AbfsCounters abfsCounters = Mockito.spy(new AbfsCountersImpl(new URI("abcd")));
+ AbfsClientContext abfsClientContext = new AbfsClientContextBuilder().withAbfsCounters(abfsCounters).build();
+
+ // Get an instance of AbfsClient.
+ AbfsClient client = new AbfsDfsClient(new URL("https://azure.com"),
+ null,
+ abfsConfiguration,
+ (AccessTokenProvider) null,
+ null,
+ abfsClientContext);
+
+ Assertions.assertThat(client.getTimer())
+ .describedAs("Timer should be initialized")
+ .isNotNull();
+
+ // Check if a thread with the name "abfs-timer-client" exists
+ Assertions.assertThat(isThreadRunning(ABFS_CLIENT_TIMER_THREAD_NAME))
+ .describedAs("Expected thread 'abfs-timer-client' not found")
+ .isEqualTo(true);
+ client.close();
+
+ // Check if the thread is removed after closing the client
+ Assertions.assertThat(isThreadRunning("abfs-timer-client"))
Review Comment:
Make sense, sure will add the delay of 500 ms
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] Hadoop-19280. Initialize ABFS client timer only if metric collection is enabled [hadoop]
Posted by "anmolanmol1234 (via GitHub)" <gi...@apache.org>.
anmolanmol1234 commented on code in PR #7061:
URL: https://github.com/apache/hadoop/pull/7061#discussion_r1771172078
##########
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/services/ITestAbfsClient.java:
##########
@@ -685,4 +692,69 @@ public void testExpectHundredContinue() throws Exception {
.describedAs("The expect header is not false")
.isFalse();
}
+
+ @Test
+ public void testTimerNotInitialize() throws Exception {
+ // Create an AzureBlobFileSystem instance.
+ final Configuration configuration = getRawConfiguration();
+ AbfsConfiguration abfsConfiguration = new AbfsConfiguration(configuration, ACCOUNT_NAME);
+
+ AbfsCounters abfsCounters = Mockito.spy(new AbfsCountersImpl(new URI("abcd")));
+ AbfsClientContext abfsClientContext = new AbfsClientContextBuilder().withAbfsCounters(abfsCounters).build();
+
+ // Get an instance of AbfsClient.
+ AbfsClient client = new AbfsDfsClient(new URL("https://azure.com"),
Review Comment:
abstract class instantiated, there are methods such as getMockAbfsClient and others in ITestAbfsClient class that can help to get an object
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] Hadoop-19280. Initialize ABFS client timer only if metric collection is enabled [hadoop]
Posted by "hadoop-yetus (via GitHub)" <gi...@apache.org>.
hadoop-yetus commented on PR #7061:
URL: https://github.com/apache/hadoop/pull/7061#issuecomment-2370877047
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 19s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 32m 42s | | trunk passed |
| +1 :green_heart: | compile | 0m 24s | | trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 |
| +1 :green_heart: | compile | 0m 23s | | trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05 |
| +1 :green_heart: | checkstyle | 0m 22s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 27s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 27s | | trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 |
| +1 :green_heart: | javadoc | 0m 19s | | trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05 |
| +1 :green_heart: | spotbugs | 0m 46s | | trunk passed |
| +1 :green_heart: | shadedclient | 20m 8s | | branch has no errors when building and testing our client artifacts. |
| -0 :warning: | patch | 20m 20s | | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 17s | | the patch passed |
| +1 :green_heart: | compile | 0m 18s | | the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 |
| +1 :green_heart: | javac | 0m 18s | | the patch passed |
| +1 :green_heart: | compile | 0m 18s | | the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05 |
| +1 :green_heart: | javac | 0m 18s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| -0 :warning: | checkstyle | 0m 11s | [/results-checkstyle-hadoop-tools_hadoop-azure.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7061/5/artifact/out/results-checkstyle-hadoop-tools_hadoop-azure.txt) | hadoop-tools/hadoop-azure: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) |
| +1 :green_heart: | mvnsite | 0m 20s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 19s | | the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 |
| +1 :green_heart: | javadoc | 0m 19s | | the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05 |
| +1 :green_heart: | spotbugs | 0m 45s | | the patch passed |
| +1 :green_heart: | shadedclient | 20m 10s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 2m 0s | | hadoop-azure in the patch passed. |
| -1 :x: | asflicense | 0m 25s | [/results-asflicense.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7061/5/artifact/out/results-asflicense.txt) | The patch generated 1 ASF License warnings. |
| | | 82m 18s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7061/5/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/7061 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 019ceead7881 5.15.0-117-generic #127-Ubuntu SMP Fri Jul 5 20:13:28 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 468987f69b03e2a3a2e8d1cb2c2594e318eb5c2b |
| Default Java | Private Build-1.8.0_422-8u422-b05-1~20.04-b05 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_422-8u422-b05-1~20.04-b05 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7061/5/testReport/ |
| Max. process+thread count | 559 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7061/5/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] Hadoop-19280. Initialize ABFS client timer only if metric collection is enabled [hadoop]
Posted by "hadoop-yetus (via GitHub)" <gi...@apache.org>.
hadoop-yetus commented on PR #7061:
URL: https://github.com/apache/hadoop/pull/7061#issuecomment-2376548532
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 21s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 32m 50s | | trunk passed |
| +1 :green_heart: | compile | 0m 24s | | trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 |
| +1 :green_heart: | compile | 0m 22s | | trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05 |
| +1 :green_heart: | checkstyle | 0m 23s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 28s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 28s | | trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 |
| +1 :green_heart: | javadoc | 0m 24s | | trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05 |
| +1 :green_heart: | spotbugs | 0m 45s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 20s | | branch has no errors when building and testing our client artifacts. |
| -0 :warning: | patch | 21m 32s | | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 17s | | the patch passed |
| +1 :green_heart: | compile | 0m 20s | | the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 |
| +1 :green_heart: | javac | 0m 20s | | the patch passed |
| +1 :green_heart: | compile | 0m 16s | | the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05 |
| +1 :green_heart: | javac | 0m 16s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| -0 :warning: | checkstyle | 0m 9s | [/results-checkstyle-hadoop-tools_hadoop-azure.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7061/9/artifact/out/results-checkstyle-hadoop-tools_hadoop-azure.txt) | hadoop-tools/hadoop-azure: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) |
| +1 :green_heart: | mvnsite | 0m 16s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 16s | | the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 |
| +1 :green_heart: | javadoc | 0m 16s | | the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05 |
| +1 :green_heart: | spotbugs | 0m 44s | | the patch passed |
| -1 :x: | shadedclient | 32m 30s | | patch has errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 1m 52s | | hadoop-azure in the patch passed. |
| +1 :green_heart: | asflicense | 0m 31s | | The patch does not generate ASF License warnings. |
| | | 95m 45s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7061/9/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/7061 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux e0f3b221eec2 5.15.0-117-generic #127-Ubuntu SMP Fri Jul 5 20:13:28 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / f729b48dfbe0baefdf1d6372cba048f57800a91e |
| Default Java | Private Build-1.8.0_422-8u422-b05-1~20.04-b05 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_422-8u422-b05-1~20.04-b05 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7061/9/testReport/ |
| Max. process+thread count | 551 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7061/9/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] Hadoop-19280. Initialize ABFS client timer only if metric collection is enabled [hadoop]
Posted by "hadoop-yetus (via GitHub)" <gi...@apache.org>.
hadoop-yetus commented on PR #7061:
URL: https://github.com/apache/hadoop/pull/7061#issuecomment-2376971460
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 49s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 49m 15s | | trunk passed |
| +1 :green_heart: | compile | 0m 40s | | trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 |
| +1 :green_heart: | compile | 0m 36s | | trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05 |
| +1 :green_heart: | checkstyle | 0m 32s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 41s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 42s | | trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 |
| +1 :green_heart: | javadoc | 0m 33s | | trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05 |
| +1 :green_heart: | spotbugs | 1m 10s | | trunk passed |
| +1 :green_heart: | shadedclient | 39m 41s | | branch has no errors when building and testing our client artifacts. |
| -0 :warning: | patch | 40m 2s | | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 30s | | the patch passed |
| +1 :green_heart: | compile | 0m 32s | | the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 |
| +1 :green_heart: | javac | 0m 32s | | the patch passed |
| +1 :green_heart: | compile | 0m 28s | | the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05 |
| +1 :green_heart: | javac | 0m 28s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | checkstyle | 0m 20s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 31s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 28s | | the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 |
| +1 :green_heart: | javadoc | 0m 24s | | the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05 |
| +1 :green_heart: | spotbugs | 1m 7s | | the patch passed |
| +1 :green_heart: | shadedclient | 40m 1s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 2m 35s | | hadoop-azure in the patch passed. |
| +1 :green_heart: | asflicense | 0m 37s | | The patch does not generate ASF License warnings. |
| | | 143m 21s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7061/10/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/7061 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux b7be1fb62134 5.15.0-118-generic #128-Ubuntu SMP Fri Jul 5 09:28:59 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 7cc5e4b17baa6559a54a4018812bb39331addc08 |
| Default Java | Private Build-1.8.0_422-8u422-b05-1~20.04-b05 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_422-8u422-b05-1~20.04-b05 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7061/10/testReport/ |
| Max. process+thread count | 527 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7061/10/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] Hadoop-19280. Initialize ABFS client timer only if metric collection is enabled [hadoop]
Posted by "hadoop-yetus (via GitHub)" <gi...@apache.org>.
hadoop-yetus commented on PR #7061:
URL: https://github.com/apache/hadoop/pull/7061#issuecomment-2368303416
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 50s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 51m 1s | | trunk passed |
| +1 :green_heart: | compile | 0m 39s | | trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 |
| +1 :green_heart: | compile | 0m 35s | | trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05 |
| +1 :green_heart: | checkstyle | 0m 32s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 41s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 41s | | trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 |
| +1 :green_heart: | javadoc | 0m 34s | | trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05 |
| +1 :green_heart: | spotbugs | 1m 6s | | trunk passed |
| +1 :green_heart: | shadedclient | 39m 7s | | branch has no errors when building and testing our client artifacts. |
| -0 :warning: | patch | 39m 29s | | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 29s | | the patch passed |
| +1 :green_heart: | compile | 0m 32s | | the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 |
| +1 :green_heart: | javac | 0m 32s | | the patch passed |
| +1 :green_heart: | compile | 0m 28s | | the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05 |
| +1 :green_heart: | javac | 0m 28s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | checkstyle | 0m 20s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 31s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 29s | | the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 |
| +1 :green_heart: | javadoc | 0m 25s | | the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05 |
| +1 :green_heart: | spotbugs | 1m 9s | | the patch passed |
| +1 :green_heart: | shadedclient | 39m 44s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 2m 33s | | hadoop-azure in the patch passed. |
| +1 :green_heart: | asflicense | 0m 37s | | The patch does not generate ASF License warnings. |
| | | 144m 26s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7061/4/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/7061 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 58d218fbdb45 5.15.0-118-generic #128-Ubuntu SMP Fri Jul 5 09:28:59 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 3a20550896964a0c673c98ea9446aeea681bfd2c |
| Default Java | Private Build-1.8.0_422-8u422-b05-1~20.04-b05 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_422-8u422-b05-1~20.04-b05 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7061/4/testReport/ |
| Max. process+thread count | 527 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7061/4/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] Hadoop-19280. Initialize ABFS client timer only if metric collection is enabled [hadoop]
Posted by "hadoop-yetus (via GitHub)" <gi...@apache.org>.
hadoop-yetus commented on PR #7061:
URL: https://github.com/apache/hadoop/pull/7061#issuecomment-2367595434
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 6m 47s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 32m 24s | | trunk passed |
| +1 :green_heart: | compile | 0m 27s | | trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 |
| +1 :green_heart: | compile | 0m 25s | | trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05 |
| +1 :green_heart: | checkstyle | 0m 23s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 28s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 29s | | trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 |
| +1 :green_heart: | javadoc | 0m 23s | | trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05 |
| +1 :green_heart: | spotbugs | 0m 45s | | trunk passed |
| +1 :green_heart: | shadedclient | 20m 45s | | branch has no errors when building and testing our client artifacts. |
| -0 :warning: | patch | 20m 59s | | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. |
|||| _ Patch Compile Tests _ |
| -1 :x: | mvninstall | 0m 18s | [/patch-mvninstall-hadoop-tools_hadoop-azure.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7061/2/artifact/out/patch-mvninstall-hadoop-tools_hadoop-azure.txt) | hadoop-azure in the patch failed. |
| -1 :x: | compile | 0m 20s | [/patch-compile-hadoop-tools_hadoop-azure-jdkUbuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7061/2/artifact/out/patch-compile-hadoop-tools_hadoop-azure-jdkUbuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.txt) | hadoop-azure in the patch failed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04. |
| -1 :x: | javac | 0m 20s | [/patch-compile-hadoop-tools_hadoop-azure-jdkUbuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7061/2/artifact/out/patch-compile-hadoop-tools_hadoop-azure-jdkUbuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.txt) | hadoop-azure in the patch failed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04. |
| -1 :x: | compile | 0m 17s | [/patch-compile-hadoop-tools_hadoop-azure-jdkPrivateBuild-1.8.0_422-8u422-b05-1~20.04-b05.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7061/2/artifact/out/patch-compile-hadoop-tools_hadoop-azure-jdkPrivateBuild-1.8.0_422-8u422-b05-1~20.04-b05.txt) | hadoop-azure in the patch failed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05. |
| -1 :x: | javac | 0m 18s | [/patch-compile-hadoop-tools_hadoop-azure-jdkPrivateBuild-1.8.0_422-8u422-b05-1~20.04-b05.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7061/2/artifact/out/patch-compile-hadoop-tools_hadoop-azure-jdkPrivateBuild-1.8.0_422-8u422-b05-1~20.04-b05.txt) | hadoop-azure in the patch failed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05. |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| -0 :warning: | checkstyle | 0m 12s | [/results-checkstyle-hadoop-tools_hadoop-azure.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7061/2/artifact/out/results-checkstyle-hadoop-tools_hadoop-azure.txt) | hadoop-tools/hadoop-azure: The patch generated 2 new + 2 unchanged - 0 fixed = 4 total (was 2) |
| -1 :x: | mvnsite | 0m 18s | [/patch-mvnsite-hadoop-tools_hadoop-azure.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7061/2/artifact/out/patch-mvnsite-hadoop-tools_hadoop-azure.txt) | hadoop-azure in the patch failed. |
| +1 :green_heart: | javadoc | 0m 17s | | the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 |
| +1 :green_heart: | javadoc | 0m 16s | | the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05 |
| -1 :x: | spotbugs | 0m 18s | [/patch-spotbugs-hadoop-tools_hadoop-azure.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7061/2/artifact/out/patch-spotbugs-hadoop-tools_hadoop-azure.txt) | hadoop-azure in the patch failed. |
| +1 :green_heart: | shadedclient | 21m 38s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| -1 :x: | unit | 0m 19s | [/patch-unit-hadoop-tools_hadoop-azure.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7061/2/artifact/out/patch-unit-hadoop-tools_hadoop-azure.txt) | hadoop-azure in the patch failed. |
| +1 :green_heart: | asflicense | 0m 25s | | The patch does not generate ASF License warnings. |
| | | 86m 53s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7061/2/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/7061 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux cb29b90dd303 5.15.0-117-generic #127-Ubuntu SMP Fri Jul 5 20:13:28 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / d2276933d12b6f6f9dfeb8ccda14508ca00a51e9 |
| Default Java | Private Build-1.8.0_422-8u422-b05-1~20.04-b05 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_422-8u422-b05-1~20.04-b05 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7061/2/testReport/ |
| Max. process+thread count | 552 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7061/2/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] Hadoop-19280. Initialize ABFS client timer only if metric collection is enabled [hadoop]
Posted by "steveloughran (via GitHub)" <gi...@apache.org>.
steveloughran commented on code in PR #7061:
URL: https://github.com/apache/hadoop/pull/7061#discussion_r1778666941
##########
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/services/TestAbfsClient.java:
##########
@@ -0,0 +1,120 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.fs.azurebfs.services;
+
+import org.apache.hadoop.conf.Configuration;
Review Comment:
import ordering.
This is always a source of trouble, but we'd like to keep it under control to ease cherry picking and emerging in new PRs. Lead to an effective policy of "try and get things in order the first time you create a file and add in new imports -but don't rearrange things once they are in a mess."
The ideal ordering is
```
java.*
javax.*
other imports
org.apache.*
static members, avoiding .* imports
```
##########
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/services/TestAbfsClient.java:
##########
@@ -0,0 +1,120 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.fs.azurebfs.services;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.azurebfs.AbfsConfiguration;
+import org.apache.hadoop.fs.azurebfs.AbfsCountersImpl;
+import org.apache.hadoop.fs.azurebfs.oauth2.AccessTokenProvider;
+import org.apache.hadoop.fs.azurebfs.utils.Base64;
+import org.apache.hadoop.fs.azurebfs.utils.MetricFormat;
+import org.assertj.core.api.Assertions;
+import org.junit.Test;
+import org.mockito.Mockito;
+
+import java.net.URI;
+import java.net.URL;
+import java.util.Map;
+
+import static org.apache.hadoop.fs.azurebfs.constants.ConfigurationKeys.FS_AZURE_METRIC_ACCOUNT_KEY;
+import static org.apache.hadoop.fs.azurebfs.constants.ConfigurationKeys.FS_AZURE_METRIC_ACCOUNT_NAME;
+import static org.apache.hadoop.fs.azurebfs.constants.ConfigurationKeys.FS_AZURE_METRIC_FORMAT;
+import static org.apache.hadoop.fs.azurebfs.services.AbfsClient.ABFS_CLIENT_TIMER_THREAD_NAME;
+
+public class TestAbfsClient {
Review Comment:
Add a little Java explaining what it is trying to test.
##########
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/services/TestAbfsClient.java:
##########
@@ -0,0 +1,120 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.fs.azurebfs.services;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.azurebfs.AbfsConfiguration;
+import org.apache.hadoop.fs.azurebfs.AbfsCountersImpl;
+import org.apache.hadoop.fs.azurebfs.oauth2.AccessTokenProvider;
+import org.apache.hadoop.fs.azurebfs.utils.Base64;
+import org.apache.hadoop.fs.azurebfs.utils.MetricFormat;
+import org.assertj.core.api.Assertions;
+import org.junit.Test;
+import org.mockito.Mockito;
+
+import java.net.URI;
+import java.net.URL;
+import java.util.Map;
+
+import static org.apache.hadoop.fs.azurebfs.constants.ConfigurationKeys.FS_AZURE_METRIC_ACCOUNT_KEY;
+import static org.apache.hadoop.fs.azurebfs.constants.ConfigurationKeys.FS_AZURE_METRIC_ACCOUNT_NAME;
+import static org.apache.hadoop.fs.azurebfs.constants.ConfigurationKeys.FS_AZURE_METRIC_FORMAT;
+import static org.apache.hadoop.fs.azurebfs.services.AbfsClient.ABFS_CLIENT_TIMER_THREAD_NAME;
+
+public class TestAbfsClient {
+ private static final String ACCOUNT_NAME = "bogusAccountName.dfs.core.windows.net";
+ private static final String ACCOUNT_KEY = "testKey";
+ private static final long SLEEP_DURATION_MS = 500;
+
+ @Test
+ public void testTimerNotInitialize() throws Exception {
+ final Configuration configuration = new Configuration();
+ AbfsConfiguration abfsConfiguration = new AbfsConfiguration(configuration, ACCOUNT_NAME);
+
+ AbfsCounters abfsCounters = Mockito.spy(new AbfsCountersImpl(new URI("abcd")));
+ AbfsClientContext abfsClientContext = new AbfsClientContextBuilder().withAbfsCounters(abfsCounters).build();
+
+ // Get an instance of AbfsClient.
+ AbfsClient client = new AbfsDfsClient(new URL("https://azure.com"),
+ null,
+ abfsConfiguration,
+ (AccessTokenProvider) null,
+ null,
+ abfsClientContext);
+
+ Assertions.assertThat(client.getTimer())
+ .describedAs("Timer should not be initialized")
+ .isNull();
+
+ // Check if a thread with the name "abfs-timer-client" exists
+ Assertions.assertThat(isThreadRunning(ABFS_CLIENT_TIMER_THREAD_NAME))
+ .describedAs("Expected thread 'abfs-timer-client' not found")
+ .isEqualTo(false);
+ client.close();
+ }
+
+ @Test
+ public void testTimerInitialize() throws Exception {
Review Comment:
2. Change the title of the method to something like `testTimerIsInitializedWithMetricCollection`
##########
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/services/TestAbfsClient.java:
##########
@@ -0,0 +1,120 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.fs.azurebfs.services;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.azurebfs.AbfsConfiguration;
+import org.apache.hadoop.fs.azurebfs.AbfsCountersImpl;
+import org.apache.hadoop.fs.azurebfs.oauth2.AccessTokenProvider;
+import org.apache.hadoop.fs.azurebfs.utils.Base64;
+import org.apache.hadoop.fs.azurebfs.utils.MetricFormat;
+import org.assertj.core.api.Assertions;
+import org.junit.Test;
+import org.mockito.Mockito;
+
+import java.net.URI;
+import java.net.URL;
+import java.util.Map;
+
+import static org.apache.hadoop.fs.azurebfs.constants.ConfigurationKeys.FS_AZURE_METRIC_ACCOUNT_KEY;
+import static org.apache.hadoop.fs.azurebfs.constants.ConfigurationKeys.FS_AZURE_METRIC_ACCOUNT_NAME;
+import static org.apache.hadoop.fs.azurebfs.constants.ConfigurationKeys.FS_AZURE_METRIC_FORMAT;
+import static org.apache.hadoop.fs.azurebfs.services.AbfsClient.ABFS_CLIENT_TIMER_THREAD_NAME;
+
+public class TestAbfsClient {
+ private static final String ACCOUNT_NAME = "bogusAccountName.dfs.core.windows.net";
+ private static final String ACCOUNT_KEY = "testKey";
+ private static final long SLEEP_DURATION_MS = 500;
+
+ @Test
+ public void testTimerNotInitialize() throws Exception {
Review Comment:
1. Add a comment stating that this verifies that's a timer is only created when metrics are being collected.
2. Change the title of the method to something like `testTimerNotInitializedWithoutMetricCollection`
##########
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/services/TestAbfsClient.java:
##########
@@ -0,0 +1,120 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.fs.azurebfs.services;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.azurebfs.AbfsConfiguration;
+import org.apache.hadoop.fs.azurebfs.AbfsCountersImpl;
+import org.apache.hadoop.fs.azurebfs.oauth2.AccessTokenProvider;
+import org.apache.hadoop.fs.azurebfs.utils.Base64;
+import org.apache.hadoop.fs.azurebfs.utils.MetricFormat;
+import org.assertj.core.api.Assertions;
+import org.junit.Test;
+import org.mockito.Mockito;
+
+import java.net.URI;
+import java.net.URL;
+import java.util.Map;
+
+import static org.apache.hadoop.fs.azurebfs.constants.ConfigurationKeys.FS_AZURE_METRIC_ACCOUNT_KEY;
+import static org.apache.hadoop.fs.azurebfs.constants.ConfigurationKeys.FS_AZURE_METRIC_ACCOUNT_NAME;
+import static org.apache.hadoop.fs.azurebfs.constants.ConfigurationKeys.FS_AZURE_METRIC_FORMAT;
+import static org.apache.hadoop.fs.azurebfs.services.AbfsClient.ABFS_CLIENT_TIMER_THREAD_NAME;
+
+public class TestAbfsClient {
+ private static final String ACCOUNT_NAME = "bogusAccountName.dfs.core.windows.net";
+ private static final String ACCOUNT_KEY = "testKey";
+ private static final long SLEEP_DURATION_MS = 500;
+
+ @Test
+ public void testTimerNotInitialize() throws Exception {
+ final Configuration configuration = new Configuration();
+ AbfsConfiguration abfsConfiguration = new AbfsConfiguration(configuration, ACCOUNT_NAME);
+
+ AbfsCounters abfsCounters = Mockito.spy(new AbfsCountersImpl(new URI("abcd")));
+ AbfsClientContext abfsClientContext = new AbfsClientContextBuilder().withAbfsCounters(abfsCounters).build();
+
+ // Get an instance of AbfsClient.
+ AbfsClient client = new AbfsDfsClient(new URL("https://azure.com"),
+ null,
+ abfsConfiguration,
+ (AccessTokenProvider) null,
+ null,
+ abfsClientContext);
+
+ Assertions.assertThat(client.getTimer())
+ .describedAs("Timer should not be initialized")
+ .isNull();
+
+ // Check if a thread with the name "abfs-timer-client" exists
+ Assertions.assertThat(isThreadRunning(ABFS_CLIENT_TIMER_THREAD_NAME))
+ .describedAs("Expected thread 'abfs-timer-client' not found")
+ .isEqualTo(false);
+ client.close();
+ }
+
+ @Test
+ public void testTimerInitialize() throws Exception {
+ final Configuration configuration = new Configuration();
+ configuration.set(FS_AZURE_METRIC_FORMAT, String.valueOf(MetricFormat.INTERNAL_BACKOFF_METRIC_FORMAT));
+ configuration.set(FS_AZURE_METRIC_ACCOUNT_NAME, ACCOUNT_NAME);
+ configuration.set(FS_AZURE_METRIC_ACCOUNT_KEY, Base64.encode(ACCOUNT_KEY.getBytes()));
+ AbfsConfiguration abfsConfiguration = new AbfsConfiguration(configuration, ACCOUNT_NAME);
+
+ AbfsCounters abfsCounters = Mockito.spy(new AbfsCountersImpl(new URI("abcd")));
+ AbfsClientContext abfsClientContext = new AbfsClientContextBuilder().withAbfsCounters(abfsCounters).build();
+
+ // Get an instance of AbfsClient.
+ AbfsClient client = new AbfsDfsClient(new URL("https://azure.com"),
+ null,
+ abfsConfiguration,
+ (AccessTokenProvider) null,
+ null,
+ abfsClientContext);
+
+ Assertions.assertThat(client.getTimer())
+ .describedAs("Timer should be initialized")
+ .isNotNull();
+
+ // Check if a thread with the name "abfs-timer-client" exists
+ Assertions.assertThat(isThreadRunning(ABFS_CLIENT_TIMER_THREAD_NAME))
+ .describedAs("Expected thread 'abfs-timer-client' not found")
+ .isEqualTo(true);
+ client.close();
+
+ // Check if the thread is removed after closing the client
+ Thread.sleep(SLEEP_DURATION_MS);
+ Assertions.assertThat(isThreadRunning(ABFS_CLIENT_TIMER_THREAD_NAME))
+ .describedAs("Unexpected thread 'abfs-timer-client' found")
+ .isEqualTo(false);
+ }
+
+ private boolean isThreadRunning(String threadName) {
Review Comment:
Explain what this does
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] Hadoop-19280. Initialize ABFS client timer only if metric collection is enabled [hadoop]
Posted by "anujmodi2021 (via GitHub)" <gi...@apache.org>.
anujmodi2021 commented on code in PR #7061:
URL: https://github.com/apache/hadoop/pull/7061#discussion_r1773666021
##########
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClient.java:
##########
@@ -1597,6 +1597,11 @@ KeepAliveCache getKeepAliveCache() {
return keepAliveCache;
}
+ @VisibleForTesting
+ Timer getTimer() {
Review Comment:
Let's have this package protected.
Unit test accessing this will still be able to access.
##########
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClient.java:
##########
@@ -258,9 +258,9 @@ private AbfsClient(final URL baseUrl,
throw new IOException("Exception while initializing metric credentials " + e);
}
}
- this.timer = new Timer(
- "abfs-timer-client", true);
if (isMetricCollectionEnabled) {
+ this.timer = new Timer(
+ "abfs-timer-client", true);
Review Comment:
While we are changing this, let's also define a Constant for the Thread name and use that here instead of hardcoded string. We can define constant in the same file as
`public static String ABFS_TIMER_THREAD_NAME = abfs-timer-client`
##########
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/services/TestAbfsClient.java:
##########
@@ -0,0 +1,117 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.fs.azurebfs.services;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.azurebfs.AbfsConfiguration;
+import org.apache.hadoop.fs.azurebfs.AbfsCountersImpl;
+import org.apache.hadoop.fs.azurebfs.oauth2.AccessTokenProvider;
+import org.apache.hadoop.fs.azurebfs.utils.Base64;
+import org.apache.hadoop.fs.azurebfs.utils.MetricFormat;
+import org.assertj.core.api.Assertions;
+import org.junit.Test;
+import org.mockito.Mockito;
+
+import java.net.URI;
+import java.net.URL;
+import java.util.Map;
+
+import static org.apache.hadoop.fs.azurebfs.constants.ConfigurationKeys.FS_AZURE_METRIC_ACCOUNT_KEY;
+import static org.apache.hadoop.fs.azurebfs.constants.ConfigurationKeys.FS_AZURE_METRIC_ACCOUNT_NAME;
+import static org.apache.hadoop.fs.azurebfs.constants.ConfigurationKeys.FS_AZURE_METRIC_FORMAT;
+
+public class TestAbfsClient {
+ private static final String ACCOUNT_NAME = "bogusAccountName.dfs.core.windows.net";
+ private static final String ACCOUNT_KEY = "testKey";
+
+ @Test
+ public void testTimerNotInitialize() throws Exception {
+ final Configuration configuration = new Configuration();
+ AbfsConfiguration abfsConfiguration = new AbfsConfiguration(configuration, ACCOUNT_NAME);
+
+ AbfsCounters abfsCounters = Mockito.spy(new AbfsCountersImpl(new URI("abcd")));
+ AbfsClientContext abfsClientContext = new AbfsClientContextBuilder().withAbfsCounters(abfsCounters).build();
+
+ // Get an instance of AbfsClient.
+ AbfsClient client = new AbfsDfsClient(new URL("https://azure.com"),
+ null,
+ abfsConfiguration,
+ (AccessTokenProvider) null,
+ null,
+ abfsClientContext);
+
+ Assertions.assertThat(client.getTimer())
+ .describedAs("Timer should not be initialized")
+ .isNull();
+
+ // Check if a thread with the name "abfs-timer-client" exists
+ Assertions.assertThat(isThreadRunning("abfs-timer-client"))
Review Comment:
Use the same constant defined in ABFSClient for thread name throughout the class.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] Hadoop-19280. Initialize ABFS client timer only if metric collection is enabled [hadoop]
Posted by "anmolanmol1234 (via GitHub)" <gi...@apache.org>.
anmolanmol1234 commented on code in PR #7061:
URL: https://github.com/apache/hadoop/pull/7061#discussion_r1771106538
##########
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/services/ITestAbfsClient.java:
##########
@@ -685,4 +692,69 @@ public void testExpectHundredContinue() throws Exception {
.describedAs("The expect header is not false")
.isFalse();
}
+
+ @Test
+ public void testTimerNotInitialize() throws Exception {
+ // Create an AzureBlobFileSystem instance.
+ final Configuration configuration = getRawConfiguration();
+ AbfsConfiguration abfsConfiguration = new AbfsConfiguration(configuration, ACCOUNT_NAME);
+
+ AbfsCounters abfsCounters = Mockito.spy(new AbfsCountersImpl(new URI("abcd")));
+ AbfsClientContext abfsClientContext = new AbfsClientContextBuilder().withAbfsCounters(abfsCounters).build();
+
+ // Get an instance of AbfsClient.
+ AbfsClient client = new AbfsDfsClient(new URL("https://azure.com"),
+ null,
+ abfsConfiguration,
+ (AccessTokenProvider) null,
+ null,
+ abfsClientContext);
+
+ assertNull(client.getTimer());
+ boolean isTimerThreadPresent = isThreadRunning("abfs-timer-client");
+
+ // Check if a thread with the name "abfs-timer-client" exists
+ assertFalse("Unexpected thread 'abfs-timer-client' found", isTimerThreadPresent);
Review Comment:
Use assertions.assertThat
##########
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/services/ITestAbfsClient.java:
##########
@@ -685,4 +692,69 @@ public void testExpectHundredContinue() throws Exception {
.describedAs("The expect header is not false")
.isFalse();
}
+
+ @Test
+ public void testTimerNotInitialize() throws Exception {
+ // Create an AzureBlobFileSystem instance.
+ final Configuration configuration = getRawConfiguration();
+ AbfsConfiguration abfsConfiguration = new AbfsConfiguration(configuration, ACCOUNT_NAME);
+
+ AbfsCounters abfsCounters = Mockito.spy(new AbfsCountersImpl(new URI("abcd")));
+ AbfsClientContext abfsClientContext = new AbfsClientContextBuilder().withAbfsCounters(abfsCounters).build();
+
+ // Get an instance of AbfsClient.
+ AbfsClient client = new AbfsDfsClient(new URL("https://azure.com"),
+ null,
+ abfsConfiguration,
+ (AccessTokenProvider) null,
+ null,
+ abfsClientContext);
+
+ assertNull(client.getTimer());
+ boolean isTimerThreadPresent = isThreadRunning("abfs-timer-client");
+
+ // Check if a thread with the name "abfs-timer-client" exists
+ assertFalse("Unexpected thread 'abfs-timer-client' found", isTimerThreadPresent);
+ client.close();
+ }
+
+ @Test
+ public void testTimerInitialize() throws Exception {
+ // Create an AzureBlobFileSystem instance.
+ final Configuration configuration = getRawConfiguration();
+ configuration.set(FS_AZURE_METRIC_FORMAT, String.valueOf(MetricFormat.INTERNAL_BACKOFF_METRIC_FORMAT));
+ configuration.set(FS_AZURE_METRIC_ACCOUNT_NAME, ACCOUNT_NAME);
+ configuration.set(FS_AZURE_METRIC_ACCOUNT_KEY, Base64.encode(ACCOUNT_KEY.getBytes()));
+ AbfsConfiguration abfsConfiguration = new AbfsConfiguration(configuration, ACCOUNT_NAME);
+
+ AbfsCounters abfsCounters = Mockito.spy(new AbfsCountersImpl(new URI("abcd")));
+ AbfsClientContext abfsClientContext = new AbfsClientContextBuilder().withAbfsCounters(abfsCounters).build();
+
+ // Get an instance of AbfsClient.
+ AbfsClient client = new AbfsDfsClient(new URL("https://azure.com"),
+ null,
+ abfsConfiguration,
+ (AccessTokenProvider) null,
+ null,
+ abfsClientContext);
+
+ assertNotNull(client.getTimer());
+ // Check if a thread with the name "abfs-timer-client" exists
+ boolean isTimerThreadPresent = isThreadRunning("abfs-timer-client");
+ assertTrue("Expected thread 'abfs-timer-client' not found", isTimerThreadPresent);
+ client.close();
Review Comment:
Also add assertions that after client is closed, whether the thread is still running
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] Hadoop-19280. Initialize ABFS client timer only if metric collection is enabled [hadoop]
Posted by "hadoop-yetus (via GitHub)" <gi...@apache.org>.
hadoop-yetus commented on PR #7061:
URL: https://github.com/apache/hadoop/pull/7061#issuecomment-2372209897
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 47s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 50m 32s | | trunk passed |
| +1 :green_heart: | compile | 0m 40s | | trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 |
| +1 :green_heart: | compile | 0m 36s | | trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05 |
| +1 :green_heart: | checkstyle | 0m 32s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 41s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 41s | | trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 |
| +1 :green_heart: | javadoc | 0m 32s | | trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05 |
| +1 :green_heart: | spotbugs | 1m 9s | | trunk passed |
| +1 :green_heart: | shadedclient | 40m 5s | | branch has no errors when building and testing our client artifacts. |
| -0 :warning: | patch | 40m 26s | | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 30s | | the patch passed |
| +1 :green_heart: | compile | 0m 32s | | the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 |
| +1 :green_heart: | javac | 0m 32s | | the patch passed |
| +1 :green_heart: | compile | 0m 28s | | the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05 |
| +1 :green_heart: | javac | 0m 28s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | checkstyle | 0m 19s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 32s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 28s | | the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 |
| +1 :green_heart: | javadoc | 0m 25s | | the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05 |
| +1 :green_heart: | spotbugs | 1m 6s | | the patch passed |
| +1 :green_heart: | shadedclient | 39m 55s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 2m 34s | | hadoop-azure in the patch passed. |
| +1 :green_heart: | asflicense | 0m 37s | | The patch does not generate ASF License warnings. |
| | | 145m 8s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7061/7/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/7061 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux d7ea76dba3b2 5.15.0-118-generic #128-Ubuntu SMP Fri Jul 5 09:28:59 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 401edded40bed5769d6af0665136d28f2bdd0620 |
| Default Java | Private Build-1.8.0_422-8u422-b05-1~20.04-b05 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_422-8u422-b05-1~20.04-b05 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7061/7/testReport/ |
| Max. process+thread count | 586 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7061/7/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] Hadoop-19280. Initialize ABFS client timer only if metric collection is enabled [hadoop]
Posted by "hadoop-yetus (via GitHub)" <gi...@apache.org>.
hadoop-yetus commented on PR #7061:
URL: https://github.com/apache/hadoop/pull/7061#issuecomment-2367500262
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 1m 18s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 49m 14s | | trunk passed |
| +1 :green_heart: | compile | 0m 43s | | trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 |
| +1 :green_heart: | compile | 0m 37s | | trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05 |
| +1 :green_heart: | checkstyle | 0m 32s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 42s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 42s | | trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 |
| +1 :green_heart: | javadoc | 0m 33s | | trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05 |
| +1 :green_heart: | spotbugs | 1m 9s | | trunk passed |
| +1 :green_heart: | shadedclient | 39m 52s | | branch has no errors when building and testing our client artifacts. |
| -0 :warning: | patch | 40m 13s | | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. |
|||| _ Patch Compile Tests _ |
| -1 :x: | mvninstall | 0m 27s | [/patch-mvninstall-hadoop-tools_hadoop-azure.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7061/1/artifact/out/patch-mvninstall-hadoop-tools_hadoop-azure.txt) | hadoop-azure in the patch failed. |
| -1 :x: | compile | 0m 31s | [/patch-compile-hadoop-tools_hadoop-azure-jdkUbuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7061/1/artifact/out/patch-compile-hadoop-tools_hadoop-azure-jdkUbuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.txt) | hadoop-azure in the patch failed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04. |
| -1 :x: | javac | 0m 31s | [/patch-compile-hadoop-tools_hadoop-azure-jdkUbuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7061/1/artifact/out/patch-compile-hadoop-tools_hadoop-azure-jdkUbuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.txt) | hadoop-azure in the patch failed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04. |
| -1 :x: | compile | 0m 27s | [/patch-compile-hadoop-tools_hadoop-azure-jdkPrivateBuild-1.8.0_422-8u422-b05-1~20.04-b05.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7061/1/artifact/out/patch-compile-hadoop-tools_hadoop-azure-jdkPrivateBuild-1.8.0_422-8u422-b05-1~20.04-b05.txt) | hadoop-azure in the patch failed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05. |
| -1 :x: | javac | 0m 27s | [/patch-compile-hadoop-tools_hadoop-azure-jdkPrivateBuild-1.8.0_422-8u422-b05-1~20.04-b05.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7061/1/artifact/out/patch-compile-hadoop-tools_hadoop-azure-jdkPrivateBuild-1.8.0_422-8u422-b05-1~20.04-b05.txt) | hadoop-azure in the patch failed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05. |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| -0 :warning: | checkstyle | 0m 20s | [/results-checkstyle-hadoop-tools_hadoop-azure.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7061/1/artifact/out/results-checkstyle-hadoop-tools_hadoop-azure.txt) | hadoop-tools/hadoop-azure: The patch generated 2 new + 2 unchanged - 0 fixed = 4 total (was 2) |
| -1 :x: | mvnsite | 0m 29s | [/patch-mvnsite-hadoop-tools_hadoop-azure.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7061/1/artifact/out/patch-mvnsite-hadoop-tools_hadoop-azure.txt) | hadoop-azure in the patch failed. |
| +1 :green_heart: | javadoc | 0m 27s | | the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 |
| +1 :green_heart: | javadoc | 0m 25s | | the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05 |
| -1 :x: | spotbugs | 0m 28s | [/patch-spotbugs-hadoop-tools_hadoop-azure.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7061/1/artifact/out/patch-spotbugs-hadoop-tools_hadoop-azure.txt) | hadoop-azure in the patch failed. |
| +1 :green_heart: | shadedclient | 42m 49s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| -1 :x: | unit | 0m 33s | [/patch-unit-hadoop-tools_hadoop-azure.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7061/1/artifact/out/patch-unit-hadoop-tools_hadoop-azure.txt) | hadoop-azure in the patch failed. |
| +1 :green_heart: | asflicense | 0m 37s | | The patch does not generate ASF License warnings. |
| | | 141m 38s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7061/1/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/7061 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 4d828596df44 5.15.0-118-generic #128-Ubuntu SMP Fri Jul 5 09:28:59 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / af594f4622719ab358b6b1835069303cea7c87a4 |
| Default Java | Private Build-1.8.0_422-8u422-b05-1~20.04-b05 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_422-8u422-b05-1~20.04-b05 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7061/1/testReport/ |
| Max. process+thread count | 526 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7061/1/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] Hadoop-19280. Initialize ABFS client timer only if metric collection is enabled [hadoop]
Posted by "bhattmanish98 (via GitHub)" <gi...@apache.org>.
bhattmanish98 commented on PR #7061:
URL: https://github.com/apache/hadoop/pull/7061#issuecomment-2370701507
============================================================
HNS-OAuth
============================================================
[ERROR] testFileSystemCapabilities(org.apache.hadoop.fs.azurebfs.ITestFileSystemInitialization) Time elapsed: 0.402 s <<< FAILURE!
[ERROR] testFsActionWRITE(org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemCheckAccess) Time elapsed: 2.069 s <<< FAILURE!
[ERROR] testFsActionNONE(org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemCheckAccess) Time elapsed: 1.737 s <<< FAILURE!
[ERROR] testCheckAccessForNonExistentFile(org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemCheckAccess) Time elapsed: 2.085 s <<< FAILURE!
[ERROR] testFsActionREAD(org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemCheckAccess) Time elapsed: 1.843 s <<< FAILURE!
[ERROR] testFsActionEXECUTE(org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemCheckAccess) Time elapsed: 2.057 s <<< FAILURE!
[ERROR] testFsActionREADEXECUTE(org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemCheckAccess) Time elapsed: 1.978 s <<< FAILURE!
[ERROR] testFsActionWRITEEXECUTE(org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemCheckAccess) Time elapsed: 2.205 s <<< FAILURE!
[ERROR] testEtagConsistencyAcrossListAndHead(org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractEtag) Time elapsed: 0.427 s <<< FAILURE!
[ERROR] testDeleteTargetPaths(org.apache.hadoop.fs.azurebfs.commit.ITestAbfsRenameStageFailure) Time elapsed: 2.091 s <<< FAILURE!
[ERROR] testBackoffRetryMetrics(org.apache.hadoop.fs.azurebfs.services.TestAbfsRestOperation) Time elapsed: 1.597 s <<< ERROR!
[ERROR] testReadFooterMetrics(org.apache.hadoop.fs.azurebfs.ITestAbfsReadFooterMetrics) Time elapsed: 0.479 s <<< ERROR!
[ERROR] testMetricWithIdlePeriod(org.apache.hadoop.fs.azurebfs.ITestAbfsReadFooterMetrics) Time elapsed: 0.462 s <<< ERROR!
[ERROR] testReadFooterMetricsWithParquetAndNonParquet(org.apache.hadoop.fs.azurebfs.ITestAbfsReadFooterMetrics) Time elapsed: 0.449 s <<< ERROR!
[ERROR] Tests run: 157, Failures: 0, Errors: 1, Skipped: 2
[ERROR] Tests run: 651, Failures: 8, Errors: 3, Skipped: 82
[ERROR] Tests run: 433, Failures: 2, Errors: 0, Skipped: 60
Test failures are known, and some are fixed in the below PRs.
1. https://github.com/apache/hadoop/pull/6847
2. Failing tests are known and we are working on the fixes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] Hadoop-19280. Initialize ABFS client timer only if metric collection is enabled [hadoop]
Posted by "saikatroy038 (via GitHub)" <gi...@apache.org>.
saikatroy038 commented on code in PR #7061:
URL: https://github.com/apache/hadoop/pull/7061#discussion_r1776428028
##########
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClient.java:
##########
@@ -1418,7 +1419,7 @@ private TracingContext getMetricTracingContext() {
boolean timerOrchestrator(TimerFunctionality timerFunctionality, TimerTask timerTask) {
switch (timerFunctionality) {
case RESUME:
- if (isMetricCollectionStopped.get()) {
+ if (isMetricCollectionEnabled) {
Review Comment:
@bhattmanish98 The double-check on isMetricCollectionStopped.get() is to avoid unnecessary synchronization and improve performance
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org