You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "Will Xiao (Jira)" <ji...@apache.org> on 2024/03/12 16:53:00 UTC
[jira] [Comment Edited] (HDDS-10311) Speed up TestOmMetrics
[ https://issues.apache.org/jira/browse/HDDS-10311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17825733#comment-17825733 ]
Will Xiao edited comment on HDDS-10311 at 3/12/24 4:52 PM:
-----------------------------------------------------------
Hi [~adoroszlai], I tried to use @BeforeAll and @AfterAll to build cluster only once
{code:java}
@TestInstance(Lifecycle.PER_CLASS)
@Timeout(300)
public class TestOmMetrics {
private MiniOzoneCluster cluster;
private MiniOzoneCluster.Builder clusterBuilder;
private OzoneConfiguration conf;
private OzoneManager ozoneManager;
private OzoneManagerProtocol writeClient;
private final OMException exception =
new OMException("dummyException", OMException.ResultCodes.TIMEOUT);
private OzoneClient client;
@BeforeAll
public void setup() throws Exception {
conf = new OzoneConfiguration();
conf.setTimeDuration(OMConfigKeys.OZONE_OM_METRICS_SAVE_INTERVAL,
1000, TimeUnit.MILLISECONDS);
clusterBuilder = MiniOzoneCluster.newBuilder(conf).withoutDatanodes();
startCluster();
}
private void startCluster() throws Exception {
cluster = clusterBuilder.build();
cluster.waitForClusterToBeReady();
ozoneManager = cluster.getOzoneManager();
client = cluster.newClient();
writeClient = client.getObjectStore()
.getClientProxy().getOzoneManagerClient();
}
@AfterAll
public void shutdown() {
IOUtils.closeQuietly(client);
if (cluster != null) {
cluster.shutdown();
}
}{code}
Only keep testVolumeOps() and testBucketOps() and comment out other test classes.
removed startCluster() from each test class.
The mvn test failed at this line testVolumeOps:122
{code:java}
MetricsRecordBuilder omMetrics = getMetrics("OMMetrics");{code}
ERROR:
{code:java}
[ERROR] org.apache.hadoop.ozone.om.TestOmMetrics.testBucketOps Time elapsed: 4.072 s [ERROR] org.apache.hadoop.ozone.om.TestOmMetrics.testVolumeOps Time elapsed: 4.094 s <<< ERROR! java.lang.NullPointerException at org.apache.ozone.test.MetricsAsserts.getMetrics(MetricsAsserts.java:101) at org.apache.ozone.test.MetricsAsserts.getMetrics(MetricsAsserts.java:110) at org.apache.ozone.test.MetricsAsserts.getMetrics(MetricsAsserts.java:106) at org.apache.hadoop.ozone.om.TestOmMetrics.testVolumeOps(TestOmMetrics.java:122) at java.lang.reflect.Method.invoke(Method.java:498) at java.util.ArrayList.forEach(ArrayList.java:1259) at java.util.ArrayList.forEach(ArrayList.java:1259) {code}
I tried to remove all other test classes and only keep testVolumeOps(), and it was successful. looks like getMetrics in both tests are in conflict.
Any idea how to handle getMetrics if we want both test classes to share one cluster?
was (Author: JIRAUSER296695):
Hi [~adoroszlai], I tried to use @BeforeAll and @AfterAll to build cluster only once
{code:java}
@TestInstance(Lifecycle.PER_CLASS)
@Timeout(300)
public class TestOmMetrics {
private MiniOzoneCluster cluster;
private MiniOzoneCluster.Builder clusterBuilder;
private OzoneConfiguration conf;
private OzoneManager ozoneManager;
private OzoneManagerProtocol writeClient;
private final OMException exception =
new OMException("dummyException", OMException.ResultCodes.TIMEOUT);
private OzoneClient client;
@BeforeAll
public void setup() throws Exception {
conf = new OzoneConfiguration();
conf.setTimeDuration(OMConfigKeys.OZONE_OM_METRICS_SAVE_INTERVAL,
1000, TimeUnit.MILLISECONDS);
clusterBuilder = MiniOzoneCluster.newBuilder(conf).withoutDatanodes();
startCluster();
}
private void startCluster() throws Exception {
cluster = clusterBuilder.build();
cluster.waitForClusterToBeReady();
ozoneManager = cluster.getOzoneManager();
client = cluster.newClient();
writeClient = client.getObjectStore()
.getClientProxy().getOzoneManagerClient();
}
@AfterAll
public void shutdown() {
IOUtils.closeQuietly(client);
if (cluster != null) {
cluster.shutdown();
}
}{code}
Only keep testVolumeOps() and testBucketOps() and comment out other test classes.
removed startCluster() from each test class.
The mvn test failed at this line testVolumeOps:122
{code:java}
MetricsRecordBuilder omMetrics = getMetrics("OMMetrics");{code}
ERROR:
{code:java}
[ERROR] org.apache.hadoop.ozone.om.TestOmMetrics.testBucketOps Time elapsed: 4.072 s [ERROR] org.apache.hadoop.ozone.om.TestOmMetrics.testVolumeOps Time elapsed: 4.094 s <<< ERROR! java.lang.NullPointerException at org.apache.ozone.test.MetricsAsserts.getMetrics(MetricsAsserts.java:101) at org.apache.ozone.test.MetricsAsserts.getMetrics(MetricsAsserts.java:110) at org.apache.ozone.test.MetricsAsserts.getMetrics(MetricsAsserts.java:106) at org.apache.hadoop.ozone.om.TestOmMetrics.testVolumeOps(TestOmMetrics.java:122) at java.lang.reflect.Method.invoke(Method.java:498) at java.util.ArrayList.forEach(ArrayList.java:1259) at java.util.ArrayList.forEach(ArrayList.java:1259) {code}
I tried to remove all other test classes and only keep testVolumeOps(), and it was successful. looks like getMetrics in both tests are in conflict.
Any idea how to handle getMetrics if we want both test classes to share one cluster?
> Speed up TestOmMetrics
> ----------------------
>
> Key: HDDS-10311
> URL: https://issues.apache.org/jira/browse/HDDS-10311
> Project: Apache Ozone
> Issue Type: Sub-task
> Components: test
> Reporter: Attila Doroszlai
> Assignee: Will Xiao
> Priority: Major
>
> {code}
> Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 115.365 s - in org.apache.hadoop.ozone.om.TestOmMetrics
> {code}
> # assertions about metrics should be merged into functional tests (no need to test e.g. {{listKeys}} and metrics for {{listKeys}} separately
> # no need to start new cluster for each test case
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org