You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by GitBox <gi...@apache.org> on 2020/04/05 05:37:12 UTC
[GitHub] [hadoop-ozone] avijayanhwx opened a new pull request #772:
HDDS-3335. Recon unit tests cleanup.
avijayanhwx opened a new pull request #772: HDDS-3335. Recon unit tests cleanup.
URL: https://github.com/apache/hadoop-ozone/pull/772
## What changes were proposed in this pull request?
- Created class to setup a recon test injector, with any combination of sub modules that are specified.
- Created class that provides a Recon SQL DB with all the tables created, and APIs to access the DAOs easily.
- Cleaned up injector usage in Recon tests.
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-3335
## How was this patch tested?
Manually tested.
Unit tests.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] avijayanhwx merged pull request #772: HDDS-3335.
Recon unit tests cleanup.
Posted by GitBox <gi...@apache.org>.
avijayanhwx merged pull request #772: HDDS-3335. Recon unit tests cleanup.
URL: https://github.com/apache/hadoop-ozone/pull/772
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] avijayanhwx commented on issue #772: HDDS-3335.
Recon unit tests cleanup.
Posted by GitBox <gi...@apache.org>.
avijayanhwx commented on issue #772: HDDS-3335. Recon unit tests cleanup.
URL: https://github.com/apache/hadoop-ozone/pull/772#issuecomment-609913507
@vivekratnavel / @swagle Please review.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] vivekratnavel commented on a change in pull request
#772: HDDS-3335. Recon unit tests cleanup.
Posted by GitBox <gi...@apache.org>.
vivekratnavel commented on a change in pull request #772: HDDS-3335. Recon unit tests cleanup.
URL: https://github.com/apache/hadoop-ozone/pull/772#discussion_r404338301
##########
File path: hadoop-ozone/recon/src/test/java/org/apache/hadoop/ozone/recon/ReconTestInjector.java
##########
@@ -0,0 +1,341 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ * <p>
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * <p>
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.ozone.recon;
+
+import static org.apache.hadoop.hdds.recon.ReconConfigKeys.OZONE_RECON_DATANODE_ADDRESS_KEY;
+import static org.apache.hadoop.ozone.recon.ReconServerConfigKeys.OZONE_RECON_DB_DIR;
+import static org.apache.hadoop.ozone.recon.ReconServerConfigKeys.OZONE_RECON_OM_SNAPSHOT_DB_DIR;
+import static org.apache.hadoop.ozone.recon.ReconServerConfigKeys.OZONE_RECON_SCM_DB_DIR;
+
+import java.io.File;
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+
+import org.apache.commons.collections.CollectionUtils;
+import org.apache.hadoop.hdds.conf.OzoneConfiguration;
+import org.apache.hadoop.hdds.scm.server.OzoneStorageContainerManager;
+import org.apache.hadoop.hdds.utils.db.DBStore;
+import org.apache.hadoop.ozone.recon.persistence.AbstractReconSqlDBTest;
+import org.apache.hadoop.ozone.recon.recovery.ReconOMMetadataManager;
+import org.apache.hadoop.ozone.recon.spi.ContainerDBServiceProvider;
+import org.apache.hadoop.ozone.recon.spi.OzoneManagerServiceProvider;
+import org.apache.hadoop.ozone.recon.spi.impl.ContainerDBServiceProviderImpl;
+import org.apache.hadoop.ozone.recon.spi.impl.ReconContainerDBProvider;
+import org.junit.Assert;
+import org.junit.rules.TemporaryFolder;
+
+import com.google.inject.AbstractModule;
+import com.google.inject.Guice;
+import com.google.inject.Injector;
+import com.google.inject.Module;
+import com.google.inject.Singleton;
+
+/**
+ * Class to setup a recon test injector, with any combination of sub modules
+ * that are specified. This Recon specific abstraction to Guice API has
+ * been created to simplify the process of setting up a test environment for
+ * unit testing.
+ */
+public class ReconTestInjector {
+
+ private Injector injector;
+ private OzoneManagerServiceProvider ozoneManagerServiceProvider;
+ private ReconOMMetadataManager reconOMMetadataManager;
+ private OzoneStorageContainerManager reconScm;
+ private AbstractReconSqlDBTest reconSqlDB;
+ private boolean withContainerDB = false;
+ private List<Module> additionalModules = new ArrayList<>();
+ private boolean withReconSqlDb = false;
+ private TemporaryFolder temporaryFolder;
+ private Map<Class, Class> extraInheritedBindings = new HashMap<>();
+ private Map<Class, Object> extraInstanceBindings = new HashMap<>();
+ private Set<Class> extraClassBindings = new HashSet<>();
+
+ public ReconTestInjector(TemporaryFolder temporaryFolder) {
+ this.temporaryFolder = temporaryFolder;
+ }
+
+ public void setWithReconSqlDb(boolean withReconSqlDb) {
+ this.withReconSqlDb = withReconSqlDb;
+ }
+
+ public void setOzoneManagerServiceProvider(
+ OzoneManagerServiceProvider ozoneManagerServiceProvider) {
+ this.ozoneManagerServiceProvider = ozoneManagerServiceProvider;
+ }
+
+ public void setReconOMMetadataManager(
+ ReconOMMetadataManager reconOMMetadataManager) {
+ this.reconOMMetadataManager = reconOMMetadataManager;
+ }
+
+ public void setReconScm(OzoneStorageContainerManager reconScm) {
+ this.reconScm = reconScm;
+ }
+
+ public void withContainerDB(boolean containerDbIncluded) {
+ this.withContainerDB = containerDbIncluded;
+ }
+
+ public OzoneManagerServiceProvider getOzoneManagerServiceProvider() {
+ return ozoneManagerServiceProvider;
+ }
+
+ public ReconOMMetadataManager getReconOMMetadataManager() {
+ return reconOMMetadataManager;
+ }
+
+ public OzoneStorageContainerManager getReconScm() {
+ return reconScm;
+ }
+
+ public List<Module> getAdditionalModules() {
+ return additionalModules;
+ }
+
+ public Map<Class, Object> getExtraInstanceBindings() {
+ return extraInstanceBindings;
+ }
+
+ public Map<Class, Class> getExtraInheritedBindings() {
+ return extraInheritedBindings;
+ }
+
+ public Set<Class> getExtraClassBindings() {
+ return extraClassBindings;
+ }
+
+ /**
+ * Wrapper to get the bound instance.
+ * @param type type
+ * @param <T> type
+ * @return bound instance of type T.
+ */
+ public <T> T getInstance(Class<T> type) {
+ return injector.getInstance(type);
+ }
+
+ /**
+ * The goal of the class is to discourage the use of injector to
+ * create more child injectors explicitly.
+ * Use this API wisely!
+ * @return injector.
+ */
+ public Injector getInjector() {
+ return injector;
+ }
+
+ void setupInjector() throws IOException {
+ List<Module> modules = new ArrayList<>();
+
+ modules.add(new AbstractModule() {
+ @Override
+ protected void configure() {
+ try {
+ bind(OzoneConfiguration.class).toInstance(
+ getTestOzoneConfiguration(temporaryFolder.newFolder()));
+
+ if (reconOMMetadataManager != null) {
+ bind(ReconOMMetadataManager.class)
+ .toInstance(reconOMMetadataManager);
+ }
+ if (reconOMMetadataManager != null) {
+ bind(ReconOMMetadataManager.class)
+ .toInstance(reconOMMetadataManager);
+ }
Review comment:
```suggestion
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] avijayanhwx commented on a change in pull request
#772: HDDS-3335. Recon unit tests cleanup.
Posted by GitBox <gi...@apache.org>.
avijayanhwx commented on a change in pull request #772: HDDS-3335. Recon unit tests cleanup.
URL: https://github.com/apache/hadoop-ozone/pull/772#discussion_r405642888
##########
File path: hadoop-ozone/recon/src/test/java/org/apache/hadoop/ozone/recon/fsck/TestMissingContainerTask.java
##########
@@ -93,12 +82,13 @@ public void testRun() throws Exception {
when(containerManagerMock.getContainerReplicas(new ContainerID(3L)))
.thenReturn(Collections.emptySet());
- List<MissingContainers> all = missingContainersTableHandle.findAll();
+ MissingContainersDao missingContainersDao =
+ getDao(MissingContainersDao.class);
+ List<MissingContainers> all = missingContainersDao.findAll();
Review comment:
Thank you @adoroszlai. Fixed it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] avijayanhwx commented on issue #772: HDDS-3335.
Recon unit tests cleanup.
Posted by GitBox <gi...@apache.org>.
avijayanhwx commented on issue #772: HDDS-3335. Recon unit tests cleanup.
URL: https://github.com/apache/hadoop-ozone/pull/772#issuecomment-611815295
Thank you for the reviews @vivekratnavel & @adoroszlai.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] adoroszlai commented on a change in pull request
#772: HDDS-3335. Recon unit tests cleanup.
Posted by GitBox <gi...@apache.org>.
adoroszlai commented on a change in pull request #772: HDDS-3335. Recon unit tests cleanup.
URL: https://github.com/apache/hadoop-ozone/pull/772#discussion_r405350140
##########
File path: hadoop-ozone/recon/src/test/java/org/apache/hadoop/ozone/recon/fsck/TestMissingContainerTask.java
##########
@@ -93,12 +82,13 @@ public void testRun() throws Exception {
when(containerManagerMock.getContainerReplicas(new ContainerID(3L)))
.thenReturn(Collections.emptySet());
- List<MissingContainers> all = missingContainersTableHandle.findAll();
+ MissingContainersDao missingContainersDao =
+ getDao(MissingContainersDao.class);
+ List<MissingContainers> all = missingContainersDao.findAll();
Review comment:
Do we need to get the dao again? `missingContainersTableHandle` already has an instance.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org