[jira] [Comment Edited] (HDDS-1458) Create a maven profile to run fault injection tests

["Eric Yang (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HDDS-1458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16831213#comment-16831213 ] 

Eric Yang edited comment on HDDS-1458 at 5/1/19 7:59 PM:
---------------------------------------------------------

[~elek] Patch 002 move blockade tests from dist to fault-injection-test/network-tests.  It also created a setup for disk based tests.  There is a problem with Ozone docker image is that it mounts ozone tarball in expanded form from dist/target directory.  This prevents integration-test to reiterate on the same ozone binaries.  I made a fix for docker image to build as a separate submodule using maven-assembly instead of dist-tar-stitching.  Dist-tar-stitching was invented to fix symlink support in Hadoop tarball.  However, it introduced a regression in maven dependency.  Maven is unable to reference ozone tarball for docker image build.  I verified that ozone tarball does not require symlink, therefore, switching to maven-assembly can produce Docker Ozone image in maven build more efficiently. 

In YARN-9523, I suggest to use dist profile to build docker images.  For consistency, Ozone docker image will be built if -Pdist flag is given.  There were oppositions in YARN community to make docker build part of the default build.  [~jeagles] [~ebadger], please speak up if you have concerns to rename docker profile to dist profile.  As a compromise, we made docker build optional with -Pdocker profile.  However, it might be worth while to review if we can use dist profile to build distribution.  If we have intend to release docker images as part of the release.  Without a inline build docker image, the fault-injection test will try to use docker image which has the same tag name as apache/ozone:[version] from either local cache, or docker hub.


was (Author: eyang):
[~elek] Patch 002 move blockade tests from dist to fault-injection-test/network-tests.  It also created a setup for disk based tests.  There is a problem with Ozone docker image is that it mounts ozone tarball in expanded form from dist/target directory.  This prevents integration-test to reiterate on the same ozone binaries.  I made a fix for docker image to build as a separate submodule using maven-assembly instead of dist-tar-stitching.  Dist-tar-stitching was invented to fix symlink support in tarball.  However, it introduced a regression in maven dependency.  Maven is unable to reference ozone tarball for docker image build.  I verified that ozone tarball does not require symlink, therefore, switching to maven-assembly can produce Docker Ozone image in maven build more efficiently. 

> Create a maven profile to run fault injection tests
> ---------------------------------------------------
>
>                 Key: HDDS-1458
>                 URL: https://issues.apache.org/jira/browse/HDDS-1458
>             Project: Hadoop Distributed Data Store
>          Issue Type: Test
>            Reporter: Eric Yang
>            Assignee: Eric Yang
>            Priority: Major
>         Attachments: HDDS-1458.001.patch, HDDS-1458.002.patch
>
>
> Some fault injection tests have been written using blockade.  It would be nice to have ability to start docker compose and exercise the blockade test cases against Ozone docker containers, and generate reports.  This is optional integration tests to catch race conditions and fault tolerance defects. 
> We can introduce a profile with id: it (short for integration tests).  This will launch docker compose via maven-exec-plugin and run blockade to simulate container failures and timeout.
> Usage command:
> {code}
> mvn clean verify -Pit
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org