You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@ozone.apache.org by el...@apache.org on 2020/06/04 12:01:18 UTC
[hadoop-ozone] branch acceptance-coverage created (now 8ef19b2)
This is an automated email from the ASF dual-hosted git repository.
elek pushed a change to branch acceptance-coverage
in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git.
at 8ef19b2 retrigger ci
This branch includes the following new commits:
new 1d744f2 collect coverage data from basic acceptance tests
new a3d332e merge the results to one file
new bfa9ef0 collect acceptnce test results when results are combined
new c32ad42 fix jacoco test collection
new 15873f2 Handle the case when only one acceptance test is covered
new 41b2cc3 use top level check script
new ad874f9 simplified acceptance test
new 56a7775 fix checkout
new fb31356 fix checkout
new 089d167 test
new 9b00d1b do full clone
new 8c261f4 debug acceptance script
new 6960836 use cache
new facee0e fix actions structure
new 857181d I said run not runs
new ccddc8b fix the location of the target dir creation
new 3d1d0d2 fix repository caching
new 8ef19b2 retrigger ci
The 18 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-commits-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-commits-help@hadoop.apache.org
[hadoop-ozone] 17/18: fix repository caching
Posted by el...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
elek pushed a commit to branch acceptance-coverage
in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git
commit 3d1d0d29557604e2b10b996e0d18eef41b489447
Author: Elek Márton <el...@apache.org>
AuthorDate: Thu Jun 4 13:59:38 2020 +0200
fix repository caching
---
.github/workflows/post-commit.yml | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/.github/workflows/post-commit.yml b/.github/workflows/post-commit.yml
index f05a9f8..5409dcb 100644
--- a/.github/workflows/post-commit.yml
+++ b/.github/workflows/post-commit.yml
@@ -22,8 +22,8 @@ jobs:
steps:
- uses: actions/cache@v2
with:
- path: ~/.m2/reposttory
- key: maven-dependencies
+ path: ~/.m2/repository
+ key: maven-repo-${{ hashFiles('**/pom.xml') }}
- run: |
sudo chmod 777 /mnt
git clone https://github.com/${GITHUB_REPOSITORY}.git /mnt/ozone
@@ -43,4 +43,4 @@ jobs:
#Never cache local artifacts
rm -rf ~/.m2/repository/org/apache/hadoop/hdds
rm -rf ~/.m2/repository/org/apache/hadoop/ozone
- if: always()
\ No newline at end of file
+ if: always()
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-commits-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-commits-help@hadoop.apache.org
[hadoop-ozone] 06/18: use top level check script
Posted by el...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
elek pushed a commit to branch acceptance-coverage
in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git
commit 41b2cc35201c576db07889a01c835f8384f5b639
Author: Elek Márton <el...@apache.org>
AuthorDate: Sat May 30 15:31:50 2020 +0200
use top level check script
---
.github/workflows/post-commit.yml | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/.github/workflows/post-commit.yml b/.github/workflows/post-commit.yml
index 6823032..d10d696 100644
--- a/.github/workflows/post-commit.yml
+++ b/.github/workflows/post-commit.yml
@@ -39,11 +39,11 @@ jobs:
- run: sudo pip install robotframework
- run: sudo mv ./hadoop-ozone/dist/target/ozone-* /mnt/ozone && sudo chown runner -R /mnt/ozone
- run: cd /mnt/ozone && mkdir .aws && sudo chown 1000 .aws
- - run: cd /mnt/ozone/compose && ./test-all.sh
+ - run: cd /mnt/ozone/compose && hadoop-ozone/dev-support/checks/acceptance.sh
env:
KEEP_IMAGE: false
- uses: actions/upload-artifact@master
if: always()
with:
name: acceptance
- path: /mnt/ozone/compose/result
+ path: /mnt/ozone/target/acceptance
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-commits-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-commits-help@hadoop.apache.org
[hadoop-ozone] 03/18: collect acceptnce test results when results
are combined
Posted by el...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
elek pushed a commit to branch acceptance-coverage
in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git
commit bfa9ef0963414277d6a6553b0d2e55165a1315f3
Author: Elek Márton <el...@apache.org>
AuthorDate: Fri May 29 12:00:50 2020 +0200
collect acceptnce test results when results are combined
---
hadoop-ozone/dist/src/main/compose/test-all.sh | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/hadoop-ozone/dist/src/main/compose/test-all.sh b/hadoop-ozone/dist/src/main/compose/test-all.sh
index 809123b..3673bdd 100755
--- a/hadoop-ozone/dist/src/main/compose/test-all.sh
+++ b/hadoop-ozone/dist/src/main/compose/test-all.sh
@@ -41,7 +41,7 @@ for test in $(find "$SCRIPT_DIR" -name test.sh | sort); do
echo "ERROR: Test execution of $(dirname "$test") is FAILED!!!!"
fi
RESULT_DIR="$(dirname "$test")/result"
- cp "$RESULT_DIR"/robot-*.xml "$RESULT_DIR"/docker-*.log "$RESULT_DIR"/*.out* "$ALL_RESULT_DIR"/
+ cp "$RESULT_DIR"/robot-*.xml "$RESULT_DIR"/docker-*.log "$RESULT_DIR"/*.out* "$RESULT_DIR/*.exec" "$ALL_RESULT_DIR"/
done
rebot -N "smoketests" -d "$SCRIPT_DIR/result" "$SCRIPT_DIR/result/robot-*.xml"
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-commits-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-commits-help@hadoop.apache.org
[hadoop-ozone] 04/18: fix jacoco test collection
Posted by el...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
elek pushed a commit to branch acceptance-coverage
in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git
commit c32ad42dd155d42e40b3c3cf4be814de9afdc6f0
Author: Elek Márton <el...@apache.org>
AuthorDate: Fri May 29 15:42:44 2020 +0200
fix jacoco test collection
---
hadoop-ozone/dev-support/checks/acceptance.sh | 4 ++--
hadoop-ozone/dist/src/main/compose/test-all.sh | 2 +-
2 files changed, 3 insertions(+), 3 deletions(-)
diff --git a/hadoop-ozone/dev-support/checks/acceptance.sh b/hadoop-ozone/dev-support/checks/acceptance.sh
index fc510f1..272d61f 100755
--- a/hadoop-ozone/dev-support/checks/acceptance.sh
+++ b/hadoop-ozone/dev-support/checks/acceptance.sh
@@ -34,7 +34,7 @@ cd "$DIST_DIR/compose" || exit 1
RES=$?
cp result/* "$REPORT_DIR/"
cp "$REPORT_DIR/log.html" "$REPORT_DIR/summary.html"
-
-"$DIST_DIR/share/jacoco/jacoco-cli.jar" merge $(find "$REPORT_DIR" -name "*.jacoco.exec") --destdir "$REPORT_DIR/jacoco-combined.exec"
+set -x
+java -jar "$DIST_DIR/share/jacoco/jacoco-cli.jar" merge $(find "$REPORT_DIR" -name "*.jacoco.exec") --destdir "$REPORT_DIR/jacoco-combined.exec"
exit $RES
diff --git a/hadoop-ozone/dist/src/main/compose/test-all.sh b/hadoop-ozone/dist/src/main/compose/test-all.sh
index 3673bdd..a84abb2 100755
--- a/hadoop-ozone/dist/src/main/compose/test-all.sh
+++ b/hadoop-ozone/dist/src/main/compose/test-all.sh
@@ -41,7 +41,7 @@ for test in $(find "$SCRIPT_DIR" -name test.sh | sort); do
echo "ERROR: Test execution of $(dirname "$test") is FAILED!!!!"
fi
RESULT_DIR="$(dirname "$test")/result"
- cp "$RESULT_DIR"/robot-*.xml "$RESULT_DIR"/docker-*.log "$RESULT_DIR"/*.out* "$RESULT_DIR/*.exec" "$ALL_RESULT_DIR"/
+ cp "$RESULT_DIR"/robot-*.xml "$RESULT_DIR"/docker-*.log "$RESULT_DIR"/*.out* "$RESULT_DIR"/*.exec "$ALL_RESULT_DIR"/
done
rebot -N "smoketests" -d "$SCRIPT_DIR/result" "$SCRIPT_DIR/result/robot-*.xml"
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-commits-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-commits-help@hadoop.apache.org
[hadoop-ozone] 08/18: fix checkout
Posted by el...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
elek pushed a commit to branch acceptance-coverage
in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git
commit 56a77752052da3418c625c83b315aebe971a50bf
Author: Elek Márton <el...@apache.org>
AuthorDate: Sat May 30 22:36:03 2020 +0200
fix checkout
---
.github/workflows/post-commit.yml | 2 +-
.../dist/src/main/compose/ozone-csi/test.sh | 33 -------------
.../src/main/compose/ozone-mr/hadoop31/test.sh | 45 -----------------
.../src/main/compose/ozone-mr/hadoop32/test.sh | 40 ----------------
.../dist/src/main/compose/ozone-om-ha-s3/test.sh | 36 --------------
.../dist/src/main/compose/ozone-topology/test.sh | 56 ----------------------
hadoop-ozone/dist/src/main/compose/ozone/test.sh | 14 ------
.../dist/src/main/compose/ozones3-haproxy/test.sh | 30 ------------
.../dist/src/main/compose/ozonesecure-mr/test.sh | 44 -----------------
.../src/main/compose/ozonesecure-om-ha/test.sh | 35 --------------
.../dist/src/main/compose/ozonesecure/test.sh | 46 ------------------
11 files changed, 1 insertion(+), 380 deletions(-)
diff --git a/.github/workflows/post-commit.yml b/.github/workflows/post-commit.yml
index e99a89e..eda2a2b 100644
--- a/.github/workflows/post-commit.yml
+++ b/.github/workflows/post-commit.yml
@@ -20,7 +20,7 @@ jobs:
name: acceptance
runs-on: ubuntu-18.04
steps:
- - runs: |
+ - run: |
sudo chmod 777 /mnt
git clone --depth=1 https://github.com/${GITHUB_REPOSITORY}.git /mnt/ozone
cd /mnt/ozone
diff --git a/hadoop-ozone/dist/src/main/compose/ozone-csi/test.sh b/hadoop-ozone/dist/src/main/compose/ozone-csi/test.sh
deleted file mode 100755
index 2a6e5f1..0000000
--- a/hadoop-ozone/dist/src/main/compose/ozone-csi/test.sh
+++ /dev/null
@@ -1,33 +0,0 @@
-#!/usr/bin/env bash
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements. See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership. The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-COMPOSE_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" >/dev/null 2>&1 && pwd)"
-export COMPOSE_DIR
-
-export SECURITY_ENABLED=false
-export OZONE_REPLICATION_FACTOR=3
-
-# shellcheck source=/dev/null
-source "$COMPOSE_DIR/../testlib.sh"
-
-start_docker_env
-
-execute_robot_test csi csi.robot
-
-stop_docker_env
-
-generate_report
diff --git a/hadoop-ozone/dist/src/main/compose/ozone-mr/hadoop31/test.sh b/hadoop-ozone/dist/src/main/compose/ozone-mr/hadoop31/test.sh
deleted file mode 100755
index 362ff98..0000000
--- a/hadoop-ozone/dist/src/main/compose/ozone-mr/hadoop31/test.sh
+++ /dev/null
@@ -1,45 +0,0 @@
-#!/usr/bin/env bash
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements. See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership. The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-COMPOSE_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
-export COMPOSE_DIR
-
-# shellcheck source=/dev/null
-source "$COMPOSE_DIR/.env"
-source "$COMPOSE_DIR/../../testlib.sh"
-
-start_docker_env
-
-#rm is the container name (resource manager) and not the rm command
-execute_command_in_container rm bash -c "if test -e /sbin/apk; then sudo -E apk add --update py-pip; sudo -E pip install robotframework; fi"
-
-execute_robot_test scm createmrenv.robot
-
-# reinitialize the directories to use
-export OZONE_DIR=/opt/ozone
-# shellcheck source=/dev/null
-source "$COMPOSE_DIR/../../testlib.sh"
-
-execute_robot_test rm ozonefs/hadoopo3fs.robot
-
-execute_robot_test rm -v "hadoop.version:${HADOOP_VERSION}" mapreduce.robot
-
-stop_docker_env
-
-generate_report
-
-cleanup_docker_images "${HADOOP_IMAGE}:${HADOOP_VERSION}"
diff --git a/hadoop-ozone/dist/src/main/compose/ozone-mr/hadoop32/test.sh b/hadoop-ozone/dist/src/main/compose/ozone-mr/hadoop32/test.sh
deleted file mode 100755
index b1910a5..0000000
--- a/hadoop-ozone/dist/src/main/compose/ozone-mr/hadoop32/test.sh
+++ /dev/null
@@ -1,40 +0,0 @@
-#!/usr/bin/env bash
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements. See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership. The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-COMPOSE_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
-export COMPOSE_DIR
-
-# shellcheck source=/dev/null
-source "$COMPOSE_DIR/../../testlib.sh"
-
-start_docker_env
-
-execute_robot_test scm createmrenv.robot
-
-# reinitialize the directories to use
-export OZONE_DIR=/opt/ozone
-
-# shellcheck source=/dev/null
-source "$COMPOSE_DIR/../../testlib.sh"
-
-execute_robot_test rm ozonefs/hadoopo3fs.robot
-
-execute_robot_test rm mapreduce.robot
-
-stop_docker_env
-
-generate_report
diff --git a/hadoop-ozone/dist/src/main/compose/ozone-om-ha-s3/test.sh b/hadoop-ozone/dist/src/main/compose/ozone-om-ha-s3/test.sh
deleted file mode 100644
index 719ad4c..0000000
--- a/hadoop-ozone/dist/src/main/compose/ozone-om-ha-s3/test.sh
+++ /dev/null
@@ -1,36 +0,0 @@
-#!/usr/bin/env bash
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements. See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership. The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-COMPOSE_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
-export COMPOSE_DIR
-
-export SECURITY_ENABLED=false
-export OM_SERVICE_ID="id1"
-
-# shellcheck source=/dev/null
-source "$COMPOSE_DIR/../testlib.sh"
-
-start_docker_env
-
-execute_robot_test scm s3
-
-execute_robot_test scm freon
-
-stop_docker_env
-
-generate_report
-
diff --git a/hadoop-ozone/dist/src/main/compose/ozone-topology/test.sh b/hadoop-ozone/dist/src/main/compose/ozone-topology/test.sh
deleted file mode 100755
index d4efa4f..0000000
--- a/hadoop-ozone/dist/src/main/compose/ozone-topology/test.sh
+++ /dev/null
@@ -1,56 +0,0 @@
-#!/usr/bin/env bash
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements. See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership. The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-COMPOSE_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
-export COMPOSE_DIR
-
-# shellcheck source=/dev/null
-source "$COMPOSE_DIR/../testlib.sh"
-
-start_docker_env 4
-
-#Due to the limitation of the current auditparser test, it should be the
-#first test in a clean cluster.
-
-#Disabling for now, audit parser tool during parse getting exception.
-#execute_robot_test om auditparser
-
-execute_robot_test scm basic/basic.robot
-
-execute_robot_test scm topology/cli.robot
-
-# Ensure data can be read even when a full rack
-# is stopped.
-execute_robot_test scm topology/loaddata.robot
-
-stop_containers datanode_1 datanode_2 datanode_3
-
-execute_robot_test scm topology/readdata.robot
-
-start_containers datanode_1 datanode_2 datanode_3
-
-wait_for_port datanode_1 9858 60
-wait_for_port datanode_2 9858 60
-wait_for_port datanode_3 9858 60
-
-stop_containers datanode_4 datanode_5 datanode_6
-
-execute_robot_test scm topology/readdata.robot
-
-stop_docker_env
-
-generate_report
diff --git a/hadoop-ozone/dist/src/main/compose/ozone/test.sh b/hadoop-ozone/dist/src/main/compose/ozone/test.sh
index b77bd55..f48175d 100644
--- a/hadoop-ozone/dist/src/main/compose/ozone/test.sh
+++ b/hadoop-ozone/dist/src/main/compose/ozone/test.sh
@@ -34,20 +34,6 @@ start_docker_env
execute_robot_test scm basic
-execute_robot_test scm gdpr
-
-execute_robot_test scm ozonefs/ozonefs.robot
-
-execute_robot_test scm security/ozone-secure-token.robot
-
-execute_robot_test scm s3
-
-execute_robot_test scm recon
-
-execute_robot_test scm om-ratis
-
-execute_robot_test scm freon
-
stop_docker_env
generate_report
diff --git a/hadoop-ozone/dist/src/main/compose/ozones3-haproxy/test.sh b/hadoop-ozone/dist/src/main/compose/ozones3-haproxy/test.sh
deleted file mode 100755
index f4bfcc3..0000000
--- a/hadoop-ozone/dist/src/main/compose/ozones3-haproxy/test.sh
+++ /dev/null
@@ -1,30 +0,0 @@
-#!/usr/bin/env bash
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements. See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership. The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-COMPOSE_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
-export COMPOSE_DIR
-
-# shellcheck source=/dev/null
-source "$COMPOSE_DIR/../testlib.sh"
-
-start_docker_env
-
-execute_robot_test scm basic/basic.robot
-
-stop_docker_env
-
-generate_report
diff --git a/hadoop-ozone/dist/src/main/compose/ozonesecure-mr/test.sh b/hadoop-ozone/dist/src/main/compose/ozonesecure-mr/test.sh
deleted file mode 100755
index cc6ebf0..0000000
--- a/hadoop-ozone/dist/src/main/compose/ozonesecure-mr/test.sh
+++ /dev/null
@@ -1,44 +0,0 @@
-#!/usr/bin/env bash
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements. See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership. The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-COMPOSE_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
-export COMPOSE_DIR
-
-# shellcheck source=/dev/null
-source "$COMPOSE_DIR/../testlib.sh"
-
-export SECURITY_ENABLED=true
-
-start_docker_env
-
-execute_robot_test om kinit.robot
-
-execute_robot_test om createmrenv.robot
-
-# reinitialize the directories to use
-export OZONE_DIR=/opt/ozone
-
-# shellcheck source=/dev/null
-source "$COMPOSE_DIR/../testlib.sh"
-
-execute_robot_test rm kinit-hadoop.robot
-
-execute_robot_test rm mapreduce.robot
-
-stop_docker_env
-
-generate_report
diff --git a/hadoop-ozone/dist/src/main/compose/ozonesecure-om-ha/test.sh b/hadoop-ozone/dist/src/main/compose/ozonesecure-om-ha/test.sh
deleted file mode 100755
index 8893ef6..0000000
--- a/hadoop-ozone/dist/src/main/compose/ozonesecure-om-ha/test.sh
+++ /dev/null
@@ -1,35 +0,0 @@
-#!/usr/bin/env bash
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements. See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership. The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-COMPOSE_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
-export COMPOSE_DIR
-
-export SECURITY_ENABLED=true
-export OM_SERVICE_ID="id1"
-
-# shellcheck source=/dev/null
-source "$COMPOSE_DIR/../testlib.sh"
-
-start_docker_env
-
-execute_robot_test scm kinit.robot
-
-execute_robot_test scm freon
-
-stop_docker_env
-
-generate_report
diff --git a/hadoop-ozone/dist/src/main/compose/ozonesecure/test.sh b/hadoop-ozone/dist/src/main/compose/ozonesecure/test.sh
deleted file mode 100755
index d1bdd0d..0000000
--- a/hadoop-ozone/dist/src/main/compose/ozonesecure/test.sh
+++ /dev/null
@@ -1,46 +0,0 @@
-#!/usr/bin/env bash
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements. See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership. The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-COMPOSE_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
-export COMPOSE_DIR
-
-# shellcheck source=/dev/null
-source "$COMPOSE_DIR/../testlib.sh"
-
-export SECURITY_ENABLED=true
-
-start_docker_env
-
-execute_robot_test scm kinit.robot
-
-execute_robot_test scm basic
-
-execute_robot_test scm security
-
-execute_robot_test scm ozonefs/ozonefs.robot
-
-execute_robot_test s3g s3
-
-execute_robot_test scm admincli
-
-execute_robot_test scm recon
-
-execute_robot_test scm spnego
-
-stop_docker_env
-
-generate_report
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-commits-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-commits-help@hadoop.apache.org
[hadoop-ozone] 02/18: merge the results to one file
Posted by el...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
elek pushed a commit to branch acceptance-coverage
in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git
commit a3d332e9231c45fe49692828d4a722997c60a2e9
Author: Elek Márton <el...@apache.org>
AuthorDate: Fri May 29 10:44:28 2020 +0200
merge the results to one file
---
hadoop-ozone/dev-support/checks/acceptance.sh | 3 +++
1 file changed, 3 insertions(+)
diff --git a/hadoop-ozone/dev-support/checks/acceptance.sh b/hadoop-ozone/dev-support/checks/acceptance.sh
index 30259c7..fc510f1 100755
--- a/hadoop-ozone/dev-support/checks/acceptance.sh
+++ b/hadoop-ozone/dev-support/checks/acceptance.sh
@@ -34,4 +34,7 @@ cd "$DIST_DIR/compose" || exit 1
RES=$?
cp result/* "$REPORT_DIR/"
cp "$REPORT_DIR/log.html" "$REPORT_DIR/summary.html"
+
+"$DIST_DIR/share/jacoco/jacoco-cli.jar" merge $(find "$REPORT_DIR" -name "*.jacoco.exec") --destdir "$REPORT_DIR/jacoco-combined.exec"
+
exit $RES
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-commits-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-commits-help@hadoop.apache.org
[hadoop-ozone] 09/18: fix checkout
Posted by el...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
elek pushed a commit to branch acceptance-coverage
in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git
commit fb313569d8fc4280f0341145bc18ce34dd65c259
Author: Elek Márton <el...@apache.org>
AuthorDate: Sat May 30 22:41:48 2020 +0200
fix checkout
---
.github/workflows/post-commit.yml | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/.github/workflows/post-commit.yml b/.github/workflows/post-commit.yml
index eda2a2b..b41906e 100644
--- a/.github/workflows/post-commit.yml
+++ b/.github/workflows/post-commit.yml
@@ -24,7 +24,7 @@ jobs:
sudo chmod 777 /mnt
git clone --depth=1 https://github.com/${GITHUB_REPOSITORY}.git /mnt/ozone
cd /mnt/ozone
- git checkout ${GITHUB_REF}
+ git checkout ${GITHUB_SHA}
git reset --hard
- run: sudo pip install robotframework
- run: cd /mnt/ozone && hadoop-ozone/dev-support/checks/acceptance.sh
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-commits-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-commits-help@hadoop.apache.org
[hadoop-ozone] 13/18: use cache
Posted by el...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
elek pushed a commit to branch acceptance-coverage
in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git
commit 6960836f1f1f15c5bf2c3f7e033b9de4925e7149
Author: Elek Márton <el...@apache.org>
AuthorDate: Wed Jun 3 14:17:18 2020 +0200
use cache
---
.github/workflows/post-commit.yml | 11 +++++++++--
1 file changed, 9 insertions(+), 2 deletions(-)
diff --git a/.github/workflows/post-commit.yml b/.github/workflows/post-commit.yml
index 220542e..436e105 100644
--- a/.github/workflows/post-commit.yml
+++ b/.github/workflows/post-commit.yml
@@ -20,9 +20,11 @@ jobs:
name: acceptance
runs-on: ubuntu-18.04
steps:
+ with:
+ - uses: actions/cache@v2
+ path: ~/.m2/repository
+ key: maven-dependencies
- run: |
- set -x
- echo $GITHUB_REPOSITORY
sudo chmod 777 /mnt
git clone https://github.com/${GITHUB_REPOSITORY}.git /mnt/ozone
cd /mnt/ozone
@@ -37,3 +39,8 @@ jobs:
with:
name: acceptance
path: /mnt/ozone/target/acceptance
+ - runs: |
+ #Never cache local artifacts
+ rm -rf ~/.m2/repository/org/apache/hadoop/hdds
+ rm -rf ~/.m2/repository/org/apache/hadoop/ozone
+ if: always()
\ No newline at end of file
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-commits-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-commits-help@hadoop.apache.org
[hadoop-ozone] 10/18: test
Posted by el...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
elek pushed a commit to branch acceptance-coverage
in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git
commit 089d167baa300e7627d36cf3cc3e3a285d7171e8
Author: Elek Márton <el...@apache.org>
AuthorDate: Sun May 31 10:59:10 2020 +0200
test
---
.github/workflows/post-commit.yml | 2 ++
1 file changed, 2 insertions(+)
diff --git a/.github/workflows/post-commit.yml b/.github/workflows/post-commit.yml
index b41906e..60a9e5d 100644
--- a/.github/workflows/post-commit.yml
+++ b/.github/workflows/post-commit.yml
@@ -21,6 +21,8 @@ jobs:
runs-on: ubuntu-18.04
steps:
- run: |
+ set -x
+ echo $GITHUB_REPOSITORY
sudo chmod 777 /mnt
git clone --depth=1 https://github.com/${GITHUB_REPOSITORY}.git /mnt/ozone
cd /mnt/ozone
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-commits-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-commits-help@hadoop.apache.org
[hadoop-ozone] 12/18: debug acceptance script
Posted by el...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
elek pushed a commit to branch acceptance-coverage
in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git
commit 8c261f4458f4a03a4a848834094ea739d8ff6df0
Author: Elek Márton <el...@apache.org>
AuthorDate: Wed Jun 3 14:08:35 2020 +0200
debug acceptance script
---
hadoop-ozone/dev-support/checks/acceptance.sh | 1 +
1 file changed, 1 insertion(+)
diff --git a/hadoop-ozone/dev-support/checks/acceptance.sh b/hadoop-ozone/dev-support/checks/acceptance.sh
index 9ec6c8b..c707907 100755
--- a/hadoop-ozone/dev-support/checks/acceptance.sh
+++ b/hadoop-ozone/dev-support/checks/acceptance.sh
@@ -13,6 +13,7 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
+set -x
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
cd "$DIR/../../.." || exit 1
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-commits-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-commits-help@hadoop.apache.org
[hadoop-ozone] 14/18: fix actions structure
Posted by el...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
elek pushed a commit to branch acceptance-coverage
in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git
commit facee0e3da5441021a77bd4679359dbe5a084314
Author: Elek Márton <el...@apache.org>
AuthorDate: Wed Jun 3 15:38:05 2020 +0200
fix actions structure
---
.github/workflows/post-commit.yml | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/.github/workflows/post-commit.yml b/.github/workflows/post-commit.yml
index 436e105..6a62890 100644
--- a/.github/workflows/post-commit.yml
+++ b/.github/workflows/post-commit.yml
@@ -20,10 +20,10 @@ jobs:
name: acceptance
runs-on: ubuntu-18.04
steps:
- with:
- uses: actions/cache@v2
- path: ~/.m2/repository
- key: maven-dependencies
+ with:
+ path: ~/.m2/reposttory
+ key: maven-dependencies
- run: |
sudo chmod 777 /mnt
git clone https://github.com/${GITHUB_REPOSITORY}.git /mnt/ozone
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-commits-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-commits-help@hadoop.apache.org
[hadoop-ozone] 05/18: Handle the case when only one acceptance test
is covered
Posted by el...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
elek pushed a commit to branch acceptance-coverage
in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git
commit 15873f25f7eccfdf40d7aeb93cb3053783412435
Author: Elek Márton <el...@apache.org>
AuthorDate: Fri May 29 19:10:26 2020 +0200
Handle the case when only one acceptance test is covered
---
hadoop-ozone/dev-support/checks/acceptance.sh | 10 +++++++++-
1 file changed, 9 insertions(+), 1 deletion(-)
diff --git a/hadoop-ozone/dev-support/checks/acceptance.sh b/hadoop-ozone/dev-support/checks/acceptance.sh
index 272d61f..9ec6c8b 100755
--- a/hadoop-ozone/dev-support/checks/acceptance.sh
+++ b/hadoop-ozone/dev-support/checks/acceptance.sh
@@ -35,6 +35,14 @@ RES=$?
cp result/* "$REPORT_DIR/"
cp "$REPORT_DIR/log.html" "$REPORT_DIR/summary.html"
set -x
-java -jar "$DIST_DIR/share/jacoco/jacoco-cli.jar" merge $(find "$REPORT_DIR" -name "*.jacoco.exec") --destdir "$REPORT_DIR/jacoco-combined.exec"
+
+EXEC_FILES=$(find "$REPORT_DIR" -name "*.jacoco.exec" | wc -l)
+if [[ $EXEC_FILES -gt 1 ]]; then
+ java -jar "$DIST_DIR/share/jacoco/jacoco-cli.jar" merge $(find "$REPORT_DIR" -name "*.jacoco.exec") --destdir "$REPORT_DIR/jacoco-combined.exec"
+fi
+
+if [[ $EXEC_FILES == 1 ]]; then
+ cp "$REPORT_DIR"/*.jacoco.exec "$REPORT_DIR/jacoco-combined.exec"
+fi
exit $RES
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-commits-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-commits-help@hadoop.apache.org
[hadoop-ozone] 01/18: collect coverage data from basic acceptance
tests
Posted by el...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
elek pushed a commit to branch acceptance-coverage
in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git
commit 1d744f2bec1cef359f3408e8c4def124bb8bea03
Author: Elek Márton <el...@apache.org>
AuthorDate: Thu May 28 16:00:15 2020 +0200
collect coverage data from basic acceptance tests
---
.github/workflows/post-commit.yml | 180 ---------------------
hadoop-ozone/dev-support/checks/acceptance.sh | 4 +-
hadoop-ozone/dev-support/checks/build.sh | 2 +-
hadoop-ozone/dist/pom.xml | 42 +++++
.../src/main/compose/ozone/docker-compose.yaml | 5 +
hadoop-ozone/dist/src/main/compose/ozone/test.sh | 0
hadoop-ozone/dist/src/main/compose/testlib.sh | 2 +
7 files changed, 53 insertions(+), 182 deletions(-)
diff --git a/.github/workflows/post-commit.yml b/.github/workflows/post-commit.yml
index 5a920e5..6823032 100644
--- a/.github/workflows/post-commit.yml
+++ b/.github/workflows/post-commit.yml
@@ -24,78 +24,6 @@ jobs:
- uses: ./.github/buildenv
with:
args: ./hadoop-ozone/dev-support/checks/build.sh
- rat:
- name: rat
- runs-on: ubuntu-18.04
- steps:
- - uses: actions/checkout@master
- - uses: ./.github/buildenv
- with:
- args: ./hadoop-ozone/dev-support/checks/rat.sh
- - uses: actions/upload-artifact@master
- if: always()
- with:
- name: rat
- path: target/rat
- author:
- name: author
- runs-on: ubuntu-18.04
- steps:
- - uses: actions/checkout@master
- - uses: ./.github/buildenv
- with:
- args: ./hadoop-ozone/dev-support/checks/author.sh
- - uses: actions/upload-artifact@master
- if: always()
- with:
- name: author
- path: target/author
- unit:
- name: unit
- runs-on: ubuntu-18.04
- steps:
- - uses: actions/checkout@master
- - uses: ./.github/buildenv
- with:
- args: ./hadoop-ozone/dev-support/checks/unit.sh
- - uses: ./.github/buildenv
- if: github.repository == 'apache/hadoop-ozone'
- with:
- args: ./hadoop-ozone/dev-support/checks/sonar.sh
- env:
- SONAR_TOKEN: ${{ secrets.SONARCLOUD_TOKEN }}
- GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- - uses: actions/upload-artifact@master
- if: always()
- with:
- name: unit
- path: target/unit
- checkstyle:
- name: checkstyle
- runs-on: ubuntu-18.04
- steps:
- - uses: actions/checkout@master
- - uses: ./.github/buildenv
- with:
- args: ./hadoop-ozone/dev-support/checks/checkstyle.sh
- - uses: actions/upload-artifact@master
- if: always()
- with:
- name: checkstyle
- path: target/checkstyle
- findbugs:
- name: findbugs
- runs-on: ubuntu-18.04
- steps:
- - uses: actions/checkout@master
- - uses: ./.github/buildenv
- with:
- args: ./hadoop-ozone/dev-support/checks/findbugs.sh
- - uses: actions/upload-artifact@master
- if: always()
- with:
- name: findbugs
- path: target/findbugs
acceptance:
name: acceptance
runs-on: ubuntu-18.04
@@ -119,111 +47,3 @@ jobs:
with:
name: acceptance
path: /mnt/ozone/compose/result
- it-freon:
- name: it-freon
- runs-on: ubuntu-18.04
- needs:
- - build
- steps:
- - run: sudo mkdir mnt && sudo mount --bind /mnt `pwd`/mnt && sudo chmod 777 mnt
- - uses: actions/checkout@master
- with:
- path: mnt/ozone
- - uses: ./mnt/ozone/.github/buildenv
- with:
- args: ./mnt/ozone/hadoop-ozone/dev-support/checks/integration.sh -Pfreon
- - uses: actions/upload-artifact@master
- if: always()
- with:
- name: it-freon
- path: mnt/ozone/target/integration
- it-filesystem:
- name: it-filesystem
- runs-on: ubuntu-18.04
- needs:
- - build
- steps:
- - run: sudo mkdir mnt && sudo mount --bind /mnt `pwd`/mnt && sudo chmod 777 mnt
- - uses: actions/checkout@master
- with:
- path: mnt/ozone
- - uses: ./mnt/ozone/.github/buildenv
- with:
- args: ./mnt/ozone/hadoop-ozone/dev-support/checks/integration.sh -Pfilesystem
- - uses: actions/upload-artifact@master
- if: always()
- with:
- name: it-filesystem
- path: mnt/ozone/target/integration
- it-filesystem-contract:
- name: it-filesystem-contract
- runs-on: ubuntu-18.04
- needs:
- - build
- steps:
- - run: sudo mkdir mnt && sudo mount --bind /mnt `pwd`/mnt && sudo chmod 777 mnt
- - uses: actions/checkout@master
- with:
- path: mnt/ozone
- - uses: ./mnt/ozone/.github/buildenv
- with:
- args: ./mnt/ozone/hadoop-ozone/dev-support/checks/integration.sh -Pfilesystem-contract
- - uses: actions/upload-artifact@master
- if: always()
- with:
- name: it-filesystem-contract
- path: mnt/ozone/target/integration
- it-client:
- name: it-client
- runs-on: ubuntu-18.04
- needs:
- - build
- steps:
- - run: sudo mkdir mnt && sudo mount --bind /mnt `pwd`/mnt && sudo chmod 777 mnt
- - uses: actions/checkout@master
- with:
- path: mnt/ozone
- - uses: ./mnt/ozone/.github/buildenv
- with:
- args: ./mnt/ozone/hadoop-ozone/dev-support/checks/integration.sh -Pclient
- - uses: actions/upload-artifact@master
- if: always()
- with:
- name: it-client
- path: mnt/ozone/target/integration
- it-hdds-om:
- name: it-hdds-om
- runs-on: ubuntu-18.04
- needs:
- - build
- steps:
- - run: sudo mkdir mnt && sudo mount --bind /mnt `pwd`/mnt && sudo chmod 777 mnt
- - uses: actions/checkout@master
- with:
- path: mnt/ozone
- - uses: ./mnt/ozone/.github/buildenv
- with:
- args: ./mnt/ozone/hadoop-ozone/dev-support/checks/integration.sh -Phdds-om
- - uses: actions/upload-artifact@master
- if: always()
- with:
- name: it-hdds-om
- path: mnt/ozone/target/integration
- it-ozone:
- name: it-ozone
- runs-on: ubuntu-18.04
- needs:
- - build
- steps:
- - run: sudo mkdir mnt && sudo mount --bind /mnt `pwd`/mnt && sudo chmod 777 mnt
- - uses: actions/checkout@master
- with:
- path: mnt/ozone
- - uses: ./mnt/ozone/.github/buildenv
- with:
- args: ./mnt/ozone/hadoop-ozone/dev-support/checks/integration.sh -Pozone
- - uses: actions/upload-artifact@master
- if: always()
- with:
- name: it-ozone
- path: mnt/ozone/target/integration
\ No newline at end of file
diff --git a/hadoop-ozone/dev-support/checks/acceptance.sh b/hadoop-ozone/dev-support/checks/acceptance.sh
index c3c1aa0..30259c7 100755
--- a/hadoop-ozone/dev-support/checks/acceptance.sh
+++ b/hadoop-ozone/dev-support/checks/acceptance.sh
@@ -24,9 +24,11 @@ DIST_DIR="$DIR/../../dist/target/ozone-$OZONE_VERSION"
if [ ! -d "$DIST_DIR" ]; then
echo "Distribution dir is missing. Doing a full build"
- "$DIR/build.sh"
+ "$DIR/build.sh" -Pjacoco
fi
+export HADOOP_OPTS='-javaagent:/opt/hadoop/share/jacoco/jacoco-agent.jar=destfile=/tmp/jacoco.exec,includes=org.apache.hadoop.ozone.*:org.apache.hadoop.hdds'
+
cd "$DIST_DIR/compose" || exit 1
./test-all.sh
RES=$?
diff --git a/hadoop-ozone/dev-support/checks/build.sh b/hadoop-ozone/dev-support/checks/build.sh
index f461da1..2cdc4fe 100755
--- a/hadoop-ozone/dev-support/checks/build.sh
+++ b/hadoop-ozone/dev-support/checks/build.sh
@@ -17,5 +17,5 @@ DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
cd "$DIR/../../.." || exit 1
export MAVEN_OPTS="-Xmx4096m"
-mvn -B -Dmaven.javadoc.skip=true -DskipTests clean install
+mvn -B -Dmaven.javadoc.skip=true -DskipTests clean install "$@"
exit $?
diff --git a/hadoop-ozone/dist/pom.xml b/hadoop-ozone/dist/pom.xml
index 896831b..752a8c5 100644
--- a/hadoop-ozone/dist/pom.xml
+++ b/hadoop-ozone/dist/pom.xml
@@ -379,7 +379,49 @@
<artifactId>hadoop-ozone-recon</artifactId>
</dependency>
</dependencies>
+ </profile>
+ <profile>
+ <id>jacoco</id>
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-dependency-plugin</artifactId>
+ <version>3.1.1</version>
+ <executions>
+ <execution>
+ <id>copy-jacoco-files</id>
+ <phase>prepare-package</phase>
+ <goals>
+ <goal>copy</goal>
+ </goals>
+ <configuration>
+ <outputDirectory>
+ target/ozone-${ozone.version}/share/jacoco
+ </outputDirectory>
+ <artifactItems>
+ <artifactItem>
+ <groupId>org.jacoco</groupId>
+ <artifactId>org.jacoco.agent</artifactId>
+ <version>0.8.5</version>
+ <classifier>runtime</classifier>
+ <destFileName>jacoco-agent.jar</destFileName>
+ </artifactItem>
+ <artifactItem>
+ <groupId>org.jacoco</groupId>
+ <artifactId>org.jacoco.cli</artifactId>
+ <version>0.8.5</version>
+ <classifier>nodeps</classifier>
+ <destFileName>jacoco-cli.jar</destFileName>
+ </artifactItem>
+ </artifactItems>
+ </configuration>
+ </execution>
+ </executions>
+ </plugin>
+ </plugins>
+ </build>
</profile>
<profile>
diff --git a/hadoop-ozone/dist/src/main/compose/ozone/docker-compose.yaml b/hadoop-ozone/dist/src/main/compose/ozone/docker-compose.yaml
index 869aa05..2597612 100644
--- a/hadoop-ozone/dist/src/main/compose/ozone/docker-compose.yaml
+++ b/hadoop-ozone/dist/src/main/compose/ozone/docker-compose.yaml
@@ -37,11 +37,13 @@ services:
- 9882
environment:
<<: *replication
+ HADOOP_OPTS: ${HADOOP_OPTS}
command: ["ozone","datanode"]
om:
<<: *common-config
environment:
ENSURE_OM_INITIALIZED: /data/metadata/om/current/VERSION
+ HADOOP_OPTS: ${HADOOP_OPTS}
<<: *replication
ports:
- 9874:9874
@@ -54,11 +56,13 @@ services:
environment:
ENSURE_SCM_INITIALIZED: /data/metadata/scm/current/VERSION
OZONE-SITE.XML_hdds.scm.safemode.min.datanode: ${OZONE_SAFEMODE_MIN_DATANODES:-1}
+ HADOOP_OPTS: ${HADOOP_OPTS}
<<: *replication
command: ["ozone","scm"]
s3g:
<<: *common-config
environment:
+ HADOOP_OPTS: ${HADOOP_OPTS}
<<: *replication
ports:
- 9878:9878
@@ -68,5 +72,6 @@ services:
ports:
- 9888:9888
environment:
+ HADOOP_OPTS: ${HADOOP_OPTS}
<<: *replication
command: ["ozone","recon"]
diff --git a/hadoop-ozone/dist/src/main/compose/ozone/test.sh b/hadoop-ozone/dist/src/main/compose/ozone/test.sh
old mode 100755
new mode 100644
diff --git a/hadoop-ozone/dist/src/main/compose/testlib.sh b/hadoop-ozone/dist/src/main/compose/testlib.sh
old mode 100755
new mode 100644
index 064b19d..5948ac9
--- a/hadoop-ozone/dist/src/main/compose/testlib.sh
+++ b/hadoop-ozone/dist/src/main/compose/testlib.sh
@@ -108,7 +108,9 @@ execute_robot_test(){
local -i rc=$?
FULL_CONTAINER_NAME=$(docker-compose -f "$COMPOSE_FILE" ps | grep "_${CONTAINER}_" | head -n 1 | awk '{print $1}')
+
docker cp "$FULL_CONTAINER_NAME:$OUTPUT_PATH" "$RESULT_DIR/"
+ docker cp "$FULL_CONTAINER_NAME:/tmp/jacoco.exec" "$RESULT_DIR/$OUTPUT_NAME.jacoco.exec" || true
copy_daemon_logs
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-commits-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-commits-help@hadoop.apache.org
[hadoop-ozone] 18/18: retrigger ci
Posted by el...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
elek pushed a commit to branch acceptance-coverage
in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git
commit 8ef19b2415da2b4941ee126db2a6c36e89b0f06d
Author: Elek Márton <el...@apache.org>
AuthorDate: Thu Jun 4 14:00:23 2020 +0200
retrigger ci
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-commits-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-commits-help@hadoop.apache.org
[hadoop-ozone] 11/18: do full clone
Posted by el...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
elek pushed a commit to branch acceptance-coverage
in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git
commit 9b00d1bd93db03b580500c2286f89cf8b72526dd
Author: Elek Márton <el...@apache.org>
AuthorDate: Sun May 31 11:02:11 2020 +0200
do full clone
---
.github/workflows/post-commit.yml | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/.github/workflows/post-commit.yml b/.github/workflows/post-commit.yml
index 60a9e5d..220542e 100644
--- a/.github/workflows/post-commit.yml
+++ b/.github/workflows/post-commit.yml
@@ -24,7 +24,7 @@ jobs:
set -x
echo $GITHUB_REPOSITORY
sudo chmod 777 /mnt
- git clone --depth=1 https://github.com/${GITHUB_REPOSITORY}.git /mnt/ozone
+ git clone https://github.com/${GITHUB_REPOSITORY}.git /mnt/ozone
cd /mnt/ozone
git checkout ${GITHUB_SHA}
git reset --hard
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-commits-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-commits-help@hadoop.apache.org
[hadoop-ozone] 16/18: fix the location of the target dir creation
Posted by el...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
elek pushed a commit to branch acceptance-coverage
in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git
commit ccddc8b235a95feb25a14d0e337fb9b5f2c12347
Author: Elek Márton <el...@apache.org>
AuthorDate: Wed Jun 3 18:28:38 2020 +0200
fix the location of the target dir creation
---
hadoop-ozone/dev-support/checks/acceptance.sh | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/hadoop-ozone/dev-support/checks/acceptance.sh b/hadoop-ozone/dev-support/checks/acceptance.sh
index c707907..49b825b 100755
--- a/hadoop-ozone/dev-support/checks/acceptance.sh
+++ b/hadoop-ozone/dev-support/checks/acceptance.sh
@@ -18,7 +18,6 @@ DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
cd "$DIR/../../.." || exit 1
REPORT_DIR=${OUTPUT_DIR:-"$DIR/../../../target/acceptance"}
-mkdir -p "$REPORT_DIR"
OZONE_VERSION=$(grep "<ozone.version>" "pom.xml" | sed 's/<[^>]*>//g'| sed 's/^[ \t]*//')
DIST_DIR="$DIR/../../dist/target/ozone-$OZONE_VERSION"
@@ -28,6 +27,8 @@ if [ ! -d "$DIST_DIR" ]; then
"$DIR/build.sh" -Pjacoco
fi
+mkdir -p "$REPORT_DIR"
+
export HADOOP_OPTS='-javaagent:/opt/hadoop/share/jacoco/jacoco-agent.jar=destfile=/tmp/jacoco.exec,includes=org.apache.hadoop.ozone.*:org.apache.hadoop.hdds'
cd "$DIST_DIR/compose" || exit 1
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-commits-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-commits-help@hadoop.apache.org
[hadoop-ozone] 15/18: I said run not runs
Posted by el...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
elek pushed a commit to branch acceptance-coverage
in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git
commit 857181d295c370fd90459ea85189394fafe0fd53
Author: Elek Márton <el...@apache.org>
AuthorDate: Wed Jun 3 16:23:13 2020 +0200
I said run not runs
---
.github/workflows/post-commit.yml | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/.github/workflows/post-commit.yml b/.github/workflows/post-commit.yml
index 6a62890..f05a9f8 100644
--- a/.github/workflows/post-commit.yml
+++ b/.github/workflows/post-commit.yml
@@ -39,7 +39,7 @@ jobs:
with:
name: acceptance
path: /mnt/ozone/target/acceptance
- - runs: |
+ - run: |
#Never cache local artifacts
rm -rf ~/.m2/repository/org/apache/hadoop/hdds
rm -rf ~/.m2/repository/org/apache/hadoop/ozone
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-commits-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-commits-help@hadoop.apache.org
[hadoop-ozone] 07/18: simplified acceptance test
Posted by el...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
elek pushed a commit to branch acceptance-coverage
in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git
commit ad874f9eb5eeb6b6ed30a60b6d2f3fc2f6a65981
Author: Elek Márton <el...@apache.org>
AuthorDate: Sat May 30 15:42:07 2020 +0200
simplified acceptance test
---
.github/workflows/post-commit.yml | 26 +++++++-------------------
1 file changed, 7 insertions(+), 19 deletions(-)
diff --git a/.github/workflows/post-commit.yml b/.github/workflows/post-commit.yml
index d10d696..e99a89e 100644
--- a/.github/workflows/post-commit.yml
+++ b/.github/workflows/post-commit.yml
@@ -16,30 +16,18 @@ name: build-branch
on:
- push
jobs:
- build:
- name: compile
- runs-on: ubuntu-18.04
- steps:
- - uses: actions/checkout@master
- - uses: ./.github/buildenv
- with:
- args: ./hadoop-ozone/dev-support/checks/build.sh
acceptance:
name: acceptance
runs-on: ubuntu-18.04
steps:
- - uses: actions/checkout@master
- - uses: ./.github/buildenv
- with:
- args: ./hadoop-ozone/dev-support/checks/build.sh
- # remove image created for 'buildenv'
- - run: docker image rm $(docker images -a -q | head -1) || true
- # remove its big parent build image
- - run: docker image rm apache/ozone-build || true
+ - runs: |
+ sudo chmod 777 /mnt
+ git clone --depth=1 https://github.com/${GITHUB_REPOSITORY}.git /mnt/ozone
+ cd /mnt/ozone
+ git checkout ${GITHUB_REF}
+ git reset --hard
- run: sudo pip install robotframework
- - run: sudo mv ./hadoop-ozone/dist/target/ozone-* /mnt/ozone && sudo chown runner -R /mnt/ozone
- - run: cd /mnt/ozone && mkdir .aws && sudo chown 1000 .aws
- - run: cd /mnt/ozone/compose && hadoop-ozone/dev-support/checks/acceptance.sh
+ - run: cd /mnt/ozone && hadoop-ozone/dev-support/checks/acceptance.sh
env:
KEEP_IMAGE: false
- uses: actions/upload-artifact@master
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-commits-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-commits-help@hadoop.apache.org