You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by GitBox <gi...@apache.org> on 2021/03/11 20:27:34 UTC
[GitHub] [ozone] adoroszlai opened a new pull request #2029: HDDS-4969. Extract check dependency installation from Github Actions workflow
adoroszlai opened a new pull request #2029:
URL: https://github.com/apache/ozone/pull/2029
## What changes were proposed in this pull request?
Move installation of external dependencies for some CI checks from Github Actions workflow to the script of the checks. This reduces complexity of Github-specific code in the workflow definition, and allows simpler prototyping/testing.
https://issues.apache.org/jira/browse/HDDS-4969
## How was this patch tested?
https://github.com/adoroszlai/hadoop-ozone/actions/runs/643852771
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] elek commented on a change in pull request #2029: HDDS-4969. Extract check dependency installation from Github Actions workflow
Posted by GitBox <gi...@apache.org>.
elek commented on a change in pull request #2029:
URL: https://github.com/apache/ozone/pull/2029#discussion_r598530527
##########
File path: hadoop-ozone/dev-support/checks/kubernetes.sh
##########
@@ -13,9 +13,37 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
+
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
cd "$DIR/../../.." || exit 1
+if [[ "$(uname -s)" = "Darwin" ]]; then
+ echo "k3s is not supported on Mac" >&2
+ exit 1
+fi
+
+export KUBECONFIG=/etc/rancher/k3s/k3s.yaml
Review comment:
```suggestion
: ${KUBECONFIG:=/etc/rancher/k3s/k3s.yaml}
```
Just to support minikube and other users....
##########
File path: hadoop-ozone/dev-support/checks/bats.sh
##########
@@ -13,9 +13,18 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
+
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
cd "${DIR}/../../.." || exit 1
+_install_tool_callback() {
+ curl -LSs https://github.com/bats-core/bats-core/archive/v1.2.1.tar.gz | tar xzf -
+}
+
+source "${DIR}/_lib.sh"
+install_tool
+export PATH="${PATH}:${TOOL_DIR}/bats-core-1.2.1/bin"
Review comment:
I like this one (my local install always has higher priority) but in this case if I have bats on my own PATH, we don't need to download the new one...
I don't know what is the best approach, but we may need to skip the download/install of the binaries if they are already available...
Maybe we can switch to a tool based directory hierarchy (`.dev-tools/k3s/...`) instead of check based hierarchy (`.dev-tools/acceptance/...`). It would also help us to reuse tools if they are used from multiple checks. (but with matrix build it may not be required any more...).
Sg. like `_install_tool <binary_name> <callback>` but you may have a better idea for a good abstraction....
##########
File path: hadoop-ozone/dev-support/checks/kubernetes.sh
##########
@@ -13,9 +13,37 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
+
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
cd "$DIR/../../.." || exit 1
+if [[ "$(uname -s)" = "Darwin" ]]; then
+ echo "k3s is not supported on Mac" >&2
+ exit 1
+fi
+
+export KUBECONFIG=/etc/rancher/k3s/k3s.yaml
+
+_install_tool_callback() {
+ # Install k3s
+ curl -sfL https://get.k3s.io | sh -
+
+ sudo chmod a+r $KUBECONFIG
+
+ # Install flekszible
+ wget https://github.com/elek/flekszible/releases/download/v1.8.1/flekszible_1.8.1_Linux_x86_64.tar.gz -O - | tar -zx
+ chmod +x flekszible
+ mkdir -p ${TOOL_DIR}/bin
+ mv -iv flekszible ${TOOL_DIR}/bin/
+
+ # Install robotframework
+ sudo pip install robotframework
Review comment:
You can use something like this to avoid sudo:
```
virtualenv "${TOOL_DIR}"/robotenv
source "${TOOL_DIR}"/robotenv/bin/activate
pip install robotframework
```
and you can add `"${TOOL_DIR}"/robotenv/bin` to the path
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] adoroszlai commented on pull request #2029: HDDS-4969. Extract check dependency installation from Github Actions workflow
Posted by GitBox <gi...@apache.org>.
adoroszlai commented on pull request #2029:
URL: https://github.com/apache/ozone/pull/2029#issuecomment-815755397
Thanks @elek for reviewing and committing it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] elek merged pull request #2029: HDDS-4969. Extract check dependency installation from Github Actions workflow
Posted by GitBox <gi...@apache.org>.
elek merged pull request #2029:
URL: https://github.com/apache/ozone/pull/2029
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] elek commented on pull request #2029: HDDS-4969. Extract check dependency installation from Github Actions workflow
Posted by GitBox <gi...@apache.org>.
elek commented on pull request #2029:
URL: https://github.com/apache/ozone/pull/2029#issuecomment-803902819
One more comment: we may need to add `.dev-tools` to `.gitignore`...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] adoroszlai commented on pull request #2029: HDDS-4969. Extract check dependency installation from Github Actions workflow
Posted by GitBox <gi...@apache.org>.
adoroszlai commented on pull request #2029:
URL: https://github.com/apache/ozone/pull/2029#issuecomment-797027981
@elek this is based on your idea from #1984. Installation of spotbugs is not included here because it is still run inside the builder image, which already has the tool.
If this is approved, I'll rebase the other PR.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org