You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "afilpp (via GitHub)" <gi...@apache.org> on 2024/03/29 10:16:01 UTC
[PR] HDDS-10612. Add Robot test to verify Container Balancer for RATIS containers [ozone]
afilpp opened a new pull request, #6457:
URL: https://github.com/apache/ozone/pull/6457
## What changes were proposed in this pull request?
HDDS-10612. Add Robot test to verify Container Balancer for RATIS containers
Currently there are only unit tests for Container Balancer and no acceptance tests at all. At a minimum, we should add a Robot test to verify Container Balancer for RATIS containers. And probably in the future we should add robot test for EC case.
Test case:
1. Move 1 datanode to maintenance mode (we use 4 datanodes in this test)
2. Create multiple keys (after loading the data, we check that 3 datanodes are ~60% busy, and the one that is in maintenance mode is empty)
4. Start datanode recommission (wait until datanode recommissioning is completed)
5. Start container balancer (wait until container balancer is completed)
6. Check results (after balancing on all 4 datanodes, we should see approximately the same data distribution.)
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-10612
## How was this patch tested?
Added Robot test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-10612. Add Robot test to verify Container Balancer for RATIS containers [ozone]
Posted by "afilpp (via GitHub)" <gi...@apache.org>.
afilpp commented on code in PR #6457:
URL: https://github.com/apache/ozone/pull/6457#discussion_r1546251061
##########
hadoop-ozone/dist/src/main/smoketest/balancer/testBalancer.robot:
##########
@@ -0,0 +1,141 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+*** Settings ***
+Documentation Smoketest ozone cluster startup
+Library OperatingSystem
+Library Collections
+Resource ../commonlib.robot
+Resource ../ozone-lib/shell.robot
+
+Test Timeout 20 minutes
+
+*** Variables ***
+${SECURITY_ENABLED} false
+${HOST} datanode1
+${VOLUME} volume1
+${BUCKET} bucket1
+${SIZE} 104857600
+
+
+** Keywords ***
+Prepare For Tests
+ Execute dd if=/dev/urandom of=/tmp/100mb bs=1048576 count=100
+ Run Keyword if '${SECURITY_ENABLED}' == 'true' Kinit test user testuser testuser.keytab
+ Execute ozone sh volume create /${VOLUME}
+ Execute ozone sh bucket create /${VOLUME}/${BUCKET}
+
+
+Datanode In Maintenance Mode
+ ${result} = Execute /opt/hadoop/bin/ozone admin datanode maintenance ${HOST}
+ Should Contain ${result} Entering maintenance mode on datanode
+ ${result} = Execute /opt/hadoop/bin/ozone admin datanode list | grep "Operational State:*"
+ Wait Until Keyword Succeeds 30sec 5sec Should contain ${result} ENTERING_MAINTENANCE
+ Wait Until Keyword Succeeds 3min 10sec Related pipelines are closed
+ Sleep 60000ms
+
+Related pipelines are closed
+ ${result} = Execute /opt/hadoop/bin/ozone admin datanode list | awk -v RS= '{$1=$1}1'|grep MAINT | sed -e 's/^.*pipelines: \\(.*\\)$/\\1/'
+ Should Contain Any ${result} CLOSED No related pipelines or the node is not in Healthy state.
+
+Datanode Recommission
+ ${result} = Execute /opt/hadoop/bin/ozone admin datanode recommission ${HOST}
+ Should Contain ${result} Started recommissioning datanode
+ Wait Until Keyword Succeeds 1min 10sec Datanode Recommission is Finished
+ Sleep 300000ms
+
+Datanode Recommission is Finished
+ ${result} = Execute /opt/hadoop/bin/ozone admin datanode list | grep "Operational State:*"
+ Should Not Contain ${result} ENTERING_MAINTENANCE
+
+Container Balancer
Review Comment:
Done)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-10612. Add Robot test to verify Container Balancer for RATIS containers [ozone]
Posted by "afilpp (via GitHub)" <gi...@apache.org>.
afilpp commented on code in PR #6457:
URL: https://github.com/apache/ozone/pull/6457#discussion_r1546250070
##########
hadoop-ozone/dist/src/main/compose/ozone-balancer/docker-compose.yaml:
##########
@@ -0,0 +1,183 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+version: "3.8"
+
+# reusable fragments (see https://docs.docker.com/compose/compose-file/#extension-fields)
+x-common-config:
+ &common-config
+ image: ${OZONE_RUNNER_IMAGE}:${OZONE_RUNNER_VERSION}
+ volumes:
+ - ../..:/opt/hadoop
+ env_file:
+ - docker-config
+
+x-replication:
+ &replication
+ OZONE-SITE.XML_ozone.server.default.replication: ${OZONE_REPLICATION_FACTOR:-3}
+
+services:
+ datanode1:
+ <<: *common-config
+ ports:
+ - 19864
+ - 9882
+ environment:
+ <<: *replication
+ OZONE_OPTS: -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=*:5005
Review Comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-10612. Add Robot test to verify Container Balancer for RATIS containers [ozone]
Posted by "ivandika3 (via GitHub)" <gi...@apache.org>.
ivandika3 commented on PR #6457:
URL: https://github.com/apache/ozone/pull/6457#issuecomment-2036095724
Seems there is a intermittent failure on the acceptance test
https://github.com/apache/ozone/actions/runs/8546730074/job/23418032793
@afilpp Could you take a look?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-10612. Add Robot test to verify Container Balancer for RATIS containers [ozone]
Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai commented on code in PR #6457:
URL: https://github.com/apache/ozone/pull/6457#discussion_r1546064122
##########
hadoop-ozone/dist/src/main/compose/ozone-balancer/docker-compose.yaml:
##########
@@ -0,0 +1,183 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+version: "3.8"
+
+# reusable fragments (see https://docs.docker.com/compose/compose-file/#extension-fields)
+x-common-config:
+ &common-config
+ image: ${OZONE_RUNNER_IMAGE}:${OZONE_RUNNER_VERSION}
+ volumes:
+ - ../..:/opt/hadoop
+ env_file:
+ - docker-config
+
+x-replication:
+ &replication
+ OZONE-SITE.XML_ozone.server.default.replication: ${OZONE_REPLICATION_FACTOR:-3}
+
+services:
+ datanode1:
+ <<: *common-config
+ ports:
+ - 19864
+ - 9882
+ environment:
+ <<: *replication
+ OZONE_OPTS: -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=*:5005
Review Comment:
Leaving it empty like this:
https://github.com/apache/ozone/blob/83d75861b0266160b219acde72d769eb0f9d5ac4/hadoop-ozone/dist/src/main/compose/ozone/docker-compose.yaml#L46
allows Docker to pass it through from the environment. This way you can set any value locally instead of hard-coding it:
```bash
export OZONE_OPTS='-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=*:5005'
docker-compose up -d
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-10612. Add Robot test to verify Container Balancer for RATIS containers [ozone]
Posted by "afilpp (via GitHub)" <gi...@apache.org>.
afilpp commented on code in PR #6457:
URL: https://github.com/apache/ozone/pull/6457#discussion_r1546250654
##########
hadoop-ozone/dist/src/main/compose/ozone-balancer/test.sh:
##########
@@ -0,0 +1,37 @@
+#!/usr/bin/env bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+#suite:balancer
+
+COMPOSE_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
+export COMPOSE_DIR
+export OM_SERVICE_ID="om"
+export OM=om1
+export SCM=scm1
+export OZONE_REPLICATION_FACTOR=3
+
+# shellcheck source=/dev/null
+source "$COMPOSE_DIR/../testlib.sh"
+
+# We need 4 dataNodes in this tests
+start_docker_env 4
+
+# Start OMs separately. In this test, the OMs will be stopped and restarted multiple times.
+# So we do not want the container to be tied to the OM process.
+#startOMs
Review Comment:
Fixed
##########
hadoop-ozone/dist/src/main/smoketest/balancer/testBalancer.robot:
##########
@@ -0,0 +1,141 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+*** Settings ***
+Documentation Smoketest ozone cluster startup
+Library OperatingSystem
+Library Collections
+Resource ../commonlib.robot
+Resource ../ozone-lib/shell.robot
+
+Test Timeout 20 minutes
+
+*** Variables ***
+${SECURITY_ENABLED} false
+${HOST} datanode1
+${VOLUME} volume1
+${BUCKET} bucket1
+${SIZE} 104857600
+
+
+** Keywords ***
+Prepare For Tests
+ Execute dd if=/dev/urandom of=/tmp/100mb bs=1048576 count=100
+ Run Keyword if '${SECURITY_ENABLED}' == 'true' Kinit test user testuser testuser.keytab
+ Execute ozone sh volume create /${VOLUME}
+ Execute ozone sh bucket create /${VOLUME}/${BUCKET}
+
+
+Datanode In Maintenance Mode
+ ${result} = Execute /opt/hadoop/bin/ozone admin datanode maintenance ${HOST}
Review Comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-10612. Add Robot test to verify Container Balancer for RATIS containers [ozone]
Posted by "afilpp (via GitHub)" <gi...@apache.org>.
afilpp commented on code in PR #6457:
URL: https://github.com/apache/ozone/pull/6457#discussion_r1546250306
##########
hadoop-ozone/dist/src/main/smoketest/balancer/testBalancer.robot:
##########
@@ -0,0 +1,141 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+*** Settings ***
+Documentation Smoketest ozone cluster startup
+Library OperatingSystem
+Library Collections
+Resource ../commonlib.robot
+Resource ../ozone-lib/shell.robot
+
+Test Timeout 20 minutes
+
+*** Variables ***
+${SECURITY_ENABLED} false
+${HOST} datanode1
+${VOLUME} volume1
+${BUCKET} bucket1
+${SIZE} 104857600
+
+
+** Keywords ***
+Prepare For Tests
+ Execute dd if=/dev/urandom of=/tmp/100mb bs=1048576 count=100
+ Run Keyword if '${SECURITY_ENABLED}' == 'true' Kinit test user testuser testuser.keytab
+ Execute ozone sh volume create /${VOLUME}
+ Execute ozone sh bucket create /${VOLUME}/${BUCKET}
+
+
+Datanode In Maintenance Mode
+ ${result} = Execute /opt/hadoop/bin/ozone admin datanode maintenance ${HOST}
+ Should Contain ${result} Entering maintenance mode on datanode
+ ${result} = Execute /opt/hadoop/bin/ozone admin datanode list | grep "Operational State:*"
+ Wait Until Keyword Succeeds 30sec 5sec Should contain ${result} ENTERING_MAINTENANCE
+ Wait Until Keyword Succeeds 3min 10sec Related pipelines are closed
+ Sleep 60000ms
+
+Related pipelines are closed
+ ${result} = Execute /opt/hadoop/bin/ozone admin datanode list | awk -v RS= '{$1=$1}1'|grep MAINT | sed -e 's/^.*pipelines: \\(.*\\)$/\\1/'
+ Should Contain Any ${result} CLOSED No related pipelines or the node is not in Healthy state.
+
+Datanode Recommission
+ ${result} = Execute /opt/hadoop/bin/ozone admin datanode recommission ${HOST}
+ Should Contain ${result} Started recommissioning datanode
+ Wait Until Keyword Succeeds 1min 10sec Datanode Recommission is Finished
+ Sleep 300000ms
+
+Datanode Recommission is Finished
+ ${result} = Execute /opt/hadoop/bin/ozone admin datanode list | grep "Operational State:*"
+ Should Not Contain ${result} ENTERING_MAINTENANCE
+
+Container Balancer
+ ${result} = Execute /opt/hadoop/bin/ozone admin containerbalancer start -t 1 -d 100 -i 1
+ Should Contain ${result} Container Balancer started successfully.
+ ${result} = Execute /opt/hadoop/bin/ozone admin containerbalancer status
+ Should Contain ${result} ContainerBalancer is Running.
+ Wait Until Keyword Succeeds 3min 10sec ContainerBalancer is Not Running
+ Sleep 60000ms
+
+ContainerBalancer is Not Running
+ ${result} = Execute /opt/hadoop/bin/ozone admin containerbalancer status
+ Should contain ${result} ContainerBalancer is Not Running.
+
+Create Multiple Keys
+ [arguments] ${NUM_KEYS}
+ ${file} = Set Variable /tmp/100mb
+ FOR ${INDEX} IN RANGE ${NUM_KEYS}
+ ${fileName} = Set Variable file-${INDEX}.txt
+ ${key} = Set Variable /${VOLUME}/${BUCKET}/${fileName}
+ LOG ${fileName}
+ Create Key ${key} ${file}
+ Key Should Match Local File ${key} ${file}
+ END
+
+Datanode Usageinfo
+ [arguments] ${uuid}
+ ${result} = Execute ozone admin datanode usageinfo --uuid=${uuid}
+ Should Contain ${result} Ozone Used
+
+Get Uuid
+ ${result} = Execute /opt/hadoop/bin/ozone admin datanode list | awk -v RS= '{$1=$1}1'| grep ${HOST} | sed -e 's/Datanode: //'|sed -e 's/ .*$//'
+ [return] ${result}
+
+Close All Containers
+ FOR ${INDEX} IN RANGE 15
+ ${container} = Execute ozone admin container list --state OPEN | jq -r 'select(.replicationConfig.replicationFactor == "THREE") | .containerID' | head -1
+ EXIT FOR LOOP IF "${container}" == ""
+ Execute ozone admin container close "${container}"
+ ${output} = Execute ozone admin container info "${container}"
+ Should contain ${output} CLOS
+ END
+ Wait until keyword succeeds 3min 10sec All container is closed
+
+All container is closed
+ ${output} = Execute ozone admin container list
+ Should Not Contain ${output} OPEN
+
+** Test Cases ***
+Verify Container Balancer for RATIS containers
+ Prepare For Tests
+
+ Datanode In Maintenance Mode
+
+ ${uuid} = Get Uuid
+ Datanode Usageinfo ${uuid}
+
+ Create Multiple Keys 3
+
+ Close All Containers
+
+ ${output1} = Execute export DATANODES=$(ozone admin datanode list --json) && for datanode in $(echo "$\{DATANODES\}" | jq -r '.[].datanodeDetails.uuid'); do ozone admin datanode usageinfo --uuid=$\{datanode\} --json | jq '{(.[0].datanodeDetails.uuid) : .[0].ozoneUsed}'; done | jq -s add
Review Comment:
Done
##########
hadoop-ozone/dist/src/main/smoketest/balancer/testBalancer.robot:
##########
@@ -0,0 +1,141 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+*** Settings ***
+Documentation Smoketest ozone cluster startup
+Library OperatingSystem
+Library Collections
+Resource ../commonlib.robot
+Resource ../ozone-lib/shell.robot
+
+Test Timeout 20 minutes
+
+*** Variables ***
+${SECURITY_ENABLED} false
+${HOST} datanode1
+${VOLUME} volume1
+${BUCKET} bucket1
+${SIZE} 104857600
+
+
+** Keywords ***
+Prepare For Tests
+ Execute dd if=/dev/urandom of=/tmp/100mb bs=1048576 count=100
+ Run Keyword if '${SECURITY_ENABLED}' == 'true' Kinit test user testuser testuser.keytab
+ Execute ozone sh volume create /${VOLUME}
+ Execute ozone sh bucket create /${VOLUME}/${BUCKET}
+
+
+Datanode In Maintenance Mode
+ ${result} = Execute /opt/hadoop/bin/ozone admin datanode maintenance ${HOST}
+ Should Contain ${result} Entering maintenance mode on datanode
+ ${result} = Execute /opt/hadoop/bin/ozone admin datanode list | grep "Operational State:*"
+ Wait Until Keyword Succeeds 30sec 5sec Should contain ${result} ENTERING_MAINTENANCE
+ Wait Until Keyword Succeeds 3min 10sec Related pipelines are closed
+ Sleep 60000ms
+
+Related pipelines are closed
+ ${result} = Execute /opt/hadoop/bin/ozone admin datanode list | awk -v RS= '{$1=$1}1'|grep MAINT | sed -e 's/^.*pipelines: \\(.*\\)$/\\1/'
+ Should Contain Any ${result} CLOSED No related pipelines or the node is not in Healthy state.
+
+Datanode Recommission
+ ${result} = Execute /opt/hadoop/bin/ozone admin datanode recommission ${HOST}
+ Should Contain ${result} Started recommissioning datanode
+ Wait Until Keyword Succeeds 1min 10sec Datanode Recommission is Finished
+ Sleep 300000ms
+
+Datanode Recommission is Finished
+ ${result} = Execute /opt/hadoop/bin/ozone admin datanode list | grep "Operational State:*"
+ Should Not Contain ${result} ENTERING_MAINTENANCE
+
+Container Balancer
+ ${result} = Execute /opt/hadoop/bin/ozone admin containerbalancer start -t 1 -d 100 -i 1
+ Should Contain ${result} Container Balancer started successfully.
+ ${result} = Execute /opt/hadoop/bin/ozone admin containerbalancer status
+ Should Contain ${result} ContainerBalancer is Running.
+ Wait Until Keyword Succeeds 3min 10sec ContainerBalancer is Not Running
+ Sleep 60000ms
+
+ContainerBalancer is Not Running
+ ${result} = Execute /opt/hadoop/bin/ozone admin containerbalancer status
+ Should contain ${result} ContainerBalancer is Not Running.
+
+Create Multiple Keys
+ [arguments] ${NUM_KEYS}
+ ${file} = Set Variable /tmp/100mb
+ FOR ${INDEX} IN RANGE ${NUM_KEYS}
+ ${fileName} = Set Variable file-${INDEX}.txt
+ ${key} = Set Variable /${VOLUME}/${BUCKET}/${fileName}
+ LOG ${fileName}
+ Create Key ${key} ${file}
+ Key Should Match Local File ${key} ${file}
+ END
+
+Datanode Usageinfo
+ [arguments] ${uuid}
+ ${result} = Execute ozone admin datanode usageinfo --uuid=${uuid}
+ Should Contain ${result} Ozone Used
+
+Get Uuid
+ ${result} = Execute /opt/hadoop/bin/ozone admin datanode list | awk -v RS= '{$1=$1}1'| grep ${HOST} | sed -e 's/Datanode: //'|sed -e 's/ .*$//'
+ [return] ${result}
+
+Close All Containers
+ FOR ${INDEX} IN RANGE 15
+ ${container} = Execute ozone admin container list --state OPEN | jq -r 'select(.replicationConfig.replicationFactor == "THREE") | .containerID' | head -1
+ EXIT FOR LOOP IF "${container}" == ""
+ Execute ozone admin container close "${container}"
+ ${output} = Execute ozone admin container info "${container}"
+ Should contain ${output} CLOS
+ END
+ Wait until keyword succeeds 3min 10sec All container is closed
+
+All container is closed
+ ${output} = Execute ozone admin container list
+ Should Not Contain ${output} OPEN
+
+** Test Cases ***
+Verify Container Balancer for RATIS containers
+ Prepare For Tests
+
+ Datanode In Maintenance Mode
+
+ ${uuid} = Get Uuid
+ Datanode Usageinfo ${uuid}
+
+ Create Multiple Keys 3
+
+ Close All Containers
+
+ ${output1} = Execute export DATANODES=$(ozone admin datanode list --json) && for datanode in $(echo "$\{DATANODES\}" | jq -r '.[].datanodeDetails.uuid'); do ozone admin datanode usageinfo --uuid=$\{datanode\} --json | jq '{(.[0].datanodeDetails.uuid) : .[0].ozoneUsed}'; done | jq -s add
+
+ ${datanodePreviousUsedBytes} Execute echo '${output1}' | jq '. | to_entries | .[] | select(.key == "${uuid}") | .value'
+ Should Be True ${datanodePreviousUsedBytes} < ${SIZE}
+
+ Datanode Recommission
+
+ Container Balancer
+
+ ${output2} = Execute export DATANODES=$(ozone admin datanode list --json) && for datanode in $(echo "$\{DATANODES\}" | jq -r '.[].datanodeDetails.uuid'); do ozone admin datanode usageinfo --uuid=$\{datanode\} --json | jq '{(.[0].datanodeDetails.uuid) : .[0].ozoneUsed}'; done | jq -s add
Review Comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-10612. Add Robot test to verify Container Balancer for RATIS containers [ozone]
Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai commented on code in PR #6457:
URL: https://github.com/apache/ozone/pull/6457#discussion_r1546064715
##########
hadoop-ozone/dist/src/main/smoketest/balancer/testBalancer.robot:
##########
@@ -0,0 +1,141 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+*** Settings ***
+Documentation Smoketest ozone cluster startup
+Library OperatingSystem
+Library Collections
+Resource ../commonlib.robot
+Resource ../ozone-lib/shell.robot
+
+Test Timeout 20 minutes
+
+*** Variables ***
+${SECURITY_ENABLED} false
+${HOST} datanode1
+${VOLUME} volume1
+${BUCKET} bucket1
+${SIZE} 104857600
+
+
+** Keywords ***
+Prepare For Tests
+ Execute dd if=/dev/urandom of=/tmp/100mb bs=1048576 count=100
+ Run Keyword if '${SECURITY_ENABLED}' == 'true' Kinit test user testuser testuser.keytab
+ Execute ozone sh volume create /${VOLUME}
+ Execute ozone sh bucket create /${VOLUME}/${BUCKET}
+
+
+Datanode In Maintenance Mode
+ ${result} = Execute /opt/hadoop/bin/ozone admin datanode maintenance ${HOST}
+ Should Contain ${result} Entering maintenance mode on datanode
+ ${result} = Execute /opt/hadoop/bin/ozone admin datanode list | grep "Operational State:*"
+ Wait Until Keyword Succeeds 30sec 5sec Should contain ${result} ENTERING_MAINTENANCE
+ Wait Until Keyword Succeeds 3min 10sec Related pipelines are closed
+ Sleep 60000ms
+
+Related pipelines are closed
+ ${result} = Execute /opt/hadoop/bin/ozone admin datanode list | awk -v RS= '{$1=$1}1'|grep MAINT | sed -e 's/^.*pipelines: \\(.*\\)$/\\1/'
+ Should Contain Any ${result} CLOSED No related pipelines or the node is not in Healthy state.
+
+Datanode Recommission
+ ${result} = Execute /opt/hadoop/bin/ozone admin datanode recommission ${HOST}
+ Should Contain ${result} Started recommissioning datanode
+ Wait Until Keyword Succeeds 1min 10sec Datanode Recommission is Finished
+ Sleep 300000ms
+
+Datanode Recommission is Finished
+ ${result} = Execute /opt/hadoop/bin/ozone admin datanode list | grep "Operational State:*"
+ Should Not Contain ${result} ENTERING_MAINTENANCE
+
+Container Balancer
+ ${result} = Execute /opt/hadoop/bin/ozone admin containerbalancer start -t 1 -d 100 -i 1
+ Should Contain ${result} Container Balancer started successfully.
+ ${result} = Execute /opt/hadoop/bin/ozone admin containerbalancer status
+ Should Contain ${result} ContainerBalancer is Running.
+ Wait Until Keyword Succeeds 3min 10sec ContainerBalancer is Not Running
+ Sleep 60000ms
+
+ContainerBalancer is Not Running
+ ${result} = Execute /opt/hadoop/bin/ozone admin containerbalancer status
+ Should contain ${result} ContainerBalancer is Not Running.
+
+Create Multiple Keys
+ [arguments] ${NUM_KEYS}
+ ${file} = Set Variable /tmp/100mb
+ FOR ${INDEX} IN RANGE ${NUM_KEYS}
+ ${fileName} = Set Variable file-${INDEX}.txt
+ ${key} = Set Variable /${VOLUME}/${BUCKET}/${fileName}
+ LOG ${fileName}
+ Create Key ${key} ${file}
+ Key Should Match Local File ${key} ${file}
+ END
+
+Datanode Usageinfo
+ [arguments] ${uuid}
+ ${result} = Execute ozone admin datanode usageinfo --uuid=${uuid}
+ Should Contain ${result} Ozone Used
+
+Get Uuid
+ ${result} = Execute /opt/hadoop/bin/ozone admin datanode list | awk -v RS= '{$1=$1}1'| grep ${HOST} | sed -e 's/Datanode: //'|sed -e 's/ .*$//'
+ [return] ${result}
+
+Close All Containers
+ FOR ${INDEX} IN RANGE 15
+ ${container} = Execute ozone admin container list --state OPEN | jq -r 'select(.replicationConfig.replicationFactor == "THREE") | .containerID' | head -1
+ EXIT FOR LOOP IF "${container}" == ""
+ Execute ozone admin container close "${container}"
+ ${output} = Execute ozone admin container info "${container}"
+ Should contain ${output} CLOS
+ END
+ Wait until keyword succeeds 3min 10sec All container is closed
+
+All container is closed
+ ${output} = Execute ozone admin container list
+ Should Not Contain ${output} OPEN
+
+** Test Cases ***
+Verify Container Balancer for RATIS containers
+ Prepare For Tests
+
+ Datanode In Maintenance Mode
+
+ ${uuid} = Get Uuid
+ Datanode Usageinfo ${uuid}
+
+ Create Multiple Keys 3
+
+ Close All Containers
+
+ ${output1} = Execute export DATANODES=$(ozone admin datanode list --json) && for datanode in $(echo "$\{DATANODES\}" | jq -r '.[].datanodeDetails.uuid'); do ozone admin datanode usageinfo --uuid=$\{datanode\} --json | jq '{(.[0].datanodeDetails.uuid) : .[0].ozoneUsed}'; done | jq -s add
+
+ ${datanodePreviousUsedBytes} Execute echo '${output1}' | jq '. | to_entries | .[] | select(.key == "${uuid}") | .value'
+ Should Be True ${datanodePreviousUsedBytes} < ${SIZE}
+
+ Datanode Recommission
+
+ Container Balancer
+
+ ${output2} = Execute export DATANODES=$(ozone admin datanode list --json) && for datanode in $(echo "$\{DATANODES\}" | jq -r '.[].datanodeDetails.uuid'); do ozone admin datanode usageinfo --uuid=$\{datanode\} --json | jq '{(.[0].datanodeDetails.uuid) : .[0].ozoneUsed}'; done | jq -s add
Review Comment:
Also, please extract the duplicated statement.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-10612. Add Robot test to verify Container Balancer for RATIS containers [ozone]
Posted by "vtutrinov (via GitHub)" <gi...@apache.org>.
vtutrinov commented on code in PR #6457:
URL: https://github.com/apache/ozone/pull/6457#discussion_r1546039519
##########
hadoop-ozone/dist/src/main/smoketest/balancer/testBalancer.robot:
##########
@@ -0,0 +1,141 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+*** Settings ***
+Documentation Smoketest ozone cluster startup
+Library OperatingSystem
+Library Collections
+Resource ../commonlib.robot
+Resource ../ozone-lib/shell.robot
+
+Test Timeout 20 minutes
+
+*** Variables ***
+${SECURITY_ENABLED} false
+${HOST} datanode1
+${VOLUME} volume1
+${BUCKET} bucket1
+${SIZE} 104857600
+
+
+** Keywords ***
+Prepare For Tests
+ Execute dd if=/dev/urandom of=/tmp/100mb bs=1048576 count=100
+ Run Keyword if '${SECURITY_ENABLED}' == 'true' Kinit test user testuser testuser.keytab
+ Execute ozone sh volume create /${VOLUME}
+ Execute ozone sh bucket create /${VOLUME}/${BUCKET}
+
+
+Datanode In Maintenance Mode
+ ${result} = Execute /opt/hadoop/bin/ozone admin datanode maintenance ${HOST}
+ Should Contain ${result} Entering maintenance mode on datanode
+ ${result} = Execute /opt/hadoop/bin/ozone admin datanode list | grep "Operational State:*"
+ Wait Until Keyword Succeeds 30sec 5sec Should contain ${result} ENTERING_MAINTENANCE
+ Wait Until Keyword Succeeds 3min 10sec Related pipelines are closed
+ Sleep 60000ms
+
+Related pipelines are closed
+ ${result} = Execute /opt/hadoop/bin/ozone admin datanode list | awk -v RS= '{$1=$1}1'|grep MAINT | sed -e 's/^.*pipelines: \\(.*\\)$/\\1/'
+ Should Contain Any ${result} CLOSED No related pipelines or the node is not in Healthy state.
+
+Datanode Recommission
+ ${result} = Execute /opt/hadoop/bin/ozone admin datanode recommission ${HOST}
+ Should Contain ${result} Started recommissioning datanode
+ Wait Until Keyword Succeeds 1min 10sec Datanode Recommission is Finished
+ Sleep 300000ms
+
+Datanode Recommission is Finished
+ ${result} = Execute /opt/hadoop/bin/ozone admin datanode list | grep "Operational State:*"
+ Should Not Contain ${result} ENTERING_MAINTENANCE
+
+Container Balancer
+ ${result} = Execute /opt/hadoop/bin/ozone admin containerbalancer start -t 1 -d 100 -i 1
+ Should Contain ${result} Container Balancer started successfully.
+ ${result} = Execute /opt/hadoop/bin/ozone admin containerbalancer status
+ Should Contain ${result} ContainerBalancer is Running.
+ Wait Until Keyword Succeeds 3min 10sec ContainerBalancer is Not Running
+ Sleep 60000ms
+
+ContainerBalancer is Not Running
+ ${result} = Execute /opt/hadoop/bin/ozone admin containerbalancer status
+ Should contain ${result} ContainerBalancer is Not Running.
+
+Create Multiple Keys
+ [arguments] ${NUM_KEYS}
+ ${file} = Set Variable /tmp/100mb
+ FOR ${INDEX} IN RANGE ${NUM_KEYS}
+ ${fileName} = Set Variable file-${INDEX}.txt
+ ${key} = Set Variable /${VOLUME}/${BUCKET}/${fileName}
+ LOG ${fileName}
+ Create Key ${key} ${file}
+ Key Should Match Local File ${key} ${file}
+ END
+
+Datanode Usageinfo
+ [arguments] ${uuid}
+ ${result} = Execute ozone admin datanode usageinfo --uuid=${uuid}
+ Should Contain ${result} Ozone Used
+
+Get Uuid
+ ${result} = Execute /opt/hadoop/bin/ozone admin datanode list | awk -v RS= '{$1=$1}1'| grep ${HOST} | sed -e 's/Datanode: //'|sed -e 's/ .*$//'
+ [return] ${result}
+
+Close All Containers
+ FOR ${INDEX} IN RANGE 15
+ ${container} = Execute ozone admin container list --state OPEN | jq -r 'select(.replicationConfig.replicationFactor == "THREE") | .containerID' | head -1
+ EXIT FOR LOOP IF "${container}" == ""
+ Execute ozone admin container close "${container}"
+ ${output} = Execute ozone admin container info "${container}"
+ Should contain ${output} CLOS
+ END
+ Wait until keyword succeeds 3min 10sec All container is closed
+
+All container is closed
+ ${output} = Execute ozone admin container list
+ Should Not Contain ${output} OPEN
+
+** Test Cases ***
+Verify Container Balancer for RATIS containers
+ Prepare For Tests
+
+ Datanode In Maintenance Mode
+
+ ${uuid} = Get Uuid
+ Datanode Usageinfo ${uuid}
+
+ Create Multiple Keys 3
+
+ Close All Containers
+
+ ${output1} = Execute export DATANODES=$(ozone admin datanode list --json) && for datanode in $(echo "$\{DATANODES\}" | jq -r '.[].datanodeDetails.uuid'); do ozone admin datanode usageinfo --uuid=$\{datanode\} --json | jq '{(.[0].datanodeDetails.uuid) : .[0].ozoneUsed}'; done | jq -s add
+
+ ${datanodePreviousUsedBytes} Execute echo '${output1}' | jq '. | to_entries | .[] | select(.key == "${uuid}") | .value'
+ Should Be True ${datanodePreviousUsedBytes} < ${SIZE}
+
+ Datanode Recommission
+
+ Container Balancer
+
+ ${output2} = Execute export DATANODES=$(ozone admin datanode list --json) && for datanode in $(echo "$\{DATANODES\}" | jq -r '.[].datanodeDetails.uuid'); do ozone admin datanode usageinfo --uuid=$\{datanode\} --json | jq '{(.[0].datanodeDetails.uuid) : .[0].ozoneUsed}'; done | jq -s add
Review Comment:
same here: datanodeOzoneUsedBytesInfoAfterContainerBalancing
##########
hadoop-ozone/dist/src/main/smoketest/balancer/testBalancer.robot:
##########
@@ -0,0 +1,141 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+*** Settings ***
+Documentation Smoketest ozone cluster startup
+Library OperatingSystem
+Library Collections
+Resource ../commonlib.robot
+Resource ../ozone-lib/shell.robot
+
+Test Timeout 20 minutes
+
+*** Variables ***
+${SECURITY_ENABLED} false
+${HOST} datanode1
+${VOLUME} volume1
+${BUCKET} bucket1
+${SIZE} 104857600
+
+
+** Keywords ***
+Prepare For Tests
+ Execute dd if=/dev/urandom of=/tmp/100mb bs=1048576 count=100
+ Run Keyword if '${SECURITY_ENABLED}' == 'true' Kinit test user testuser testuser.keytab
+ Execute ozone sh volume create /${VOLUME}
+ Execute ozone sh bucket create /${VOLUME}/${BUCKET}
+
+
+Datanode In Maintenance Mode
+ ${result} = Execute /opt/hadoop/bin/ozone admin datanode maintenance ${HOST}
+ Should Contain ${result} Entering maintenance mode on datanode
+ ${result} = Execute /opt/hadoop/bin/ozone admin datanode list | grep "Operational State:*"
+ Wait Until Keyword Succeeds 30sec 5sec Should contain ${result} ENTERING_MAINTENANCE
+ Wait Until Keyword Succeeds 3min 10sec Related pipelines are closed
+ Sleep 60000ms
+
+Related pipelines are closed
+ ${result} = Execute /opt/hadoop/bin/ozone admin datanode list | awk -v RS= '{$1=$1}1'|grep MAINT | sed -e 's/^.*pipelines: \\(.*\\)$/\\1/'
+ Should Contain Any ${result} CLOSED No related pipelines or the node is not in Healthy state.
+
+Datanode Recommission
+ ${result} = Execute /opt/hadoop/bin/ozone admin datanode recommission ${HOST}
+ Should Contain ${result} Started recommissioning datanode
+ Wait Until Keyword Succeeds 1min 10sec Datanode Recommission is Finished
+ Sleep 300000ms
+
+Datanode Recommission is Finished
+ ${result} = Execute /opt/hadoop/bin/ozone admin datanode list | grep "Operational State:*"
+ Should Not Contain ${result} ENTERING_MAINTENANCE
+
+Container Balancer
+ ${result} = Execute /opt/hadoop/bin/ozone admin containerbalancer start -t 1 -d 100 -i 1
+ Should Contain ${result} Container Balancer started successfully.
+ ${result} = Execute /opt/hadoop/bin/ozone admin containerbalancer status
+ Should Contain ${result} ContainerBalancer is Running.
+ Wait Until Keyword Succeeds 3min 10sec ContainerBalancer is Not Running
+ Sleep 60000ms
+
+ContainerBalancer is Not Running
+ ${result} = Execute /opt/hadoop/bin/ozone admin containerbalancer status
+ Should contain ${result} ContainerBalancer is Not Running.
+
+Create Multiple Keys
+ [arguments] ${NUM_KEYS}
+ ${file} = Set Variable /tmp/100mb
+ FOR ${INDEX} IN RANGE ${NUM_KEYS}
+ ${fileName} = Set Variable file-${INDEX}.txt
+ ${key} = Set Variable /${VOLUME}/${BUCKET}/${fileName}
+ LOG ${fileName}
+ Create Key ${key} ${file}
+ Key Should Match Local File ${key} ${file}
+ END
+
+Datanode Usageinfo
+ [arguments] ${uuid}
+ ${result} = Execute ozone admin datanode usageinfo --uuid=${uuid}
+ Should Contain ${result} Ozone Used
+
+Get Uuid
+ ${result} = Execute /opt/hadoop/bin/ozone admin datanode list | awk -v RS= '{$1=$1}1'| grep ${HOST} | sed -e 's/Datanode: //'|sed -e 's/ .*$//'
+ [return] ${result}
+
+Close All Containers
+ FOR ${INDEX} IN RANGE 15
+ ${container} = Execute ozone admin container list --state OPEN | jq -r 'select(.replicationConfig.replicationFactor == "THREE") | .containerID' | head -1
+ EXIT FOR LOOP IF "${container}" == ""
+ Execute ozone admin container close "${container}"
+ ${output} = Execute ozone admin container info "${container}"
+ Should contain ${output} CLOS
+ END
+ Wait until keyword succeeds 3min 10sec All container is closed
+
+All container is closed
+ ${output} = Execute ozone admin container list
+ Should Not Contain ${output} OPEN
+
+** Test Cases ***
+Verify Container Balancer for RATIS containers
+ Prepare For Tests
+
+ Datanode In Maintenance Mode
+
+ ${uuid} = Get Uuid
+ Datanode Usageinfo ${uuid}
+
+ Create Multiple Keys 3
+
+ Close All Containers
+
+ ${output1} = Execute export DATANODES=$(ozone admin datanode list --json) && for datanode in $(echo "$\{DATANODES\}" | jq -r '.[].datanodeDetails.uuid'); do ozone admin datanode usageinfo --uuid=$\{datanode\} --json | jq '{(.[0].datanodeDetails.uuid) : .[0].ozoneUsed}'; done | jq -s add
Review Comment:
I'd rename the variable to 'datanodeOzoneUsedBytesInfo'
##########
hadoop-ozone/dist/src/main/compose/ozone-balancer/docker-compose.yaml:
##########
@@ -0,0 +1,183 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+version: "3.8"
+
+# reusable fragments (see https://docs.docker.com/compose/compose-file/#extension-fields)
+x-common-config:
+ &common-config
+ image: ${OZONE_RUNNER_IMAGE}:${OZONE_RUNNER_VERSION}
+ volumes:
+ - ../..:/opt/hadoop
+ env_file:
+ - docker-config
+
+x-replication:
+ &replication
+ OZONE-SITE.XML_ozone.server.default.replication: ${OZONE_REPLICATION_FACTOR:-3}
+
+services:
+ datanode1:
+ <<: *common-config
+ ports:
+ - 19864
+ - 9882
+ environment:
+ <<: *replication
+ OZONE_OPTS: -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=*:5005
Review Comment:
the same comment is relevant for the datanodes below
##########
hadoop-ozone/dist/src/main/compose/ozone-balancer/docker-compose.yaml:
##########
@@ -0,0 +1,183 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+version: "3.8"
+
+# reusable fragments (see https://docs.docker.com/compose/compose-file/#extension-fields)
+x-common-config:
+ &common-config
+ image: ${OZONE_RUNNER_IMAGE}:${OZONE_RUNNER_VERSION}
+ volumes:
+ - ../..:/opt/hadoop
+ env_file:
+ - docker-config
+
+x-replication:
+ &replication
+ OZONE-SITE.XML_ozone.server.default.replication: ${OZONE_REPLICATION_FACTOR:-3}
+
+services:
+ datanode1:
+ <<: *common-config
+ ports:
+ - 19864
+ - 9882
+ environment:
+ <<: *replication
+ OZONE_OPTS: -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=*:5005
Review Comment:
seems the env var could be dropped out here
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-10612. Add Robot test to verify Container Balancer for RATIS containers [ozone]
Posted by "myskov (via GitHub)" <gi...@apache.org>.
myskov merged PR #6457:
URL: https://github.com/apache/ozone/pull/6457
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-10612. Add Robot test to verify Container Balancer for RATIS containers [ozone]
Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai commented on code in PR #6457:
URL: https://github.com/apache/ozone/pull/6457#discussion_r1546070055
##########
hadoop-ozone/dist/src/main/smoketest/balancer/testBalancer.robot:
##########
@@ -0,0 +1,141 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+*** Settings ***
+Documentation Smoketest ozone cluster startup
+Library OperatingSystem
+Library Collections
+Resource ../commonlib.robot
+Resource ../ozone-lib/shell.robot
+
+Test Timeout 20 minutes
+
+*** Variables ***
+${SECURITY_ENABLED} false
+${HOST} datanode1
+${VOLUME} volume1
+${BUCKET} bucket1
+${SIZE} 104857600
+
+
+** Keywords ***
+Prepare For Tests
+ Execute dd if=/dev/urandom of=/tmp/100mb bs=1048576 count=100
+ Run Keyword if '${SECURITY_ENABLED}' == 'true' Kinit test user testuser testuser.keytab
+ Execute ozone sh volume create /${VOLUME}
+ Execute ozone sh bucket create /${VOLUME}/${BUCKET}
+
+
+Datanode In Maintenance Mode
+ ${result} = Execute /opt/hadoop/bin/ozone admin datanode maintenance ${HOST}
+ Should Contain ${result} Entering maintenance mode on datanode
+ ${result} = Execute /opt/hadoop/bin/ozone admin datanode list | grep "Operational State:*"
+ Wait Until Keyword Succeeds 30sec 5sec Should contain ${result} ENTERING_MAINTENANCE
+ Wait Until Keyword Succeeds 3min 10sec Related pipelines are closed
+ Sleep 60000ms
+
+Related pipelines are closed
+ ${result} = Execute /opt/hadoop/bin/ozone admin datanode list | awk -v RS= '{$1=$1}1'|grep MAINT | sed -e 's/^.*pipelines: \\(.*\\)$/\\1/'
+ Should Contain Any ${result} CLOSED No related pipelines or the node is not in Healthy state.
+
+Datanode Recommission
+ ${result} = Execute /opt/hadoop/bin/ozone admin datanode recommission ${HOST}
+ Should Contain ${result} Started recommissioning datanode
+ Wait Until Keyword Succeeds 1min 10sec Datanode Recommission is Finished
+ Sleep 300000ms
+
+Datanode Recommission is Finished
+ ${result} = Execute /opt/hadoop/bin/ozone admin datanode list | grep "Operational State:*"
+ Should Not Contain ${result} ENTERING_MAINTENANCE
+
+Container Balancer
+ ${result} = Execute /opt/hadoop/bin/ozone admin containerbalancer start -t 1 -d 100 -i 1
+ Should Contain ${result} Container Balancer started successfully.
+ ${result} = Execute /opt/hadoop/bin/ozone admin containerbalancer status
+ Should Contain ${result} ContainerBalancer is Running.
+ Wait Until Keyword Succeeds 3min 10sec ContainerBalancer is Not Running
+ Sleep 60000ms
+
+ContainerBalancer is Not Running
+ ${result} = Execute /opt/hadoop/bin/ozone admin containerbalancer status
+ Should contain ${result} ContainerBalancer is Not Running.
+
+Create Multiple Keys
+ [arguments] ${NUM_KEYS}
+ ${file} = Set Variable /tmp/100mb
+ FOR ${INDEX} IN RANGE ${NUM_KEYS}
+ ${fileName} = Set Variable file-${INDEX}.txt
+ ${key} = Set Variable /${VOLUME}/${BUCKET}/${fileName}
+ LOG ${fileName}
+ Create Key ${key} ${file}
+ Key Should Match Local File ${key} ${file}
+ END
+
+Datanode Usageinfo
+ [arguments] ${uuid}
+ ${result} = Execute ozone admin datanode usageinfo --uuid=${uuid}
+ Should Contain ${result} Ozone Used
+
+Get Uuid
+ ${result} = Execute /opt/hadoop/bin/ozone admin datanode list | awk -v RS= '{$1=$1}1'| grep ${HOST} | sed -e 's/Datanode: //'|sed -e 's/ .*$//'
+ [return] ${result}
+
+Close All Containers
+ FOR ${INDEX} IN RANGE 15
+ ${container} = Execute ozone admin container list --state OPEN | jq -r 'select(.replicationConfig.replicationFactor == "THREE") | .containerID' | head -1
+ EXIT FOR LOOP IF "${container}" == ""
+ Execute ozone admin container close "${container}"
+ ${output} = Execute ozone admin container info "${container}"
+ Should contain ${output} CLOS
+ END
+ Wait until keyword succeeds 3min 10sec All container is closed
+
+All container is closed
+ ${output} = Execute ozone admin container list
+ Should Not Contain ${output} OPEN
Review Comment:
Can apply filter `--state OPEN` (for reduced output).
##########
hadoop-ozone/dist/src/main/compose/ozone-balancer/test.sh:
##########
@@ -0,0 +1,37 @@
+#!/usr/bin/env bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+#suite:balancer
+
+COMPOSE_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
+export COMPOSE_DIR
+export OM_SERVICE_ID="om"
+export OM=om1
+export SCM=scm1
+export OZONE_REPLICATION_FACTOR=3
+
+# shellcheck source=/dev/null
+source "$COMPOSE_DIR/../testlib.sh"
+
+# We need 4 dataNodes in this tests
+start_docker_env 4
+
+# Start OMs separately. In this test, the OMs will be stopped and restarted multiple times.
+# So we do not want the container to be tied to the OM process.
+#startOMs
Review Comment:
Seems to be leftover from OM HA test.
##########
hadoop-ozone/dist/src/main/smoketest/balancer/testBalancer.robot:
##########
@@ -0,0 +1,141 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+*** Settings ***
+Documentation Smoketest ozone cluster startup
+Library OperatingSystem
+Library Collections
+Resource ../commonlib.robot
+Resource ../ozone-lib/shell.robot
+
+Test Timeout 20 minutes
+
+*** Variables ***
+${SECURITY_ENABLED} false
+${HOST} datanode1
+${VOLUME} volume1
+${BUCKET} bucket1
+${SIZE} 104857600
+
+
+** Keywords ***
+Prepare For Tests
+ Execute dd if=/dev/urandom of=/tmp/100mb bs=1048576 count=100
+ Run Keyword if '${SECURITY_ENABLED}' == 'true' Kinit test user testuser testuser.keytab
+ Execute ozone sh volume create /${VOLUME}
+ Execute ozone sh bucket create /${VOLUME}/${BUCKET}
+
+
+Datanode In Maintenance Mode
+ ${result} = Execute /opt/hadoop/bin/ozone admin datanode maintenance ${HOST}
+ Should Contain ${result} Entering maintenance mode on datanode
+ ${result} = Execute /opt/hadoop/bin/ozone admin datanode list | grep "Operational State:*"
+ Wait Until Keyword Succeeds 30sec 5sec Should contain ${result} ENTERING_MAINTENANCE
+ Wait Until Keyword Succeeds 3min 10sec Related pipelines are closed
+ Sleep 60000ms
+
+Related pipelines are closed
+ ${result} = Execute /opt/hadoop/bin/ozone admin datanode list | awk -v RS= '{$1=$1}1'|grep MAINT | sed -e 's/^.*pipelines: \\(.*\\)$/\\1/'
+ Should Contain Any ${result} CLOSED No related pipelines or the node is not in Healthy state.
+
+Datanode Recommission
+ ${result} = Execute /opt/hadoop/bin/ozone admin datanode recommission ${HOST}
+ Should Contain ${result} Started recommissioning datanode
+ Wait Until Keyword Succeeds 1min 10sec Datanode Recommission is Finished
+ Sleep 300000ms
+
+Datanode Recommission is Finished
+ ${result} = Execute /opt/hadoop/bin/ozone admin datanode list | grep "Operational State:*"
+ Should Not Contain ${result} ENTERING_MAINTENANCE
+
+Container Balancer
Review Comment:
nit: I think `Balance Containers` or `Run Container Balancer` would be better
##########
hadoop-ozone/dist/src/main/smoketest/balancer/testBalancer.robot:
##########
@@ -0,0 +1,141 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+*** Settings ***
+Documentation Smoketest ozone cluster startup
+Library OperatingSystem
+Library Collections
+Resource ../commonlib.robot
+Resource ../ozone-lib/shell.robot
+
+Test Timeout 20 minutes
+
+*** Variables ***
+${SECURITY_ENABLED} false
+${HOST} datanode1
+${VOLUME} volume1
+${BUCKET} bucket1
+${SIZE} 104857600
+
+
+** Keywords ***
+Prepare For Tests
+ Execute dd if=/dev/urandom of=/tmp/100mb bs=1048576 count=100
+ Run Keyword if '${SECURITY_ENABLED}' == 'true' Kinit test user testuser testuser.keytab
+ Execute ozone sh volume create /${VOLUME}
+ Execute ozone sh bucket create /${VOLUME}/${BUCKET}
+
+
+Datanode In Maintenance Mode
+ ${result} = Execute /opt/hadoop/bin/ozone admin datanode maintenance ${HOST}
Review Comment:
nit: let's use `ozone` consistently
```suggestion
${result} = Execute ozone admin datanode maintenance ${HOST}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-10612. Add Robot test to verify Container Balancer for RATIS containers [ozone]
Posted by "afilpp (via GitHub)" <gi...@apache.org>.
afilpp commented on code in PR #6457:
URL: https://github.com/apache/ozone/pull/6457#discussion_r1546250912
##########
hadoop-ozone/dist/src/main/smoketest/balancer/testBalancer.robot:
##########
@@ -0,0 +1,141 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+*** Settings ***
+Documentation Smoketest ozone cluster startup
+Library OperatingSystem
+Library Collections
+Resource ../commonlib.robot
+Resource ../ozone-lib/shell.robot
+
+Test Timeout 20 minutes
+
+*** Variables ***
+${SECURITY_ENABLED} false
+${HOST} datanode1
+${VOLUME} volume1
+${BUCKET} bucket1
+${SIZE} 104857600
+
+
+** Keywords ***
+Prepare For Tests
+ Execute dd if=/dev/urandom of=/tmp/100mb bs=1048576 count=100
+ Run Keyword if '${SECURITY_ENABLED}' == 'true' Kinit test user testuser testuser.keytab
+ Execute ozone sh volume create /${VOLUME}
+ Execute ozone sh bucket create /${VOLUME}/${BUCKET}
+
+
+Datanode In Maintenance Mode
+ ${result} = Execute /opt/hadoop/bin/ozone admin datanode maintenance ${HOST}
+ Should Contain ${result} Entering maintenance mode on datanode
+ ${result} = Execute /opt/hadoop/bin/ozone admin datanode list | grep "Operational State:*"
+ Wait Until Keyword Succeeds 30sec 5sec Should contain ${result} ENTERING_MAINTENANCE
+ Wait Until Keyword Succeeds 3min 10sec Related pipelines are closed
+ Sleep 60000ms
+
+Related pipelines are closed
+ ${result} = Execute /opt/hadoop/bin/ozone admin datanode list | awk -v RS= '{$1=$1}1'|grep MAINT | sed -e 's/^.*pipelines: \\(.*\\)$/\\1/'
+ Should Contain Any ${result} CLOSED No related pipelines or the node is not in Healthy state.
+
+Datanode Recommission
+ ${result} = Execute /opt/hadoop/bin/ozone admin datanode recommission ${HOST}
+ Should Contain ${result} Started recommissioning datanode
+ Wait Until Keyword Succeeds 1min 10sec Datanode Recommission is Finished
+ Sleep 300000ms
+
+Datanode Recommission is Finished
+ ${result} = Execute /opt/hadoop/bin/ozone admin datanode list | grep "Operational State:*"
+ Should Not Contain ${result} ENTERING_MAINTENANCE
+
+Container Balancer
+ ${result} = Execute /opt/hadoop/bin/ozone admin containerbalancer start -t 1 -d 100 -i 1
+ Should Contain ${result} Container Balancer started successfully.
+ ${result} = Execute /opt/hadoop/bin/ozone admin containerbalancer status
+ Should Contain ${result} ContainerBalancer is Running.
+ Wait Until Keyword Succeeds 3min 10sec ContainerBalancer is Not Running
+ Sleep 60000ms
+
+ContainerBalancer is Not Running
+ ${result} = Execute /opt/hadoop/bin/ozone admin containerbalancer status
+ Should contain ${result} ContainerBalancer is Not Running.
+
+Create Multiple Keys
+ [arguments] ${NUM_KEYS}
+ ${file} = Set Variable /tmp/100mb
+ FOR ${INDEX} IN RANGE ${NUM_KEYS}
+ ${fileName} = Set Variable file-${INDEX}.txt
+ ${key} = Set Variable /${VOLUME}/${BUCKET}/${fileName}
+ LOG ${fileName}
+ Create Key ${key} ${file}
+ Key Should Match Local File ${key} ${file}
+ END
+
+Datanode Usageinfo
+ [arguments] ${uuid}
+ ${result} = Execute ozone admin datanode usageinfo --uuid=${uuid}
+ Should Contain ${result} Ozone Used
+
+Get Uuid
+ ${result} = Execute /opt/hadoop/bin/ozone admin datanode list | awk -v RS= '{$1=$1}1'| grep ${HOST} | sed -e 's/Datanode: //'|sed -e 's/ .*$//'
+ [return] ${result}
+
+Close All Containers
+ FOR ${INDEX} IN RANGE 15
+ ${container} = Execute ozone admin container list --state OPEN | jq -r 'select(.replicationConfig.replicationFactor == "THREE") | .containerID' | head -1
+ EXIT FOR LOOP IF "${container}" == ""
+ Execute ozone admin container close "${container}"
+ ${output} = Execute ozone admin container info "${container}"
+ Should contain ${output} CLOS
+ END
+ Wait until keyword succeeds 3min 10sec All container is closed
+
+All container is closed
+ ${output} = Execute ozone admin container list
+ Should Not Contain ${output} OPEN
Review Comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org