You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by GitBox <gi...@apache.org> on 2022/02/02 18:48:55 UTC
[GitHub] [ozone] adoroszlai commented on a change in pull request #3030: HDDS-6231. EC: Create EC acceptance test environment and some basic tests
adoroszlai commented on a change in pull request #3030:
URL: https://github.com/apache/ozone/pull/3030#discussion_r797897560
##########
File path: hadoop-ozone/dist/src/main/smoketest/ec/basic.robot
##########
@@ -0,0 +1,105 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+*** Settings ***
+Documentation Test EC shell commands
+Library OperatingSystem
+Resource ../commonlib.robot
+Suite Setup Generate prefix
+
+*** Variables ***
+${SCM} scm
+
+*** Keywords ***
+
+Generate prefix
+ ${random} = Generate Random String 5 [NUMBERS]
+ Set Suite Variable ${prefix} ${random}
+ Execute dd if=/dev/urandom of=/tmp/1mb bs=1048576 count=1
+ Execute dd if=/dev/urandom of=/tmp/2mb bs=1048576 count=2
+ Execute dd if=/dev/urandom of=/tmp/3mb bs=1048576 count=3
+ Execute dd if=/dev/urandom of=/tmp/100mb bs=1048576 count=100
+
+*** Test Cases ***
+
+Test Bucket Creation
+ ${result} = Execute ozone sh volume create /${prefix}vol1
+ Should not contain ${result} Failed
+ ${result} = Execute ozone sh bucket create /${prefix}vol1/${prefix}ratis
+ Should not contain ${result} Failed
+ ${result} = Execute ozone sh bucket list /${prefix}vol1 | jq -r '.[] | select(.name | contains("${prefix}ratis")) | .replicationConfig.replicationType'
+ Should contain ${result} RATIS
+ ${result} = Execute ozone sh bucket create --replication rs-3-2-1024k --type EC /${prefix}vol1/${prefix}ec
+ Should not contain ${result} Failed
+ ${result} = Execute ozone sh bucket list /${prefix}vol1 | jq -r '.[] | select(.name | contains("${prefix}ec")) | .replicationConfig.replicationType, .replicationConfig.codec, .replicationConfig.data, .replicationConfig.parity, .replicationConfig.ecChunkSize'
+ Should Match Regexp ${result} ^(?m)EC$
+ Should Match Regexp ${result} ^(?m)RS$
+ Should Match Regexp ${result} ^(?m)3$
+ Should Match Regexp ${result} ^(?m)2$
+ Should Match Regexp ${result} ^(?m)1048576$
+
+Test key Creation
+ ${result} = Run and Return RC ozone sh key put /${prefix}vol1/${prefix}ec/${prefix}1mb /tmp/1mb
+ Should Be Equal As Integers ${result} 0
Review comment:
`Execute` is the same as `Run and Return RC` + checking that RC is 0. So this (and following similar items) can be simplified to:
```suggestion
Execute ozone sh key put /${prefix}vol1/${prefix}ec/${prefix}1mb /tmp/1mb
```
##########
File path: hadoop-ozone/dist/src/main/smoketest/ec/basic.robot
##########
@@ -0,0 +1,105 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+*** Settings ***
+Documentation Test EC shell commands
+Library OperatingSystem
+Resource ../commonlib.robot
+Suite Setup Generate prefix
+
+*** Variables ***
+${SCM} scm
+
+*** Keywords ***
+
+Generate prefix
+ ${random} = Generate Random String 5 [NUMBERS]
+ Set Suite Variable ${prefix} ${random}
+ Execute dd if=/dev/urandom of=/tmp/1mb bs=1048576 count=1
+ Execute dd if=/dev/urandom of=/tmp/2mb bs=1048576 count=2
+ Execute dd if=/dev/urandom of=/tmp/3mb bs=1048576 count=3
+ Execute dd if=/dev/urandom of=/tmp/100mb bs=1048576 count=100
+
+*** Test Cases ***
+
+Test Bucket Creation
+ ${result} = Execute ozone sh volume create /${prefix}vol1
+ Should not contain ${result} Failed
+ ${result} = Execute ozone sh bucket create /${prefix}vol1/${prefix}ratis
+ Should not contain ${result} Failed
+ ${result} = Execute ozone sh bucket list /${prefix}vol1 | jq -r '.[] | select(.name | contains("${prefix}ratis")) | .replicationConfig.replicationType'
+ Should contain ${result} RATIS
+ ${result} = Execute ozone sh bucket create --replication rs-3-2-1024k --type EC /${prefix}vol1/${prefix}ec
+ Should not contain ${result} Failed
+ ${result} = Execute ozone sh bucket list /${prefix}vol1 | jq -r '.[] | select(.name | contains("${prefix}ec")) | .replicationConfig.replicationType, .replicationConfig.codec, .replicationConfig.data, .replicationConfig.parity, .replicationConfig.ecChunkSize'
+ Should Match Regexp ${result} ^(?m)EC$
+ Should Match Regexp ${result} ^(?m)RS$
+ Should Match Regexp ${result} ^(?m)3$
+ Should Match Regexp ${result} ^(?m)2$
+ Should Match Regexp ${result} ^(?m)1048576$
+
+Test key Creation
+ ${result} = Run and Return RC ozone sh key put /${prefix}vol1/${prefix}ec/${prefix}1mb /tmp/1mb
+ Should Be Equal As Integers ${result} 0
+ ${result} = Run and Return RC ozone sh key put /${prefix}vol1/${prefix}ec/${prefix}2mb /tmp/2mb
+ Should Be Equal As Integers ${result} 0
+ ${result} = Run and Return RC ozone sh key put /${prefix}vol1/${prefix}ec/${prefix}3mb /tmp/3mb
+ Should Be Equal As Integers ${result} 0
+ ${result} = Run and Return RC ozone sh key put /${prefix}vol1/${prefix}ec/${prefix}100mb /tmp/100mb
+ Should Be Equal As Integers ${result} 0
+
+ # Now Get the files and check they match
+ ${result} = Run and Return RC ozone sh key get /${prefix}vol1/${prefix}ec/${prefix}1mb /tmp/${prefix}1mb
+ Should Be Equal As Integers ${result} 0
+ Compare files /tmp/${prefix}1mb /tmp/1mb
Review comment:
We have a custom keyword `Key Should Match Local File` for this purpose.
```suggestion
Key Should Match Local File /${prefix}vol1/${prefix}ec/${prefix}1mb /tmp/1mb
```
The following "import" is needed for the keyword:
```
Resource ../ozone-lib/shell.robot
```
##########
File path: hadoop-ozone/dist/src/main/smoketest/ec/basic.robot
##########
@@ -0,0 +1,105 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+*** Settings ***
+Documentation Test EC shell commands
+Library OperatingSystem
+Resource ../commonlib.robot
+Suite Setup Generate prefix
+
+*** Variables ***
+${SCM} scm
+
+*** Keywords ***
+
+Generate prefix
Review comment:
Nit: Since it also creates a couple of files, I think we can call this something more generic, like `Prepare for tests` (and adjust `Suite Setup` accordingly).
##########
File path: hadoop-ozone/dist/src/main/smoketest/ec/basic.robot
##########
@@ -0,0 +1,105 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+*** Settings ***
+Documentation Test EC shell commands
+Library OperatingSystem
+Resource ../commonlib.robot
+Suite Setup Generate prefix
+
+*** Variables ***
+${SCM} scm
+
+*** Keywords ***
+
+Generate prefix
+ ${random} = Generate Random String 5 [NUMBERS]
+ Set Suite Variable ${prefix} ${random}
+ Execute dd if=/dev/urandom of=/tmp/1mb bs=1048576 count=1
+ Execute dd if=/dev/urandom of=/tmp/2mb bs=1048576 count=2
+ Execute dd if=/dev/urandom of=/tmp/3mb bs=1048576 count=3
+ Execute dd if=/dev/urandom of=/tmp/100mb bs=1048576 count=100
+
+*** Test Cases ***
+
+Test Bucket Creation
+ ${result} = Execute ozone sh volume create /${prefix}vol1
+ Should not contain ${result} Failed
+ ${result} = Execute ozone sh bucket create /${prefix}vol1/${prefix}ratis
+ Should not contain ${result} Failed
+ ${result} = Execute ozone sh bucket list /${prefix}vol1 | jq -r '.[] | select(.name | contains("${prefix}ratis")) | .replicationConfig.replicationType'
+ Should contain ${result} RATIS
+ ${result} = Execute ozone sh bucket create --replication rs-3-2-1024k --type EC /${prefix}vol1/${prefix}ec
+ Should not contain ${result} Failed
+ ${result} = Execute ozone sh bucket list /${prefix}vol1 | jq -r '.[] | select(.name | contains("${prefix}ec")) | .replicationConfig.replicationType, .replicationConfig.codec, .replicationConfig.data, .replicationConfig.parity, .replicationConfig.ecChunkSize'
+ Should Match Regexp ${result} ^(?m)EC$
+ Should Match Regexp ${result} ^(?m)RS$
+ Should Match Regexp ${result} ^(?m)3$
+ Should Match Regexp ${result} ^(?m)2$
+ Should Match Regexp ${result} ^(?m)1048576$
Review comment:
Can you please create a keyword for this group of checks?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org