You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@daffodil.apache.org by da...@apache.org on 2023/09/27 19:19:29 UTC
[daffodil-vscode] branch main updated: initial testing checklist
This is an automated email from the ASF dual-hosted git repository.
davin pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/daffodil-vscode.git
The following commit(s) were added to refs/heads/main by this push:
new 7b27a84 initial testing checklist
7b27a84 is described below
commit 7b27a84decfdae662eb0cef8db5249dd7a65bf66
Author: Davin Shearer <sc...@gmail.com>
AuthorDate: Thu Aug 3 13:40:52 2023 -0400
initial testing checklist
---
src/tests/README.md | 176 ++++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 176 insertions(+)
diff --git a/src/tests/README.md b/src/tests/README.md
new file mode 100644
index 0000000..c715987
--- /dev/null
+++ b/src/tests/README.md
@@ -0,0 +1,176 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# Testing Checklist
+
+## Requirements
+
+- [ ] install Java 8 or higher
+- [ ] supported VSCode version
+- [ ] operating system
+ - [ ] Windows
+ - [ ] x86
+ - [ ] Linux
+ - [ ] x86
+ - [ ] ARM
+ - [ ] MacOS
+ - [ ] x86
+ - [ ] Apple Silicon
+
+## Installation
+
+- [ ] install extension from file (.vsix)
+- [ ] install extension from marketplace
+- [ ] verify that the extension is installed and activated
+
+## DFDL Debugger
+
+- [ ] open a DFDL file
+- [ ] verify source highlighting
+- [ ] verify code completion
+- [ ] verify breakpoints
+- [ ] verify hex view
+- [ ] verify InfoSet view
+- [ ] verify InfoSet diff
+- [ ] verify InfoSet output in supported formats
+ - [ ] XML
+ - [ ] JSON
+
+## TDML
+
+- [ ] open a TDML file
+- [ ] generate a TDML file from debugging a daffodil schema
+- [ ] verify that the generated TDML file contains correct relative paths
+- [ ] verify parserTestCase object has name, root, model, description
+- [ ] verify that the infoSet and DocumentPart are correct
+- [ ] append to a TDML file by debugging a schema and specifying append and the file to append to
+- [ ] verify the new parserTestCase that was appended to the file
+- [ ] execute a TDML Schema through the debugger
+- [ ] verify that output generated matches original input
+- [ ] make sure that these work cross-platform
+
+## Data Editor
+
+- [ ] open a file (use small and large files of different types)
+ - [ ] verify that the tab is the base name of the file
+ - [ ] verify that the full path name is correct (if the file name is too long, there will be a horizontal scroll bar)
+ - [ ] verify that the Disk Size and Computed Size match the actual size of the file (hover over the values to see the size in bytes)
+ - [ ] verify that the file is loaded in less than 5 seconds regardless of the size of the file
+ - [ ] verify that the detected Content Type is correct for the file
+ - [ ] verify that the header min/max button shows and hides the header
+- [ ] navigation
+ - [ ] scroll through the file, using navigation buttons
+ - [ ] scroll through the file, using the pseudo scrollbar
+ - [ ] scroll through the file, using the mouse wheel
+ - [ ] scroll through the file, using keyboard shortcuts (up, down, page up, page down, home, end)
+- [ ] display modes
+ - [ ] verify changing the address radix (hexadecimal, decimal, and octal) works as expected
+ - [ ] verify changing the display radix (hexadecimal, decimal, octal, and binary) works as expected
+- [ ] profiler
+ - [ ] profile the file using the profile button
+ - [ ] verify that the radix used matches the address radix (hexadecimal/hex, decimal/dec, octal/oct)
+ - [ ] toggle the overlay (ASCII, None)
+ - [ ] toggle the scale (Linear, Logarithmic) and verify that the graph is updated accordingly
+ - [ ] hover over values in the graph to see the value and frequency (and ASCII character if in the printable range)
+ - [ ] green bars are for frequencies that are 1 standard deviation below the mean
+ - [ ] yellow bars are for frequencies that are within 1 standard deviation of the mean
+ - [ ] red bars are for frequencies that are 1 standard deviation above the mean
+ - [ ] change the start offset (use legal, illegal values, negative values, values that are too large, etc.)
+ - [ ] change the end offset (use legal, illegal values, negative values, values that are too large and too small, etc.)
+ - [ ] change the length (use legal, illegal values, negative values, values that are too large and too small, etc.), max length is 10 million bytes
+ - [ ] click the "Profile as CSV" button to download a CSV file of the profile and verify that it's populated correctly
+ - [ ] click somewhere outside of the profiler in the Data Editor tab to close the profiler
+- [ ] seek
+ - [ ] enter offsets in the seek box and use the Seek button and Enter key to navigate to the offset (use legal, illegal values, negative values, upper case and lower case for hexadecimal values, values in the wrong radix, values that are larger than the file size, etc.)
+ - [ ] change the address radix and verify that the seek box is updated accordingly (hexadecimal, decimal, octal)
+ - [ ] enter the file size into the seek box and verify that the screen scrolls to the last page
+- [ ] search
+ - [ ] enter a search term into the search box and use the Search button and Enter key to search for the term (use legal, illegal values with respect to the Edit Encoding)
+ - [ ] use terms that appear in the data and a term that does not appear in the data
+ - [ ] for Latin-1 and ASCII, try case insensitive searches
+ - [ ] verify that the search navigation buttons are enabled/disabled appropriately
+ - [ ] verify that the clear 'x' button in the search box clears the search term
+ - [ ] verify that the all search navigations buttons work as expected, and the search term is highlighted in the logical and physical views correctly with different display radixes
+ - [ ] verify that cancel resets the search and hides the search navigation buttons
+- [ ] replace
+ - [ ] enter a search term into the search box and a replacement term into the replace box and use the Replace button and Enter key to search for the term (use legal, illegal values with respect to the Edit Encoding)
+ - [ ] verify that the search navigation buttons are enabled/disabled appropriately
+ - [ ] verify that the navigation buttons work as expected, and the search term is highlighted in the logical and physical views correctly with different display radixes
+ - [ ] verify that the current highlighted search term is replaced when the Replace button is clicked and that it moves to the next search term if there is one
+ - [ ] verify that the clear 'x' button in the replace box clears the replacement term
+ - [ ] try replace with an empty replacement term to verify that the search term is removed from the data when Replace is clicked
+ - [ ] verify that cancel resets the replace and hides the replace navigation buttons
+ - [ ] verify that Redo, Undo and Revert All buttons work as expected after making replacements and the counts appear correct
+- [ ] single byte editing
+ - [ ] edit a single byte in the data and verify that the change is reflected in the logical and physical views (use legal, illegal values with respect to the Display Radix)
+ - [ ] verify that overwrite, insert backward, insert forward, and delete work as expected when Delete, Insert, and Overwrite Editing Modes are selected in both the Physical and Logical Views
+ - [ ] verify that the values are correct in the Data Inspector and the correct offset with respect to the selected Address Radix
+ - [ ] change the values from the Data Inspector and verify that the changes are reflected in the Data Inspector and the logical and physical views (use legal, illegal values with respect to the Display Radix)
+ - [ ] verify correct values in the Data Inspector when changing the endianness
+ - [ ] verify that all the values are editable by clicking on them and that the cursor is placed in the edit box with the current value ready to make changes
+ - [ ] verify that if the value is changed to a legal value different than the original value, the value is updated in the logical and physical views
+ - [ ] verify that the "X" button in the Data Inspector clears the value and makes no changes to the data
+ - [ ] verify that Redo, Undo and Revert All buttons work as expected after making edits and the counts appear correct
+ - [ ] change the Editing mode to "Overwrite Only" and verify that the insert and delete buttons are hidden
+ - [ ] change the Editing mode to "Delete, Insert, and Overwrite" and verify that the overwrite and delete buttons are available
+- [ ] multi-byte editing
+ - [ ] click and drag to highlight a range of bytes and verify that the range is highlighted in the logical and physical views and that the Data Inspector now becomes an editable text box
+ - [ ] verify that the highlighted data appears in the editable text box and in the correct Edit Encoding
+ - [ ] verify that the Selection range and size are correct and that the range matches the selected Address Radix
+ - [ ] verify that the highlighted data remains correct when scrolling (if the navigation hasn't been locked)
+ - [ ] verify that edits can be made with respect to the selected Edit Encoding (use legal, illegal values with respect to the Edit Encoding)
+ - [ ] verify that when a legal change is made that the Commit button is activated
+ - [ ] verify that data in the edit box can be selected and copied
+ - [ ] verify that data can be pasted in the edit box
+ - [ ] verify that when changing the selected data back to the original value that the Commit button is deactivated (there is no change to commit)
+ - [ ] verify that when an illegal change is made that the Commit button is deactivated and an appropriate error message is displayed
+ - [ ] verify that when the Commit button is clicked that the changes are reflected in the logical and physical views
+ - [ ] verify that Redo, Undo and Revert All buttons work as expected after committing changes and the counts appear correct
+ - [ ] verify that changes in the length of the data are reflected in the Computed Size (hover over the Computed Size to see the tooltip with the byte size)
+ - [ ] verify that clicking the 'X' button in the Edit box clears the selection, hides the Edit box, and displays the Data Inspector
+- [ ] server heartbeat information
+ - [ ] verify that the port the server is on matches the one configured (9000 is the default port)
+ - [ ] verify that the server heartbeat information is displayed in the footer when hovering over the dot
+ - [ ] ensure the latency is less than 30ms
+ - [ ] open another Data Editor tab on a different file and verify that the Session Count is incremented
+ - [ ] attempt to open the same file as one that is being edited and verify that opening the session fails and the Session Count is not incremented, then close this tab
+ - [ ] close the other Data Editor tab and verify that the Session Count is decremented
+- [ ] screen size
+ - [ ] verify that when the screen is is large enough that the buttons have icons and text
+ - [ ] verify that when the screen is smaller, that the buttons have icons only, with tooltips
+ - [ ] verify that regardless of button size, the case sensitivity "Aa" button in the search box always has a tooltip
+ - [ ] click the "Profile" button and verify that the "Profile as CSV" button matches the rest of the buttons
+- [ ] saving
+ - [ ] verify that the file can be saved and that the changes are reflected in the file
+ - [ ] verify that the file can be saved as a different file name and that the changes are reflected in the new file
+- [ ] themes
+ - [ ] verify that when changing to/from light/dark themes (command palette `Preferences: Toggle between Light/Dark Themes`) data editor is updated with the matching theme and that everything is readable
+- [ ] final checks
+ - [ ] verify that search works on changed data
+ - [ ] profile the changed data to verify that changes made are reflected in the profile
+ - [ ] verify that all changes can be undone using the Revert All button
+ - [ ] verify that closing the last Data Editor tab shuts down the Ωedit server (check the process table searching for omega)
+ - [ ] check the log files for errors (e.g., dataEditor-9000.log and serv-9000.log in the OS-specific log directory for macOS it's ~/Library/Application Support/omega_edit/)
+
+## Packaging
+
+- [ ] verify that the packaging script works on all platforms
+- [ ] verify that packages are up-to-date and have no known vulnerabilities
+
+## Documentation
+
+- [ ] version specific documentation is complete and up-to-date (https://github.com/apache/daffodil-vscode/wiki)